Aqua-Soft Forums

Millions Of Page Faults In Ad, Od And Yz

Many of you docklet developers may have wondered why and how you got millions of page faults in your Task Manager for YzDock, ObjectDock and AveDesk, especially if you run them for a longer period of time, e.g. sending your PC to hibernate instead of rebooting it each morning. (+150 /sec.)

Was it a resource leak in your Docklet? Did you allocate tons of strings in a wasteful manner so that the VC++ heap manager got fragmented?

Neither. It's a flaw in the SDK each three are suffering from the first beginning.

See the following OD-SDK code excerpt:

BOOL DockletIsVisible(HWND hwndDocklet)
{
	typedef BOOL(__stdcall *DUMMY_TYPEDEF)(HWND hwndDocklet);
	DUMMY_TYPEDEF HostDockletIsVisible = (DUMMY_TYPEDEF) GetProcAddress(GetModuleHandle(NULL), "DockletIsVisible");
	if(!HostDockletIsVisible)
  return FALSE;

	return HostDockletIsVisible(hwndDocklet);
}

Whenever you call DockletIsVisible() in your Docklet, this code loads the discardable portion of ObjectDock.exe, maps the DockletIsVisible function pointer into a temporary variable and calls the function. Each of these calls will produce a page fault, because the Win32 Loader will assume that the procedure address info is only needed once and - ignoring how many RAM you have - silently page out this portion of the exe into the pagefile.sys in your boot partition.

Since programmers are a lazy species, this applies unfortunately to the way your exported functions are called, too. Whenever any of your exports need to get called, the preload / discardable marked export section gets read from the hard disk again and again. Imagine, you have a WM_TIMER message to handle - OnProcessMessage() gets shifted from the pagefile to the RAM each time the message arrives. On my 1GHz PC, my HD LED blinks every second if I have only the CPU-Monitor Docklet on either YZ, OD or AD.

Finally, it is a stupid waste.

Exported functions can't go away between two calls since they are cemented at compile time - dumbly looking them up each time consumes unnecessary CPU time.

Ideas on how to remedy:

1. Create your own window for handling timer messages in your timer based docklets.

2. Don't export empty routine stubs.

3. Insist on the authors to fix this.

4. Research the possibility of how to force your export table to remain in RAM (YZ)

5. In the case of OD, I would recommend the attached header file for calling the Docklet API. It also keeps the window handle.

Have a nice undisturbed coding experience,

herd

DockletAPI.h

Attachment: DockletAPI.h

#114640

this has to be one of the most helpful, and least browsed threads ever. What a shame.

I salute your diligence and work, herd

thank you VERY much for the effort.

I hope the lack of replies doesnt hinder your future assistance

#117772

Originally posted by Duckie@Feb 22 2004, 03:42 AM

this has to be one of the most helpful, and least browsed threads ever.  What a shame.

I salute your diligence and work, herd

thank you VERY much for the effort.

I hope the lack of replies doesnt hinder your future assistance

totally agree, someone who could give some developers advice. hehe, wow you will go far on this website with threads like this one. what a patient man you are for taking the time to write that up, it was short and sweet [ to the point ].

thx.

-bunkka :canadian:

#117814

GREAT POST!!!!

It's excellent to see coders helping developers perfect their apps by pointing out memory leaks. Let's hope the devs see this and fix the problems.

:bows: to herd.

#117867

Thanks for your nice comments - I'll keep on posting technical stuff. Knowledge is for sharing and somehow I knew that this would be the right board for it...

As an update - AndreasV was the first developer to pick that up and had the aforementioned causes for page faults fixed in AveDesk.

Surprisingly to me, that reduced only a small fraction of the page faults - the real gross according to Andreas came from the GDI+ library function for drawing images and extracting a GDI HBITMAP from an GDI+ Bitmap class. It seems to be a MS introduced problem, though. I apologize if I caused any disturbance.

Finally, GDI+ was made as the underlying backend for the dotNet System.Drawing classes. I found only one Google hit about the subject, a C-Sharp developer who unheardly complained about the same problem... After all, considering that the dotNet runtime loads 20 Megabytes of assemblies for a simple "Hello World" program, 50 page faults per Graphics::DrawImage don't seem much. And if you let draw, say 20 images per second in all your docklets on your dock, you'll be a page fault millionairy in a few hours, too.

Somehow MS has changed its view about resource usage in the past years: Under NT 3.51 and later, causing unnecessary page faults was considered evil, having the slow performance of the early 90es hard drives in mind. Nowadays, Explorer.exe causes exactly one page fault per second for no other reason than measuring time or what? I would think that today, energy saving is issue enough to allow the HD to spin down and keep it off as long as possible. Perhaps 2000 and XP have evolved to a point where they could satisfy these kind of page faults from a memory cache? After 15 minutes idle time on OD, my HD LED does not blink anymore, yet the page faults still rise...

Windows - Worlds most sold graphics adventure game yet no one has solved.

#117905

Thanks again man for this great technical review :)

#117913

Whoops, I missed your post here, luckily I saw it at DockEx. I have posted a follow-up note at dockex that I will post here too:

------------------------------------------------------------------------------------------

Regarding herds posts about 'Millions of pagefaults' I can say that most of the pagefaults are caused by using GDI+. Herd and I have been investigating this and while storing all function pointers (using AveDesk1.1 beta), large number of pagefaults where still occuring, but only at redraw.

By removing the actuall paint-routines, the pagefaults stopped occuring.Apprently, GDI+ does some magic behind the GDI+-scenes with the virtual memory.

Using

Gdiplus::Bitmap bmp(width,height);
Gdiplus::Graphics g(&bmp);
g.DrawImage(&someOtherBitmap,0,0,someWidth,someHeight);

I got some extra 50 to 100 pagefaults per call.

Also, calls to Gdiplus::Bitmap::GetHBITMAP() created more pagefaults.

There was no difference in creating those objects on the stack or the heap.

#117921

Hmm, isn't HBITMAP a public member of the Image (and of course Bitmap) class ?

You should be able to directly access it I guess ?

#118010

According to the MS GDI+ documentation, nor Gdiplus::Image nor Gdiplus::Bitmap does have public fields.

#118382

Programmers do have their own language don't they??? ;)

#118385

Originally posted by Sid@Feb 23 2004, 12:46 PM

Programmers do have their own language don't they??? ;)

and you thought the dilbert comic was joking.

#118390

...I'll go on pretending i understood all of that. ;)

#118693

I've written a little test-application to see if it is really GDI+ that is responsible for the page faults. Some interesting results showed up.

Used:

VC++ 6.0

GDIPlus libs/header files version 1

Windows XP Taskmanager to check the number of pagefaults.

First, I created a big loop where GDI+ objects were created on the Heap. No

increase in pagefaults!

for(int i = 0; i<100000; i++)
{
  Gdiplus::Bitmap* bmp = new Gdiplus::Bitmap(400,400);
  Gdiplus::Graphics* g = new Gdiplus::Graphics(bmp);
  delete g;
  g = 0;
  delete bmp;
  bmp = 0;

  Sleep(50);   // SLEEPER
}

The line marked with SLEEPER could be commented out, it made no difference. Also creating the objects on the stack didn't result in large number of page faults.

Second try, drawing an image loaded from a file.

Gdiplus::Bitmap* arrow = new Gdiplus::Bitmap(L"c:\\arrow.png");

for(int i = 0; i<100000; i++)
{
  Gdiplus::Bitmap* bmp = new Gdiplus::Bitmap(400,400);
  Gdiplus::Graphics* g = new Gdiplus::Graphics(bmp);
  g->DrawImage(arrow,0,0);
  delete g;
  g = 0;
  delete bmp;
  bmp = 0;

  Sleep(50);   // SLEEPER
}
delete arrow;
arrow = 0;

Again, the line marked with SLEEPER could be commented out, it made no difference. Also creating the objects on the stack didn't result in large number of page faults.

Again, no extra page faults! It looked like DrawImage isn't responsible for the extra page faults, but when I changed g->DrawImage(arrow,0,0); into g->DrawImage(arrow,400,400); page faults were occuring with 1,000 at a time, but suddenly, when around 6,000 page faults was reached, the number did not increase anymore and CPU usage dropped. It looks like GDI+ caches bitmaps that are resized.

Using the following two test-applications, I could get the number of page-faults constantly going up. Changing the drawing-size of every draw, made the page-fault go up with approx. 5,000 a second which results in 5,000 / 20 = 250 page faults per draw (app 1). It looks like there is a limit of the number of objects GDI+can cache, because in app 2 using only 3 images (which also loads the same image) and drawing them with the same scaling everytime, page faults still occur with 5,000 a second.

As a little side-effect, giving the focus to another application made the number of page faults per second double. I guess this is because I don't created a window and therefore Windows acts like the application was minimized, swapping everything to the Virtual Memory.

// GDI_TEST 1 Application
#include <windows.h>
#include <gdiplus.h>
#pragma comment(lib,"gdiplus.lib")

int WINAPI WinMain(  HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR
lpCmdLine,  int nCmdShow)
{
  Gdiplus::GdiplusStartupInput g_gdiplusStartupInput;
  Gdiplus::ULONG_PTR g_gdiplusToken;
  Gdiplus::GdiplusStartup(&g_gdiplusToken, &g_gdiplusStartupInput, 0);

  Gdiplus::Bitmap* arrow = new Gdiplus::Bitmap(L"c:\\arrow.png");

  MessageBox(0,"Start",0,0);
  for(int i = 0; i<100000; i++)
  {
    Gdiplus::Bitmap bmp(400,400);
    Gdiplus::Graphics g(&bmp);

    if(i%3 == 0)
      g.DrawImage(arrow,0,0,100,100);
    else if ( i %3 == 1)
      g.DrawImage(arrow,0,0,200,200);
    else
      g.DrawImage(arrow,0,0,300,600); 

    Sleep(50);
  }

  delete arrow;
  arrow = 0;

  MessageBox(0,"End",0,0);

  Gdiplus::GdiplusShutdown(g_gdiplusToken);

  return 0;
}

// GDI_TEST 2 Application
#include <windows.h>
#include <gdiplus.h>
#pragma comment(lib,"gdiplus.lib")

int WINAPI WinMain(  HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR
lpCmdLine,  int nCmdShow)
{
  Gdiplus::GdiplusStartupInput g_gdiplusStartupInput;
  Gdiplus::ULONG_PTR g_gdiplusToken;
  Gdiplus::GdiplusStartup(&g_gdiplusToken, &g_gdiplusStartupInput, 0);

  Gdiplus::Bitmap* arrow = new Gdiplus::Bitmap(L"c:\\arrow.png");
  Gdiplus::Bitmap* arrow2 = new Gdiplus::Bitmap(L"c:\\arrow.png");
  Gdiplus::Bitmap* arrow3 = new Gdiplus::Bitmap(L"c:\\arrow.png");

  MessageBox(0,"Start",0,0);
  for(int i = 0; i<100000; i++)
  {
    Gdiplus::Bitmap bmp(400,400);
    Gdiplus::Graphics g(&bmp);

    if(i%3 == 0)
      g.DrawImage(arrow,0,0,100,100);
    else if ( i %3 == 1)
      g.DrawImage(arrow2,0,0,100,100);
    else
      g.DrawImage(arrow3,0,0,100,100);

    Sleep(50);
  }

  delete arrow;
  arrow = 0;
  delete arrow2;
  arrow2 = 0;
  delete arrow3;
  arrow3 = 0;

  MessageBox(0,"End",0,0);

  Gdiplus::GdiplusShutdown(g_gdiplusToken);

  return 0;
}

Conclusion: Scaling (and probably transforming) images result in page-faults, but only if you draw multiple images with the same scaling or the same images with different scaling.

Possible solution: When setting an image, automatically scale it once to the prefered size in a seperate buffer.

Ofcourse, I did not test all possibilites ( bmp outside the loop, no if/else construction), but I think the page fault problem is more clear now, at least to me it is.

#125832

So do you think that Microsoft will fix this problem in Windows Codename: Longhorn?

#125841

Originally posted by Jimmy S@Mar 8 2004, 12:53 PM

So do you think that Microsoft will fix this problem in Windows Codename: Longhorn?

Yes - they're completely replacing GDI+ with Avalon, so all the GDI+ bugs will be replaced with DX10/11 ones :D

#125843

So do you think that Microsoft will fix this problem in Windows Codename: Longhorn?

It seems Microsoft changed their policy about page faults as herd already said in this thread. They don't consider page faults as bad anymore.

Yes - they're completely replacing GDI+ with Avalon, so all the GDI+ bugs will be replaced with DX10/11 ones 

Actually, Microsoft "promised" that the next version of GDI+ will be hardware accelerated, so I can see some mix up with DX and GDI+ in the future (think of longhorn).

Furthermore, this is not a bug, but a feature :lol:

#125846

Originally posted by AndreasV@Mar 8 2004, 01:00 PM

[...]

Actually, Microsoft "promised" that the next version of GDI+  will be hardware accelerated, so I can see some mix up with DX and GDI+ in the future (think of longhorn).

Furthermore, this is not a bug, but a feature  :lol:

The next version of GDI+ (Avalon) is hardware accelerated, but it uses DXWhatever under the covers.

Basically, it's DXWhatever for GDI+ programmers ;)

But you can use it like GDI+ or like DX, if you so desire :P

#125854