Dmitry Vostokov, 20th September 2009 (revised 27th November 2009)
http://www.dumpanalysis.org

Sometimes we need to know whether a function was called or not. Traditional non-invasive approach without setting and triggering debugger breakpoints is to use diagnostic software tracing and record a message when program execution enters a function. The author applied the method of colorimetric computer memory dating^[10] to record the function prolog entrance. The idea is to allocate a static buffer for every function we want to trace and fill it with a characteristic RGB patten upon entrance. Here is the sample code created for testing purposes:

#include "stdafx.h"

typedef unsigned int ISOMEMOTOPE;

#define COLORED_STACK_SIZE 0x35000

#define RED   0x00FF0000
#define GREEN 0x0000FF00
#define BLUE  0x000000FF

#define COLORED_PROLOG(rgbaValue) { \
static ISOMEMOTOPE \
coloredStack[COLORED_STACK_SIZE]; \
for (int i = 0; i < COLORED_STACK_SIZE; ++i) \
coloredStack[i] = (rgbaValue); }

#define SLEEP_TIME   10*1000       

void fooR()
{
       COLORED_PROLOG(RED);
}


void fooG()
{
       COLORED_PROLOG(GREEN);
}

void fooB()
{
       COLORED_PROLOG(BLUE);
}

void barP(void *pData)
{
       COLORED_PROLOG((ISOMEMOTOPE)pData);
}

void thread_second(void *)
{
       barP((void *)0x00FFFF00);
       Sleep(SLEEP_TIME);
}

int _tmain(int argc, _TCHAR* argv[])
{
       puts("Time to dump: 1\n");
       Sleep(SLEEP_TIME);
       fooR();
       puts("Time to dump: 2\n");
       Sleep(SLEEP_TIME);
       fooB();
       puts("Time to dump: 3\n");
       Sleep(SLEEP_TIME);
       barP((void *)-1);
       puts("Time to dump: 4\n");
       Sleep(SLEEP_TIME);
       barP(NULL);
       puts("Time to dump: 5\n");
       Sleep(SLEEP_TIME);

       fooG();
       _beginthread(thread_second, 0, NULL);
       Sleep(SLEEP_TIME/10);
       puts("Time to dump: 6\n");
       Sleep(SLEEP_TIME);

       return 0;
}

We also save a process memory dump each time the running program prints “Time to dump” message (we used Windows Vista Task Manager):

Then Dump2Picture^[11] tool was used to convert resulting memory dumps into bitmap files. The first process dump shows large preallocated zero-filled coloredStack arrays grouped together into big black region in the center:

The second process dump was saved just after fooR function call that filled one such static array with a red pattern:

The third process memory dump was saved after fooB function call that filled another static array with a blue pattern:

The 4th memory dump was saved after fooP function call that had its fill pattern as a function parameter, in our case, -1, RGB (255, 255, 255) or white color:

The 5th process dump was saved after fooP function was called with another fill parameter, this time 0, and it cleared white region:

The 6th memory dump was saved after fooG function call that filled another memory buffer with a green color pattern and the second thread started calling fooP function with a yellow color fill pattern:

It is also possible to extend this approach to record different colors, for example, RGB (0,0,1), RGB (0,0,2), RGB (0,0,3), ... to show visually how often a function was called.

[10] http://www.dumpanalysis.org/blog/index.php/2008/04/16/computer-colorimet...
[11] http://www.dumpanalysis.org/blog/index.php/2007/08/04/visualizing-memory...