Software: Apache/2.0.54 (Fedora). PHP/5.0.4 

uname -a: Linux mina-info.me 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:57:02 EDT 2006 i686 

uid=48(apache) gid=48(apache) groups=48(apache)
context=system_u:system_r:httpd_sys_script_t 

Safe-mode: OFF (not secure)

/usr/share/doc/valgrind-2.4.0/   drwxr-xr-x
Free 3.84 GB of 27.03 GB (14.2%)
                            Encoder    Tools    Proc.    FTP brute    Sec.    SQL    PHP-code    Update    Feedback    Self remove    Logout    

Viewing file:     ms_main.html (13.85 KB)      -rw-r--r--

7  Massif: a heap profiler

7.1  Heap profiling

• Heap blocks;
• Heap administration blocks;
• Stack sizes.

• It can speed up your program -- a smaller program will interact better with your machine's caches, avoid paging, and so on.
• If your program uses lots of memory, it will reduce the chance that it exhausts your machine's swap space.

7.2  Why Use a Heap Profiler?

But the story is different for heap profilers. Some programming languages, particularly lazy functional languages like Haskell, have quite sophisticated heap profilers. But there are few tools as powerful for profiling C and C++ programs.

Why is this? Maybe it's because C and C++ programmers must think that they know where the memory is being allocated. After all, you can see all the calls to malloc() and new and new[], right? But, in a big program, do you really know which heap allocations are being executed, how many times, and how large each allocation is? Can you give even a vague estimate of the memory footprint for your program? Do you know this for all the libraries your program uses? What about administration bytes required by the heap allocator to track heap blocks -- have you thought about them? What about the stack? If you are unsure about any of these things, maybe you should think about heap profiling.

Massif can tell you these things.

Or maybe it's because it's relatively easy to add basic heap profiling functionality into a program, to tell you how many bytes you have allocated for certain objects, or similar. But this information might only be simple like total counts for the whole program's execution. What about space usage at different points in the program's execution, for example? And reimplementing heap profiling code for each project is a pain.

Massif can save you this effort.

7.3  Overview

Then, run your program with valgrind --tool=massif in front of the normal command line invocation. When the program finishes, Massif will print summary space statistics. It also creates a graph representing the program's heap usage in a file called massif.pid.ps, which can be read by any PostScript viewer, such as Ghostview.

It also puts detailed information about heap consumption in a file file massif.pid.txt (text format) or massif.pid.html (HTML format), where pid is the program's process id.

7.4  Basic Results of Profiling

valgrind --tool=massif prog

The program will execute (slowly). Upon completion, summary statistics that look like this will be printed:

==27519== Total spacetime:   2,258,106 ms.B
==27519== heap:              24.0%
==27519== heap admin:         2.2%
==27519== stack(s):          73.7%

Which entries you see in the breakdown depends on the command line options given. The above example measures all the possible parts of memory:

• Heap: number of words allocated on the heap, via malloc(), new and new[].

• Heap admin: each heap block allocated requires some administration data, which lets the allocator track certain things about the block. It is easy to forget about this, and if your program allocates lots of small blocks, it can add up. This value is an estimate of the space required for this administration data.

• Stack(s): the spacetime used by the programs' stack(s). (Threaded programs can have multiple stacks.) This includes signal handler stacks.

7.5  Spacetime Graphs

Massif uses a program called hp2ps to convert the raw data into the PostScript graph. It's distributed with Massif, but came originally from the Glasgow Haskell Compiler. You shouldn't need to worry about this at all. However, if the graph creation fails for any reason, Massif tell you, and will leave behind a file named massif.pid.hp, containing the raw heap profiling data.

Here's an example graph:

The graph is broken into several bands. Most bands represent a single line of your program that does some heap allocation; each such band represents all the allocations and deallocations done from that line. Up to twenty bands are shown; less significant allocation sites are merged into "other" and/or "OTHER" bands. The accompanying text/HTML file produced by Massif has more detail about these heap allocation bands. Then there are single bands for the stack(s) and heap admin bytes.

Note: it's the height of a band that's important. Don't let the ups and downs caused by other bands confuse you. For example, the read_alias_file band in the example has the same height all the time it's in existence.

The triangles on the x-axis show each point at which a memory census was taken. These aren't necessarily evenly spread; Massif only takes a census when memory is allocated or deallocated. The time on the x-axis is wallclock time, which is not ideal because you can get different graphs for different executions of the same program, due to random OS delays. But it's not too bad, and it becomes less of a problem the longer a program runs.

Massif takes censuses at an appropriate timescale; censuses take place less frequently as the program runs for longer. There is no point having more than 100-200 censuses on a single graph.

The graphs give a good overview of where your program's space use comes from, and how that varies over time. The accompanying text/HTML file gives a lot more information about heap use.

7.6  Details of Heap Allocations

Here's part of the information that accompanies the above graph.

Called from:

• 22.1%: 0x401767D0: _nl_intern_locale_data (in /lib/i686/libc-2.3.2.so)
• 8.6%: 0x4017C393: read_alias_file (in /lib/i686/libc-2.3.2.so)
• (several entries omitted)
• and 6 other insignificant places

That alone can be useful, but often isn't enough. What if one of these functions was called from several different places in the program? Which one of these is responsible for most of the memory used? For _nl_intern_locale_data(), this question is answered by clicking on the 22.1% link, which takes us to the following part of the file.

== 1 ===========================
Context accounted for 22.1% of measured spacetime
  0x401767D0: _nl_intern_locale_data (in /lib/i686/libc-2.3.2.so)

Called from:

• 22.1%: 0x40176F95: _nl_load_locale_from_archive (in /lib/i686/libc-2.3.2.so)

== 2 ===========================
Context accounted for 22.1% of measured spacetime
  0x401767D0: _nl_intern_locale_data (in /lib/i686/libc-2.3.2.so)
  0x40176F95: _nl_load_locale_from_archive (in /lib/i686/libc-2.3.2.so)

Called from:

• 22.1%: 0x40176184: _nl_find_locale (in /lib/i686/libc-2.3.2.so)

Sometimes you will get a code location like this:

• 30.8% : 0xFFFFFFFF: ???

Massif produces this information in a plain text file by default, or HTML with the --format=html option. The plain text version obviously doesn't have the links, but a similar effect can be achieved by searching on the code addresses. (In Vim, the '*' and '#' searches are ideal for this.)

7.7  Massif options

• --heap=no
--heap=yes [default]
  When enabled, profile heap usage in detail. Without it, the massif.pid.txt or massif.pid.html will be very short.

• --heap-admin=n [default: 8]
  The number of admin bytes per block to use. This can only be an estimate of the average, since it may vary. The allocator used by glibc requires somewhere between 4--15 bytes per block, depending on various factors. It also requires admin space for freed blocks, although Massif does not count this.

• --stacks=no
--stacks=yes [default]
  When enabled, include stack(s) in the profile. Threaded programs can have multiple stacks.

• --depth=n [default: 3]
  Depth of call chains to present in the detailed heap information. Increasing it will give more information, but Massif will run the program more slowly, using more memory, and produce a bigger .txt/.hp file.

• --alloc-fn=name
  Specify a function that allocates memory. This is useful for functions that are wrappers to malloc(), which can fill up the context information uselessly (and give very uninformative bands on the graph). Functions specified will be ignored in contexts, i.e. treated as though they were malloc(). This option can be specified multiple times on the command line, to name multiple functions.

• --format=text [default]
  --format=html
  Produce the detailed heap information in text or HTML format. The file suffix used will be either .txt or .html.

7.8  Accuracy

The heap admin spacetime figure is an approximation, as described above. If anyone knows how to improve its accuracy, please let us know.