Memory Leak Detection in Embedded Systems

One of the problems with developing embedded systems is the detection of memory leaks; I‘ve found three tools that are useful for this. These tools are used to detect application program errors, not kernel memory leaks. Two of these tools (mtrace and dmalloc) are part of the MontaVista Linux Professional Edition 2.1 product. The other (memwatch) is available from the Web (see Resources).

C and C++ programmers control dynamic memory allocation. Reckless use of this control can lead to memory management problems, which cause performance degradation, unpredictable execution or crashes.

Some of the problems that cause memory leaks are writing or reading beyond an allocated memory segment or trying to free memory that has already been freed. A memory leak occurs when memory is allocated and not freed after use, or when the pointer to a memory allocation is deleted, rendering the memory no longer usable. Memory leaks degrade performance due to increased paging, and over time, cause a program to run out of memory and crash. Access errors lead to data corruption, which causes a program to behave incorrectly or crash. When a program runs out of memory it also can cause the Linux kernel to crash.

Designing and programming an embedded application requires great care. The application must be robust enough to handle every possible error that can occur; care should be taken to anticipate these errors and handle them accordingly—especially in the area of memory. Often an application can run for some time before it mysteriously crashes itself or the system as a result of a memory allocation that is never freed. Finding these errors can be done through use of memory leak detectors.

These tools work by replacing malloc, free and other memory management calls. Each tool has code that intercepts calls to malloc (and other functions) and sets up tracking information for each memory request. Some tools implement memory protection fences to catch errant memory accesses.

Some of the leak detection programs are very large and require a virtual memory image of the program being searched. This requirement makes it very difficult to use on embedded systems. However, mtrace, memwatch and dmalloc are simple programs that find most errors.

All three tools were run on one example C program containing common memory handling errors. This program, together with Makefiles for building it with the three tools, is available as a downloadable file atftp.linuxjournal.com/pub/lj/listings/issue101/6059.tgz. All of these tools have been used in several different target architectures. The example code will work whether compiled natively or cross-compiled.

The simplest of the three tools is mtrace. A feature of the GNU C library, mtrace allows detection of memory leaks caused by unbalanced malloc/free calls. It is implemented as a function call, mtrace(), which turns on tracing and creates a log file of addresses malloc‘d and freed. A Perl script, also called mtrace, displays the log file, listing only the unbalanced combinations and—if the source file is available—the line number of the source where the malloc occurred. The tool can be used to check both C and C++ programs under Linux. One of the features that makes mtrace desirable is the fact that it is scalable. It can be used to do overall program debugging but can be scaled to work on a module basis as well.

Key to the use of the mtrace feature are three items: include mcheck.h, set the MALLOC_TRACE environment variable and call the mtrace() function call. If the MALLOC_TRACE variable is not set, mtrace() does nothing.

The output of mtrace includes messages such as:

- 0x0804a0f8 Free 13 was never alloc‘d
/memory_leak/memory_leaks/mtrace/my_test.c:193

to indicate memory that was freed but never malloc‘d and a “Memory not freed” section that includes the address, size and line number of calls to malloc for which no free occurred.

memwatch is a program that not only detects malloc and free errors but also fencepost conditions. Fencepost conditions occur when writing data into an allocated chunk of memory (allocated by malloc) and the data goes beyond the end of the allocated area. Some things that memwatch does not catch are writing to an address that has been freed and reading data from outside the allocated memory.

The heart of memwatch is the memwatch.c file. It implements the wrappers and code for the address checking. To use memwatch the file memwatch.h must be included in the source. The variables MEMWATCH and MW_STDIO must be defined on the compile command line (-DMEMWATCH and -DMW_STDIO). The memwatch.c file must be used with the application as well. The object module from the compile of memwatch.c must be included in the link of the application. Upon execution of the application, a message will appear on stdout if memwatch found any abnormalities. The file memwatch.log is created that contains the information about the errors encountered. Each error message contains the line number and source-code filename where the error occurred.

Comparing memwatch.log with the log from mtrace, the same errors are reported. The memwatch tool also found a fencepost condition where the memory addresses were changed to overwrite the start and end of an allocated area, showing the expanded capability of memwatch in this case. The disadvantage is that memwatch is not scalable. It has to run on the whole application.

export DMALLOC_OPTIONS=debug=0x44a40503,inter=1,log=logfile

 

Cal Erickson (cal_erickson@mvista.com) currently works for MontaVista Software as a senior Linux consultant. Prior to joining MontaVista, he was a senior support engineer at Mentor Graphics Embedded Software Division. Cal has been in the computing industry for over 30 years with experience at computer manufacturers and end-user development environments.