Thoroughly understand memory leaks, have you learned it?

Hello everyone, I am Brother Xiaofeng. Today I will talk to you about the topic of memory leaks.

When talking about memory application in these articles, I like to use the parking lot analogy. Memory application is like going to the parking lot to find a parking space. After you find the parking space, you can park your car here.

From this analogy, what is a memory leak? Memory leaks appear to be caused by vehicles in the parking lot that can only enter but not exit, resulting in the eventual inability to find a parking space. From a programmer's perspective, it means that the memory is only applied for and not released. If you ask, many people may think that this is the case with memory leaks. Something happened.

However, this is actually not comprehensive.

Requesting too much memory

First of all, only applying for memory but not releasing it does not necessarily mean a memory leak. It may be that your program does need to apply for a lot of memory, which is normal. However, if a bug causes a lot of memory to be applied for, this is a memory leak, or some people call it a memory leak. It is a space leak, which means that the memory requested exceeds the normal requirement; whether it is intentional or not, in this case you still maintain references to these memories, so you can always find these memories and delete them. It depends on whether you delete them. Don't delete.

There are many situations that can cause this problem, such as a reused structure/object that does not clean up the data left over from the last use when it is reused, a cache exists in the system, but the cache expiration policy is not set appropriately, etc.

Memory cannot be deleted

Another interesting type of memory leak is when you allocate some memory, but end up with nothing pointing to it:

void memory_leak() {
  char* mem = (char*)malloc(1024);
  // just return
}
1.
2.
3.
4.

In this code, we apply for 1k of memory, but when the memory_leak function returns, you no longer know where this memory is!

Using the example of a parking lot, some drivers are too rich and have so many cars at home that they forget to put their cars in the parking lot. As a result, no one will drive these cars away at all, so parking is wasted. parking spaces, resulting in fewer and fewer parking spaces available, and for programming, careless programmers eventually "forgot" after applying for some memory, and there will never be anything (variables/pointers) pointing to these memories, so In this case you have no way to find the memory and delete it.

memory fragmentation

This can also be regarded as a special type of memory leak. Using the example of a parking lot, a small elderly scooter is parked between two parking spaces. As a result, although the remaining space of the two parking spaces is large enough, it is just right. There is no way to park another car.

Assume that the precious memory size in our system is only 8 bytes, and two bytes have been allocated, like this:

picture

Now, the free memory in the system is 6 bytes, and the next memory request needs to allocate 5 bytes. Oops, we have no way to find a continuous 5-byte memory space, although the entire space is still available. 6 bytes, this is the so-called memory fragmentation problem.

For the memory allocator, if this happens, it will have to use the help of the operating system to expand the heap area, so it seems that our program occupies more and more memory, although in fact the program may not need that much Memory, just because of memory fragmentation, part of the memory cannot be used again.

However, for modern operating systems, especially systems with virtual memory capabilities, the memory fragmentation problem may not be as serious as we think. The reason is that the allocated memory only needs to be continuous in the virtual address space and not necessarily in the physical memory. It is also continuous. Suppose we need to store a string like "aabbccdd" in the virtual memory address space. In the virtual address space, it looks like this:

picture

But it may be stored in physical memory like this:

picture

It can be seen that by using virtual memory, we can more fully and flexibly utilize the "edge and corner" physical memory, thereby reducing the impact of memory fragmentation.

For a more detailed explanation of virtual memory, you can refer to the chapter on virtual memory in "In-depth Understanding of Operating Systems". Regarding the public account "Coder's Desert Island Survival", just reply "Operating System".

If your program needs to repeatedly apply for many objects/data/structures and release them all at once at the end, then the memory pool is a good choice to avoid memory fragmentation. The principle is that although there will be fragmentation from the perspective of the memory pool, when When we apply to release memory from the heap area in units of the memory pool size, this fragmentation will no longer exist.

Problems caused by memory leaks

In modern operating systems, unless your program runs long enough or the memory you apply for is fast enough, memory leaks may not be a big problem. You may not even notice that there is a memory leak, because it occupies the space after the process ends. The memory will be reclaimed by the operating system. In this case, you may not be too concerned about this problem, but for long-running server-side programs, database programs, operating systems, etc., memory leaks are a serious problem, because these The program must be online at all times, and any small memory leak will be very obvious over time.

What happens if memory leaks continuously?

It's possible that your system will be so slow that it explodes.

The speed of memory application will have a great impact on system performance. When the system memory is insufficient, it will be more difficult and time-consuming for the memory allocator to find a free memory block that meets the requirements. When the memory consumed by the program exceeds the physical memory size, the virtual The memory system (if there is one) comes into play and swaps out a part of the process address space that is not commonly used. At this time, the system performance will drop rapidly, and the programmer will slow down and freeze.

Of course, depending on the system configuration, like Linux systems, processes that consume a lot of memory may be killed. This is the Out of Memory killer, or oom killer for short.

Memory leak detection tool

Memory leaks are usually difficult to troubleshoot directly, especially for C/C++ programs. At this time we will have to use the necessary tools.

There are specialized tools that can help you detect memory leaks, such as Valgrind, AddressSanitizer, and MemorySanitizer. These tools can inspect programs at runtime and identify memory leaks and other memory errors.

In addition, for specific memory allocators, such as jemalloc, these memory allocators come with the memory detection tool heap profile, which can effectively analyze where the process memory is allocated and refine it to the function level, which is very convenient.