Technical question

[shiznitz]

What, exactly, is a memory leak? Use terms a non-tech person might understand. Is it memory that gets used and never emptied for re-use?

[Rasix]

When a program runs, certain data structures require memory and the program will allocate memory for this.  After some time the memory that was allocated is no longer needed for that task, and the memory is reclaimed (this is called garbage collection).  When this memory is not reclaimed, it's just lost.  

Often the task that leaves this memory behind is something that's going to be run multiple times over the course of the program.  So, you keep getting this memory that's just sitting in limbo, not being used, but no longer available and it's building up over time.  As you can see, this would lead to problems, including not having enough RAM, and thus having to use disk cache.  

I think that's about as non technical as I can make it (of course, I might be slightly off, I haven't taken a core CS class in 3 years).

[SirBruce]

It means different things depending on the context, but you've got it basically right.  When most people say a game has a memory leak, what they mean is that, over time, the game is, unintentionally, consuming more and more memory, until the point it slows down, you run out of RAM and it REALLY slows down, etc.

Everything in a computer program consumes memory.  Whenever the program no longer needs something in memory (say, you finish a level in a game and it is loading the next level), the program has to basically has to give up all that memory it was using, and start using memory for the new level.  A memory leak happens when a programmer either forgets a piece of memory the previous level allocated, so it doesn't get de-allocated, or he makes a coding error where he thinks he's de-allocating that memory but actually isn't.  Either way, some of that "old" memory space is left hanging around, and the process gets bigger and bigger over time.

Bruce

[shiznitz]

That is what I guessed. It just seems like such a common error I am surprised it happens so often. But then again, it sounds like every program has the potential to create one so they are therefore inevitable given the amount of programs in any single application.

[HaemishM]

Think millions upon millions of lines of code. Miss a semicolon and the whole thing shits itself. Omit a line that tells the program to collect garbage and it slowly eats its own body arm first.

The first error is generally an obvious one. Memory leaks, OTOH, are stone cold killer bitches to find, especially in MMOG's.

[Trippy]

Just to make one thing clear, just because a program uses a ton of memory does *not* necessarily mean it has a memory leak. The is a common mistake that I see repeated constantly on game-related boards. People will scream out "OMIGOD THIS THING IS USING UP 600 MB OF RAM -- IT HAS A MEMORY LEAK!!!!" when in fact it doesn't -- it is simply using up a lot of memory.

[Margalis]

Of course there is the opposite case, deleting or using memory you haven't actually allocated. That's the majority of random crash bugs. (Really nearly all of them)

[Biobanger]

Of course there is the opposite case, deleting or using memory you haven't actually allocated. That's the majority of random crash bugs. (Really nearly all of them)

But not anything DirectX 8+, the "garbage collector" was tweaked so this doesn't happen. The program can still crash cause of this if the programmer tried to make his own in the wrapper code for his own code.

[Miguel]

Here is an analogy that was used in one of my very early CS courses:

Imagine you have a bathtub half full of water.  You have glasses in your posession that you can use to remove water from the tub, or put water back into the tub.

You dip a glass into the tub, remove a glass full of water.  Now you have a full glass, and there's a little bit less water remaining in the tub.  Now you take another glass, dip it in, and remove another glassfull.  Now there is even less water.

Let's supposed it took you 25 glasses to remove half of the water remaining in the tub.  Now, suppose someone else walks up to the tub and demands a large quantity, more than what is currently left in the tub.  The only way to satisfy that request is for you to pour some of your glasses back into the tub to satisfy the other person.

If you are really careful and don't spill any of the water, you can take your 25 glasses and pour them back into the tub, returning the water level to what it was before you started.  You can do this over and over, taking out 25 and putting back 25, and in the end the level of water in the tub will always be the same (neglecting evaporation, of course).

However what if you weren't so careful?  What if of your original 25 glasses of water, you kicked one over and it went onto the floor, leaving you with only 24?  Well, you figure I'll at least return 24 of my 25, and that's good enough, right?  

Well, if you always want to take 25 glasses of water, however are clumsy and always manage to spill one of them, over time, the water level in the tub will slowly decrease (since the rate of water returning is less than the rate of water leaving).  Over time, it would appear the tub is 'leaking' water, even though there is nothing wrong with the tub, it's that you are spilling one glass out of every 25!  The tub's not broken, however the water is going somewhere, correct?

A real disaster is when you try to put back a glass of water that you never took!  In this scenerio, the only way to satisfy this event is to create a glass of water out of thin air!  The laws of physics being as they are, this would usualy indicate that something is wrong in the universe, and the laws of physics have gone out the window, causing the universe to fold in on itself in a flash of cataclysmic annihilation (computer crash).  

Fortunately, in our case, we can throw out all the remaining old water, and refill the tub using the faucet, bringing us back to bathtub harmony once again (reboot).

[Ardent]

Good analogy. Even a shithead like me can understand it.

As for myself, I enjoy leaking in the tub every so often.

[Samwise]

Note that if your OS isn't the thing that's leaking, rebooting isn't necessary - just restart the leaking program.  Thanks to the magic of virtual memory, each program has its own bathtub all to itself, and can foul the water as much as it wants without harming its neighbors too much.  (At worst, they'll use up all the hot water in the building, but exiting the program will still fix that.)

[Margalis]

Note that if your OS isn't the thing that's leaking, rebooting isn't necessary - just restart the leaking program.  Thanks to the magic of virtual memory, each program has its own bathtub all to itself, and can foul the water as much as it wants without harming its neighbors too much.  (At worst, they'll use up all the hot water in the building, but exiting the program will still fix that.)

Well...sort of.

On older OSs, it's very possible to overwrite memory you don't actually own, which can corrupt other processes including the OS. This is pretty much impossible in NT and I suppose XP.

Second, you aren't talking about virtual memory. Virtual memory is just using hard drive space to double as ram. What you are talking about is protected memory. In NT any time you try to dereference a pointer it checks to make sure the memory is in the process space.

That's why in NT you tend to see dialogs after a program crashes like "Could not acces 0x0DFFE7". Your process tried to mess with memory it didn't own.

[Samwise]

A pagefile is one component of virtual memory, but as I understand the term (and admittedly it's been a few years since I took that class, so my terminology might be off), virtual memory is more general than that - it simply means that the OS (or whatever implements your virtual memory) gives the application a virtual memory address space that it can use as it pleases.  The virtual address space appears as a contiguous block of physical memory, when in fact the virtual memory manager could be constructing it out of noncontiguous chunks of physical memory and/or pagefile, if need be.  Further, the application can piss all over that address space, and it won't hurt anything else, because they're not accessing the "real" memory directly - the virtual memory manager keeps them in their little box (kinda like they're plugged into the Matrix - nothing is as it seems).

If an individual application is allowed to "leak" after it's been closed, the OS is at fault, so I'd say the OS is the one leaking.  Modern OSes don't leak.  (Mac Classic and DOS don't qualify.)

[Shavnir]

I don't think Win 95 had protected memory.  98 might have, but I can't remember my youth that far back for the life of me :P

[Samwise]

Win 95, 98, and Me were all thin GUI veneers over DOS underpinnings.  I wouldn't give them credit for much.

[Roac]

I don't think Win 95 had protected memory.  98 might have, but I can't remember my youth that far back for the life of me :P

No, 95/98/ME did not protect memory well.  Reboots may be (are likely) neccessary.  I don't recall having similar problems on NT platforms (2000, XP... never used the original NT much though), but I won't say it's impossible.  Securing memory was one of the touted features of NT.

[Miguel]

The tub analogy can easily be extended to protected memory allocation techniques.  Since I have already driven it into the ground, a little more can't hurt right?

Let's say you have the same bathtub as I described before, except instead of glasses to remove the water, you have tupperware containers, complete with a lid.  So when you decide you need some water, you dunk the tupperware container into the tub to fill it, then place the lid on top to seal it.  Then you write your name on the outside of the container, so some other asshat doesn't think it belongs to him!

I may decide later that I really don't need this water, so I erase my name, open the lid, and pour it back in.  However what happens if I just decide to throw it back into the tub without opening it?

Well, the water level does increase back to it's previous level, except the water trapped in the container can't be scopped by other people and their own tupperware containers.  The water is trapped in the 'protected' region of water in the tub.

However this type of error isn't as frequent as the previous kind (where someone doesn't return all the memory they asked for earlier), since the protection mechanism is controller by the memory manager, and is invisible from the point of view of the application.  In reality, the individual applications are not reponsible for protecting and bundling the memory regions they receive as part of a memory request transaction:  the memory manager does it 'behind the scenes'.  As someone noted above, typically applications deal within their own 'virtual address space', and the memory manager of the OS takes care of translating virtual requests to their physical memory equivalents.