Date:	Sat, 25 Mar 2000 13:09:19 -0500 (EST)
From:	"Richard B. Johnson" <root@chaos.analogic.com>
To:	"Mike A. Harris" <mharris@meteng.on.ca>
Subject: Re: Endless overcommit memory thread.

On Fri, 24 Mar 2000, Mike A. Harris wrote:

> I've been reading bits and pieces of this whole thread for a
> while now, and have finally come up with an opinion.  I think
> that overcommit should be an option which defaults to
> off.  Here's why:
> 
[SNIPPED...]

Suppose we had a perfect operating system. It had 100 megabytes of
virtual RAM (physical and mechanical storage). It also had zero
overhead. It was, by definition perfect. It did not, and could
not over-commit any virtual RAM.

Just after the machine is booted, I want to run a program that
runs faster and better, the more memory it has. Perhaps it's a
Web Server or a Database.

This perfect program allocates all the memory it can, by first
trying to allocate a gigabyte. This fails, it then, by successive
approximation (perhaps Newton's method), finally is able to allocate
90 megabytes. The other virtual RAM has been perfectly allocated
by the various other daemons.

Now, three weeks from today, that program will finally touch the
last bit of RAM it allocated.

In the meantime, you can't log in. 

Even if the perfect operating system kept some virtual RAM for
'root' to log-in. It has no idea of what programs root would
want to execute. Eventually, it will fail with an out-of memory
condition.

Therefore all known Unix Operating Systems allow the use of
'currently unused' virtual RAM. This RAM may have been allocated, but
it has not been touched yet. The tasks that have allocated it will
not even know that it was 'borrowed'. This is the nature of
"over commit".

This has a side effect that has been known for a long time, as long
as Unix has existed. It is possible for a user to deliberately
attack the machine. Resource management helps, but a user can still
bring the machine to an out-of-virtual RAM condition. The immediate
solution has been for the kernel to kill off programs when necessary
to assure its survival. Some think this is a bad idea. They what
their program to "survive" while the operating system dies.

A smarter 'kill' could be obtained using a daemon that watches
all the process activity. It could detect runaway programs, either
deliberate or otherwise. The kernel would never encounter an
out-of-virtual-RAM condition if such a program existed.

Nay-sayers state that the daemon would never get a chance to operate
against a well-designed killer program. However, this is not correct.
Once disk activity occurs, there is plenty of time available for
a watcher daemon to kill off the offending task. This is because the
task must sleep while the I/O is completing.


Cheers,
Dick Johnson

Penguin : Linux version 2.3.41 on an i686 machine (800.63 BogoMips).


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/