See also: last week's Kernel page.

Kernel development

Alan Cox's current release is 2.4.5ac4. This patch is, as usual, rather larger than the Linus patch. Happily, while merging with 2.4.5, Alan separated out the list of patches which are unique to the "ac" series. There are still hundreds of them - the merging process has a while to go. Interesting stuff in there includes an important fix for a problem that causes crashes for 2.4.5 ReiserFS users, a number of fixes to bugs recently found by the Stanford checker (see below), user-mode Linux, the CMS filesystem, a version of the JFFS filesystem with compression, threaded core dumps and a tremendous number of bug fixes.

Andrea Arcangeli, meanwhile, has released 2.4.5aa1, which contains a long and different set of fixes, many of which are performance related.

The return of the Stanford checker. We first covered Dawson Engler's global analysis tools in the March 22 LWN kernel page; thereafter, that group got quiet for a while. Now that final exams are presumably over, the checker group has resurfaced with a whole new set of bug reports. They include:

• Use of floating-point arithmetic, which is not allowed in kernel code (10 occurrences).
• Use of memory which has been freed (24 occurrences).
• Memory leaks in error returns from functions (23 occurrences).
• Use of user-space pointers that are known to be bad (10 occurrences).
• Overly-large stack variables - the stack used for kernel code is quite small, and overflows can bring down the whole system (37 occurrences).
• Failure to check for null pointers returned from routines which can fail (103 occurrences).
• Dereferencing of pointers known to be null (84 occurrences).
• Dereferencing user-space pointers without checking (4 occurrences).

Unfortunately, the checker still is not generally available, though Mr. Engler has told us in the past that he intends to release it when development is complete. In the mean time, the group has announced a web site which provides access to the bug database. Have a look for a view into the sorts of problems that can be found with sufficiently sophisticated tools.

Stabilizing memory performance in 2.4. One of the remaining problem areas in the 2.4 kernel is its virtual memory subsystem. Not only are there simple performance problems, but there are also still situations that can cause the kernel to deadlock. Not quite what one wants to see in a stable series. The kernel hackers are working on dealing with these problems, however; with luck, VM difficulties will not be with us for much longer.

One problem that users of recent 2.4 kernels are likely to have noticed is heavy use of swap space. It is not unusual for a kernel to be running heavily in swap even when there is not that much going on with the system. This problem seems to have only gotten worse with recent kernels. It is, of course, the same old bug where the kernel fails to recover swap space for pages which have been brought back into memory (covered in the May 3 kernel page). This problem will get fixed, but not quite yet. Alan Cox phrased it well:

That is a giant size special edition stupid design flaw that is on the VM hackers list. But there are only a finite number of patches you can do in a day, and things like sucking completely came first I believe.

Alan didn't spell out what "sucking completely" meant, but most would probably agree that system deadlocks could be fairly described with that term. Rik van Riel this week managed to stir things up a bit with this patch which attempted to fix a couple of deadlock problems.

One deadlock problem happens on high-memory systems. "High memory," in this context, means memory beyond that which can be directly mapped into the kernel's address space, and often beyond that which can be addressed with 32-bit pointers. High memory is useful for many things, but it brings with it some special problems; one such problem is the fact that it can not, usually, be used for I/O operations. Since most peripheral devices can only address 32 bits of memory, the high memory zone tends to be beyond their reach.

That means, in turn, that any data that moves between a high memory page and, say, a disk drive must be copied by way of a "bounce buffer" in low memory. On systems with a lot of high memory and a lot of I/O, a large number of bounce buffers will be required. If the bounce buffers succeed in taking up enough low memory, buffer allocations will hang, and the system will no longer be able to allocate the memory needed to free other memory. At that point, the system locks up.

Similar problems can happen with buffer allocations even in the absence of high memory, though they are more rare.

Rik's patch attempts to fix these deadlocks by simply allowing certain memory allocation operations to fail. It is, in the end, better to fail an operation than lock up the system. Ben LaHaise, instead, has posted a patch which addresses the problem through a memory reservation mechanism. In this scheme, a set of pages is set aside just for critical tasks like bounce buffer allocation. By prioritizing these allocations, deadlocks can hopefully be prevented.

Linus, however, doesn't like any of these approaches; he thinks they are attacking symptoms. Rather than try to deal with very low-memory situations, he says it is better to try to figure out how those situations come about in the first place. A fix can then deal with the original problem.

That fix may be a little while in coming, still. Until then, 2.4 virtual memory will not be all that Linux users would like it to be.

The Advanced Linux Sound Architecture project has moved, to SourceForge, of course. Subscribers to the old ALSA lists will need to go to the ALSA mailing lists page and sign up again. ALSA is getting increasingly interesting as it heads toward a stable release and, presumably, a bid for inclusion in the 2.5 kernel.

Other patches and updates released this week include:

• Marek Zelem has released a new tool called e2salvage, which attempts to recover data from badly damaged ext2 filesystems.

• Alexander Viro has posted a new namespace patch, and is looking for testers.

• Version 1.1.15 of the NT filesystem patch was released by Anton Altaparmakov. It fixes a number of bugs, but still does not properly support write access.

• IBM has released version 0.3.3 of its Journaled Filesystem.

• Eric Mouw has written a guide to the /proc API.

• A new devfs patch was posted by Richard Gooch.

• Steve Whitehouse has released a zero-copy network block device implementation.

Section Editor: Jonathan Corbet

For other kernel news, see: Kernel traffic Kernel Newsflash Kernel Trap 2.5 Status

Other resources: L-K mailing list FAQ Linux-MM Linux Scalability Effort Kernel Newbies Linux Device Drivers

Next: Distributions