See also: last week's Kernel page.

Kernel development

Those wanting to try the testing releases should approach with caution, however; the releases through -pre5 have had some serious file corruption problems. The debugging effort took some time, and was not helped by the simultaneous emergence of a corruption bug in the pine mailer which was, ironically, triggered by a particular message on the BUGTRAQ list.

The bug (the kernel bug, that is) turns out to have been introduced in 2.3.7, over a year ago. Recent changes have brought it out, however. Once it was found, a couple of fixes of the "brown paper bag" variety went in, until Linus finally tracked it down. As he put it:

Thanks, and THIS time it really is fixed. I mean, how many times can we get it wrong? At some point, we just have to run out of really bad ideas..

As always with development kernels, be careful out there... especially given that the -pre6 patch announcement from Linus expresses a rather lower level of confidence.

Meanwhile, Daniel Stone has been working on a press release for the 2.4 kernel, whenever it should come out. (The release is optimistically dated September...)

The current stable kernel release is 2.2.17 - finally. The 2.2.17 release was officially blessed and announced on September 4. Work on 2.2.18 had actually begun before 2.2.17 came out; the latest announced prepatch is 2.2.18pre2; there is actually an unannounced "pre3" patch out there as well. The 2.2.18 series has started off with a number of "higher risk" updates - things which Alan Cox did not want to put into 2.2.17. Perhaps most significantly, the USB backport from 2.4 is there; most vendors were already shipping kernels with 2.4 USB anyway, so putting it into the official version won't change much.

The ritual call to see the updated NFS code in 2.2.18 has gone out. Alan is, so far, resisting. The new NFS code fixes a lot of problems (and adds NFSv3), but it's still a big change for a stable kernel series.

The current ancient kernel prepatch is 2.0.39pre7, which was announced on September 2. Unless something changes, this will be the last of a very long-running prepatch series, and the 2.0.39 stable kernel release will happen at some point relatively soon.

TUX is loose. Ingo Molnar got the Linux world's attention last June, when his "TUX" subsystem established the world SPECweb record for web page serving. Since then, however, things have been quiet on the TUX front; in particular, no source has been forthcoming. That changed on September 1 when Ingo announced the first Alpha source code release. TUX is now loose.

Of course, this code is not something you would want to put on your main corporate server at this point. It was bleeding edge when it ran the SPECweb test, and has seen a fair amount of development since then. There are no guarantees with this release, except that, as stated in the announcement, "if it breaks you get to keep both pieces."

It is, nonetheless, interesting to see what has been done to create such nice results. TUX is, as stated, a kernel-based HTTP server which is oriented toward static content. Even highly dynamic sites can have a lot of static content - images, for example. TUX gets that content out quickly by avoiding a lot of the overhead involved in a user-space web server. In addition, a number of tricks can be employed when you are running within the kernel. For example, TUX not only caches the busiest files, it also caches the computed TCP checksums to avoid having to do that work over and over again. TUX can also work with high-performance network cards to avoid copying file data within the kernel.

Also part of TUX is an exported hook which can facilitate the caching of dynamic content. Often dynamically-produced pages are always the same for the same URL "input," and the work of generating them can be avoided through proper caching. Making use of this capability requires changes to the (user space) code which generates the dynamic content.

TUX will never be able to handle the entire job for many sites, however. Thus, it relies on a user space web server daemon; whenever TUX encounters something it can't deal with, it just passes it on to the (presumably smarter) user space server.

The TUX developers are aiming at a stable release of the code sometime in September. Thereafter, don't be surprised if it starts showing up on web sites worldwide. (For more information, see the TUX README file that comes with the distribution).

Is zero-copy TCP worthwhile? A certain Jeff Merkey of the Timpanogas Research Group spent quite a bit of time criticizing the Linux kernel for copying network data too many times. After all, says Jeff, both NetWare and his own MANOS (NetWare replacement) system do zero-copy TCP, and thus must be better. Is Linux really that poorly implemented?

The answer turns out to be "no", for a couple of independent reasons. On the purely technical side, the standard output path for TCP data only copies that data once - and that happens as part of moving the data into kernel space and computing the TCP checksum. So it would prove hard to improve much on that code.

The other interesting point of view, however, came from Linus, who points out that zero copy is often not the performance win that people think it is. By the time you have done all the overhead required to do a zero-copy operation (pinning the user page in memory, setting up the scatter/gather DMA operation, etc., and cleaning up afterward), you might as well just have copied the data. Zero copy is a nice idea that often is not so practical in the real world.

That notwithstanding, Ingo Molnar pointed out that the TUX webserver implementation includes a zero-copy scheme.

No NDS for Linux - to spite Microsoft?. The Timpanogas Research Group, headed up by Jeff Merkey, has announced the withdrawal of its promise to provide an open source Network Directory Services (NDS) implementation for Linux. Instead, TRG will concentrate its efforts on its own (open source) "MANOS" operating system. The press release included this interesting comment:

It is not our desire at this time to hand Microsoft or anyone else an open source 'scythe' they can use to harvest the NetWare(TM) installed base until a competitive Open Source NetWare NOS is available for Novell's customers to consider.

It is an interesting view - somehow providing an open source implementation will allow Microsoft to snap up all the Novell users who are, presumably, looking for a new vendor. Mr. Merkey reiterates this claim in another post where he states: "If we post it, Microsoft will grab it and it will be in NT within 48 hours of them downloading it from our site."

This sort of thing, of course, is what the GPL is for. It is a sad world indeed if contributions to Linux are pulled out of fear that Microsoft might use them. If Microsoft picks up some Linux code and uses it in accord with the terms of the GPL, then there is nothing for anybody to complain about. After all, one of the criteria for free software is that it cannot discriminate against users. If the GPL is not respected, it's time to call out the lawyers. Fear of Microsoft is no reason to withdraw kernel contributions; one suspects that a more complicated agenda is at work here.

Why is there no Linux kernel debugger? A certain Jeff Merkey posted a message to linux-kernel giving a different reason for the withdrawal of the Linux NDS effort. It seems that the real problem is that the kernel debugging facilities for Linux are inadequate.

The lack of a Kernel Debugger and other basic kernel level facilities on Linux make TRG's job about 20 times harder on Linux and take almost 10 times as long as is possible on NT, NetWare, or MANOS to develop software as a result of this.

Mr. Merkey has found himself to be rather less alone with this one - numerous people wonder just why a kernel debugger is not part of the mainline kernel itself. See, for example, this poston the absence of kdb.

The reason there is no debugger in the Linux kernel is, of course, because Linus does not want it there. This time, though, it was David Miller who came to the defense of this policy. The basic idea, which mirrors things Linus has said over the years, is that having an interactive debugger causes developers to focus on symptoms, rather than on what is really going on. According to David, some elbow grease invested in tracking down problems the hard way pays off in both a higher quality fix and a deeper understanding of how the kernel actually works.

The whole debate is long (and ongoing). For those interested in the soundbite version:

"Hard work now leads to less work later."
   -- David Miller

"Hard work now leads to less work full stop."
   -- Alan Cox

Other patches and updates released this week include:

• Miles Lott posted a read-only implementation of the BeOS filesystem for the 2.2 kernel.

• Mikael Pettersson released version 1.5 of his x86 performance counters driver.

• Christoph Hellwig posted his patch to make the Logical Volume Manager work with devfs.

• The September 2000 release of the CITI NFSv4 implementation for Linux has been announced.

• H.J. Lu has released version 0.2 of the nfs-utils package.

Section Editor: Jonathan Corbet

For other kernel news, see: Kernel traffic Kernel Newsflash Kernel Trap 2.5 Status

Other resources: L-K mailing list FAQ Linux-MM Linux Scalability Effort Kernel Newbies Linux Device Drivers

Next: Distributions