See also: last week's Kernel page.

Kernel development

Also in 2.3.46: devfs. The story of devfs is one of impressive determination and persistence on the part of Richard Gooch. Richard's devfs replaces /dev with a virtual directory structure created at run time. It makes device handling much more flexible, deals with a number of annoyances (like the business of adding one SCSI disk renaming all the others), and replaces the typical cluttered /dev directory with something much cleaner, and which contains only entries that actually correspond to hardware on the system.

Almost since the beginning, devfs has come under attack from a large and vocal group that sees it as unnecessary kernel bloat. The lack of consensus on devfs was almost certainly one of the reasons why it did not make it into the 2.2 kernel. Undaunted, Richard kept on advocating devfs, developing the code, and releasing patches (the current, and perhaps last, is version 157. As of 2.3.46, he has achieved his goal of getting it into the mainline kernel.

For now, devfs is marked "experimental," and could remain that way into 2.4. But it has been out there and extensively used for some time, and should be stable. Those interested in playing with it should probably start with Richard's devfs README file, which explains how it works in detail.

One reason for the shakiness of recent kernels is the integration of the softnet patches. Softnet is a reworking of the core networking subsystem to make it fully multithreaded and more cleanly done in general; the work is being headed up by Linux networking master Alexey Kuznetsov, with lots of contributions from many other people. Those wanting a terse, technical history of softnet development in "changelog" form can take a look at Alexey's README.softnet file.

Softnet will make for better Linux networking, but in the short term it requires quite a bit of thrashing of the code. See, for example, this note from Dave Miller on changes in the network driver interface. These changes require changing every network driver in the system - it is a fundamental API change. Again, the end results will be good, but it leads to some instability in the short term. 2.4.0 is still somewhat distant. (Those needing to update drivers can also look at the softnet drivers HOWTO).

The integration of the new RAID code is also a prominent feature of recent kernels. RAID integration is still a work in progress; see Ingo Molnar's message for details. Current 2.3.x RAID should not, shall we say, be used on production systems. But for those who have waited a long time for the new RAID to go into the mainstream kernel, this is good news.

Scheduled Transfer Protocol for Linux. SGI this week announced the release of a Scheduled Transfer Protocol (STP) implementation for Linux. It's an alpha-level release, in the form of a patch to the 2.3.16 kernel, so it won't be showing up in the mainline kernel release for a while yet. But it's a good start.

STP is a relatively new networking protocol which is aimed at very high bandwidth and low latency. It works by setting up transfers in advance, and preallocating the necessary buffers. Thus data can go directly from the wire to its resting place in the system. It functions almost like a DMA transfer over a network cable.

STP can be used to carry higher-level protocols. For example, one can layer SCSI on top of STP, and use the result to implement a new, faster line of network-attached disk drives. Larry McVoy stirred things up a bit with a related suggestion: add a processor to each disk drive so that the drive itself runs Linux. Each drive becomes an autonomous system plugged directly into the network, able to serve up data via STP.

The idea has some appeal. Processors and memory are cheap enough to do this. Massive servers are getting harder to scale; distributed networks (think "disk farm cluster") can get better performance cheaper. Drives running Linux are easy to set up and customize to particular needs. And drive manufacturers are desperate for anything that can increase the value of their products and give them a margin they can actually survive on.

Larry may be a bit ahead of his time. But things could conceivably go that way, and SGI's latest contribution to Linux could be a step in that direction.

Storing capability bits in the filesystem has come up again. The question of how to represent capabilities (or "privileges" for the old VMS folks out there) has been an active topic of discussion for over a year. Capabilities describe the actions a given program is allowed to perform, and must be associated with the program's executable file somehow. Those who want to store capability bits in the filesystem itself would appear to have won over those who wanted to put them in the ELF executable header; the current discussion is about implementation of the filesystem option.

The problem is this: there is room in the ext2 inode structure now to hold 32 bits worth of capabilities. Linux currently has 28 capabilities defined, but everybody expects that list to grow. If it grows beyond 32 (considered likely), a more elaborate method of storing them will need to be devised.

Given that other sorts of file permissions metadata (such as access control lists) also need to be stored, it would seem that a definitive answer to the problem should be implemented now. Otherwise things will just have to be done over again in the near future.

There is, however, a point of view (championed by Ted Ts'o) that more capability bits are not necessarily a good thing. Managing of capabilities looks like it could be one of the truly big system administration challenges of the future. Given the amount of trouble people (and distributors) have with the existing permission bits, how will they cope with dozens of capability bits that must be correctly set on every file? Capabilities, poorly handled, could become a major security problem instead of the solution they were intended to be.

The human factors side of capabilities has really not been addressed so far. Without some attention in that area, there could be some real problems waiting in the near future.

Future projects for the ext2 filesystem were brainstormed at an "ext2 puffinfest" attended by Ted TS'o, Ben LaHaise, Phil Schwan, Alexander Viro, and Matthew Wilcox. The result was this list of interesting things that could be done with the Linux filesystem. It may take a while to work through the whole thing...

Other patches and updates released this week include:

• Netfilter 0.90 has been released. Netfilter is getting close to the point of being merged into the 2.3 tree, and needs testers now.

• Jens Axboe released a module for "packet writing" of CD-RW drives. Writing in this mode allows for transparent access to the device, without the need for a separate recorder program.

• Timpanogas has released a "NetWare Volume Imager," which looks a lot like a NetWare backup and restore system. There's a GPL version, and a commercial variant that adds extra functionality.

• Dave Higgen has posted a merge of the latest NFS fixes for the 2.2.14 kernel. Those who are running 2.2.14 and still having NFS problems (a number of people, seemingly) might want to give this a try.

Section Editor: Jonathan Corbet

For other kernel news, see: Kernel traffic Kernel Newsflash Kernel Trap Linux 2.5.x Porting help

Other resources: Kernel Source Reference L-K mailing list FAQ Linux-MM Linux Scalability Effort Kernel Newbies Linux Device Drivers

Next: Distributions