Linux on the IBM iSeries

July 25, 2000
J. Corbet

This talk, given by IBM's David Boutcher, went into the details of what was needed to get Linux running on the IBM iSeries server.

The iSeries is a fairly substantial machine. It runs up to 24 64-bit PowerPC processors, with 128GB of memory and up to 18.5TB of disk space. It is, essentially, "a giant database on wheels." It is also designed to be run in the partitioned, virtual machine mode that IBM has favored for a long time. Thus, it runs a "hypervisor" program whose job it is to divide the machine between various "partitions," and to keep them from interfering with each other.

The primary operating system for this box is OS/400. It shows a different design approach than Linux - IBM has, for example, integrated both a Java virtual machine and the DB2 relational database system into its kernel. At this point, an iServer system must run at least one OS/400 partition for management tasks; Linux may, eventually, be able to fill that role, but it can not now.

The 64-bit PPC processor has an interesting feature. It looks to the system like two independent CPUs, but, in fact, it is a single processor with enough duplicated hardware (i.e. register sets) to allow it to switch quickly between two threads. Whenever one thread runs into something that blocks its execution - a cache miss, for example - the processor switches to the other. The result is an approximate 30% improvement in performance.

The result is also some challenges for Linux. What happens, for example, if one of the two threads takes out a spinlock, then takes a cache miss and looses the processor? If the other thread also finds itself needing that lock, it will start spinning on the processor. The thread which must release the lock never gets to run, and everything comes to a halt.

Two approaches are taken to deal with this problem. One is to allow interrupts to happen (though their processing is deferred); interrupts bring the hypervisor into play, and can cause the spinning thread to lose the processor again. The other makes use of a hack in the hardware itself. If the processor encounters an instruction that does a logical OR on register 1 with itself, storing the result back in register 1 (i.e. a particular type of no-op), it is interpreted as giving the current thread a low priority. That will cause the other thread to run. Performing the same operation on register 3 gives the current thread a high priority instead. A thread spinning on a lock can drop its priority, thus avoiding some of these problems. This solution may seem like a hack, but it allowed IBM to craft a solution without adding new instructions that would break backward portability with older processors.

This processor provides other challenges as well, including the fact that, due to the way the paging system works, the kernel can take a page fault at absolutely any time. The Linux kernel normally is not prepared for that possibility, and a fair number of late nights were evidently required to make it all work.

Thanks to the joys of wireless networking, we were even able to see a demonstration of SuSE Linux running on an iSeries server. It looked much like Linux, with the exception that the server presented several Linux "systems" running on the same server.

More information is available on the IBM iSeries Linux page.

Back to OLS 2001 coverage