Web100: Getting that 100%

Linux Weekly News
LWN daily updates
Linux Stocks Page
Contact us

Web-100: Getting that 100%
Web 100 logo A new initiative under funding consideration by the U.S. National Science Foundation has the potential to greatly increase real effective bandwidth users get out of the Internet - and the work will be done using Linux. The benefits would be many: better network usage, Linux at the forefront of Internet research, and perhaps even federal funding paying for Linux kernel hackers to improve the system.
The National Science Foundation has spent more than $50 million dollars on the networking infrastructure to make gigabit networking available to scientists throughout the United States to facilitate their research. Unfortunately, as many of the rest of you may have discovered, having a fast and large bandwidth network connection doesn't mean that any one transfer will actually get 100% of that bandwidth. In fact, transfer rates generally run 10% or less of a really large pipe, even when the pipe is essentially unused.
What is the problem? The main issue is with how TCP is designed and very much with how it is implemented. A maximum window size is used to set the amount of data that can be transmitted before you must receive an acknowledgement or quit transmitting. When you have a very wide pipe, such as a gigabit network, the maximum window size must be set to take into account both the bandwidth and the latency of the network. If the value is set too low, then the software is forced to stop transmitting prematurely, preventing it from taking advantage of the networking power available to it.
Currently, most applications don't even attempt to find and set an optimal maximum window size. Instead, they accept the default set by the kernel, which obviously must be optimized for the most common networking situations. In fact, applications may not even give the end user the opportunity to override this default value. In some cases, versions of ftp or other applications have been created to allow this maximum window size to be set. However, figuring out the optimal value is not an easy exercise and currently requires the active participation of a network engineer with intimate knowledge of the specific network topology.
The National Science Foundation has agreed to fund further investigation into the development of open source solutions for these problems. Linux and Apache are the two primary candidate packages for an operating system and web server on which to base the proposal. The draft white paper for the Web-100 project is available and under development. A Web-100 workshop, organized by the Scientific Computing Division of the National Center for Atmospheric Research, will be held October 27th and 28th in Boulder, CO to explore NSF support for this effort. If you are interested in participating, contact Basil Irwin, who is heading up the workshop.
What about those of you who don't have Gigabit networks? Should you be interested? Well, remember that the original Internet was also funded and supported by the National Science Foundation. Their interest was in supporting quality research; the result made low-cost networking a commodity on which we've all come to count. Put in a hundred-fold improvement in the useability of the current network technology and the number of applications that want to take advantage of it will multiply as well.
Why should you care about NSF's involvement in this process? It may bring real money and reliable work to developers in this area. It will solve a problem that will impact all of us, eventually, when we get access to higher bandwidth networks in the future. It is funding that is not complicated by commercial interests, to be based on open source. If they solve the problems for us now, it will help catapult us smoothly into the future, like when we all have gigabit networking performance to our homes. Take a look at how instrumental NSF's funding was to the birth of today's Internet. The NSFnet, created and funded by NSF, was the predecessor to the Internet and the Mosaic project was the mother of our web browsers today. Then think about the possible impact of their funding of a hundred-fold improvement in today's performance.
--Liz Coolbaugh