Date: Wed, 26 Jul 2000 12:29:09 -0600 From: Theo de Raadt <deraadt@CVS.OPENBSD.ORG> Subject: Re: Chasing bugs / vulnerabilties To: BUGTRAQ@SECURITYFOCUS.COM > Try the UNIX Fuzz experiment, first conducted at the University of Wisconsin > on multiple UNIX operating systems and when tried again several years later > revealed only slightly better results (the Fuzz experiment throws garbage > input on the command line into a program and tests the response). We (check out > http://www.cerias.purdue.edu/coast/ms_penetration_testing/v11.html) tried > the same experiment on WinNT with 'interesting' results. After Michael mentioned this, I decided to download this and see if it found any bugs in OpenBSD. I was pretty sure I'd find at least a few. I hacked it up a bit, and then ran it against every single binary in a "default install" openbsd system. I told Michael what I had found, and he suggested that I tell bugtraq. I ended up finding the following: nc (netcat) - lack of buffer termination in code where it fakes argument parsing from stdin, causing strchr to run off the end - fixed mg - did not like stdin being redirected, and crashed in terminal initialization - fixed ctags - about 18 buffer overflows in token parsing - fixed lex - looks like 2 buffer overflows based on strcpy in parsing routines. - looks difficult to fix: not yet fixed. as - indexing into an array using signed char's: fixed - other bugs exist which trigger some assert()s, but since we are moving to a new version soon, we'll skip fixing those for now. makeinfo - part of texinfo. buffer overflow. still being looked at. sort - infinite loop when fed a single (and long) non-NL terminated line - not fixed yet indent - infinite loop if it encountered EOF while waiting for a '}' nesting terminator - fixed rdist - leaves /tmp files created using mkstemp() around - still trying to fix this - note that our rdist has not been setuid or setgid for roughly half of eternity cvsbug - leaves /tmp files lying around when terminating due to no action possible - fixed sendbug - leaves /tmp files lying around when terminating due to no action possible - fixed So far, nothing was security sensitive by itself. While it fixed things, and that always helps us understand the problems in all classes of code better, I don't think this is directly related to 'security' work. However, as I said, it helps us understand the mistakes people make when they write programs. It helps us understand that many of those mistakes are pretty basic C errors. It helps prove the case for getting a second or third eye to look at the source code. Too often, I look at code from 1985 and discover bugs which lead me to believe I am the second, third, or perhaps fourth eye. I would love to find a simple tool which could test a variety of command line arguments for functions. Or perhaps something written to test every single command line option on every base unix command. I suspect some vendor probably has something like this, but probably locked up in their back room; lest operating system quality as a whole improve and force them to improve theirs too. So, when we finish fixing the last few of these problems, it'll probably be about 20 hours of effort by 3 people. The way I look at it, the interesting thing is that we actually do this stuff. Thus, the important question is probably: "Is anyone else doing this?" I still consider fuzz to be somewhat of a crutch. For about half of these fixes, inspection found other things we could improve. Now that we've run this, we will probably discard the tool (mostly since it causes havoc, running wall(1) and logger(1) against these hugs nasty input files causes some grief). We've gotten 99% of the benefit it can find, and perhaps we can re-run it in 2-3 years. It's just not a typical useage tool. And most other checkers like this are not either. However, I think that once a person learns to audit code by just reading it, that skill is transferable and reusable on a daily basis. That was my main argument against *printf checkers; whether they be external or built into a C compiler. They still miss things. You still have to go read the code. Thanks for the entertainment over the last week from various people suggesting we can change the C language, redo how varargs/stdarg work, or showing a complete misunderstanding of the *fundamental weakness* of static analysis. A suggestion for those people: Learn how things work and why they work that way before suggesting how they should be changed. Read the stadards, read some implimentations. Apparently some people feel that writing diatribes is a replacement for reading.