![[LWN Logo]](/images/lwn.banner.gif)
Date: Tue, 8 Sep 1998 17:47:02 -0400 (EDT)
From: autosans@clark.net (The SANS Institute Email Robot)
To: cool@eklektix.com
Subject: About the SHADOW Intrusion Detection System
To: <cool@eklektix.com>
From: The SANS Email Robot (SANS Institute)
SHADOW requests are coming in at an overwhelming rate. Please
accept this automated response. If you need more questions answered,
please send e-mail to sans@clark.net.
This SHADOW release document has two parts:
The first (short) segment, contains dates cities and URLs for the
training programs beginning on July 24 that will provide you the
foundation knowledge needed to implement and use the SHADOW system.
The second provides software documentation, downloading instructions,
and instructions for building new filters and analyzing data.
Part I: Training seminars schedule
These training seminars are taught by Stephen Northcutt, the Naval
Surface Warfare Center security analyst most directly responsible for
the creation of SHADOW and the person who uses it to monitor traffic to
over 14,000 hosts. Mr. Northcutt also received the highest ratings of
any faculty member at the Monterey SANS conference -- a conference which
regularly draws the best speakers in the country. We urge you to attend
one of these programs if you can fit it into your schedule as they are
not likely to be repeated before 1999.
Complete agendas, exact course locations, and faculty profiles are
available at the following urls along with registration and fees
information. The courses cost $945 for two days. Prerequisites are
UNIX experience and a familiarity with TCP/IP and other networking
concepts.
August 24-25, Washington DC <http://www.sans.org/dc2/dc2.htm#NT- 209>
An even more advanced and thorough course, covering four days will be
presented in Orlando, October 26-29. See <http://www.sans.org>
For questions about any of these training programs, email <paller@sans.org>.
Part II: SHADOW Documentation
Here is some information on the SHADOW security monitor.
Much more info available at: http://www.nswc.navy.mil/ISSEC/CID.
Copyright, 1998. The SANS Institute.
This draft is for review purposes only and may not be copied or forwarded
without prior permission of the SANS Institute.
Building a Network Monitoring and Analysis Capability
Step by Step
Version 1.1.8 980720
As more organizations rely on the internet for both communications and
electronic commerce, system administrators, CIOs and business executives
increasingly share a common worry. Their joint concern is that malicious
people may be using the internet to launch attacks on their computers and
that those attacks could do great damage to their organization.
A small number of organizations have gone from wondering whether
anyone was trying to attack their systems, to knowing for a fact
that people were, how often, and what vulnerabilities were being
tapped. They have built network monitoring and analysis systems.
With that knowledge the organizations have been able to tune their
firewalls and system protection strategies based on information, instead of
on guesses.
The early adopters of intrusion detection systems crafted their own unique
tools, but now some of the pioneers in intrusion detection are joining
forces to perfect a library of public domain, or freely available tools to
protect sites. This toolset, called "CIDER" (for Cooperative Intrusion
Detection Evaluation and Response), automates the process of information
gathering and traffic analysis based intrusion detection. Tools are of
little value if people can't use them. The purpose of this booklet is to
provide the instructions, examples and background information so that every
organization with reasonable Unix experience can install and operate a
network monitoring and analysis capability.
A SHADOW system (short for SANS's Heuristic Analysis system for
Defensive Online Warfare) can be built using freely-available software and
existing hardware or hardware that can be purchased for less than $10,000.
All CIDER components come with source code and tutorials on what they do
and how to set them up and what the results mean. Because of this, an
organization will find this approach complements and enhances the impact of
commercially available intrusion detection systems.
A note from Vicki Irwin, Stephen Northcutt and Bill Ralph of the Naval
Surface Warfare Center Dahlgren, designers of the SHADOW system
Most intrusion detection systems, whether network or host based, "push"
alerts when they detect an event. Common methods for doing this include
sending e-mail to the system administrator, alerting a pager, or beeping to
the screen. We have such capabilities in commercial systems and use them.
However we have found that in day to day use these systems have some
serious drawbacks. Many of the alerts turn out to be false alarms. Many
others turn out to be of little consequence, they are simple exploit
scripts that we are well defended against. The beeping, or alerting becomes
annoying to the point we find we ignore the alerts. At that point we are
relying on firewalls, system protections and other countermeasures.
SHADOW's primary requirement is to detect as many attacks as possible, as
easily and efficiently as possible, make it east to report the attacks to
CIRTs, and support analysis of the attacks to tune defenses. We "pull" the
detects using a web browser whenever we have time and examine the kinds of
probes and attacks directed against the organization. The web display is
simple to use and we have been able to train people in its use quickly. One
person can monitor multiple sites, in fact we monitor over ten sites. There
are a lot of ways the system needs to be improved and we are interested in
your suggestions and software contributions so that SHADOW can detect more
and be faster and easier to implement and use.
We want to thank the folks who helped up improve the software, in
particular:
Scott Hoye, University of Virginia, who wrote most of the analysis section
of this guide and was the primary tester of the system,
Dean Goodwell, Joint National Test Facility, who improved the installation
instructions and wrote an appendix on a specific installation,
Olav Kolbu, USIT, University of Oslo, who ported some of our shell scripts
to Perl,
Pedro Vazquez, Unicamp Brazil, who has helped us throughout the project
with his knowledge of tcpdump filters and their application to intrusion
detection.
We also want to thank all the reviewers of the document.
Overview of the SHADOW architecture ----------------------------------
SHADOW is based on tcpdump from the Department of Energy and support
software provided by Naval Surface Warfare Center Dahlgren and greatly
enhanced by the SANS community.
The data collection capability of SHADOW is based on libpcap. This software
package developed by the Network Research Group at Lawrence Berkeley
Laboratory, provides an interface for application programs such as NFR and
SHADOW to read data collected by the network interface on Unix systems.
Though it is possible to instruct libpcap to gather more information from
each packet, it is common for libpcap to collect the first 96 bytes of a
packet. The analysis programs perform operations on the headers of the
collected packet instead of its content. This approach to intrusion
detection is called traffic analysis.
Traffic Analysis based intrusion detection compared to Content Analysis.
The strength of a traffic analysis approach is that it is possible to
examine and record every packet that passes by on the network. In certain
situations such as educational institutions where privacy is a critical
concern, a traffic analysis approach may be the only reasonable approach to
network monitoring. The weakness of this approach is that some attacks can
only be detected by content analysis. Another approach to intrusion
detection is to examine the contents of packets for certain strings that
indicate an attack, this is a content, or string based approach. The
strength of the content based approach is that it allows one to collect all
of the data of a connection once an attack string is detected. The weakness
of this approach is that it is virtually impossible to analyze and record
all of the traffic on a network, therefore many services, or ports are
filtered out making the system blind to attacks using these ports or
services.
Begin sidebar to run alongside 1st paragraph below.
Throughout the history of electronic communications it has been
common practice to identify the point of demarcation, the point
that is the end of responsibility for one group and beginning for
another. The early firewall implementors on the internet referred
to the point of demarcation between the internet service provider
(ISP) and an organization connected to the internet as the
Demilitarized Zone (DMZ). DMZs are particularly fruitful
locations for sensors, however as a rule of thumb, any point of
demarcation is probably a good location to place a sensor. The
SHADOW architecture easily supports multiple sensors.
End sidebar
=========
Overview of SHADOW Sensor and Analysis System
The sensor is (usually) located outside your firewall and between
your firewall and your internet connection, an area often called
the DMZ. One of the design features of the SHADOW sensor is that
it has no knowledge of what it is looking for, it simply collects
all packets. This way, if the sensor should ever be compromised
(systems outside of your firewall should be considered high risk)
the attacker will not get any idea of what your organization is
filtering for. We recommend that you install a small hub, to
support the sensor. This way if you ever need to plug in an
additional sensor such as a protocol analyzer to support your
incident response team it is easy to do so.
NOTE: In a situation where you are collecting evidence and have a
reasonable expectation the evidence will be used in court
dedicating the sensor to evidence collection is recommended. It
is possible that all of the sensor logs and possibly even the
sensor itself may be considered part of the evidence. Also, if
you know what system(s) you want evidence from you may want to
focus the dedicated sensor on the target system(s). This way less
information about your organization will have to be turned over
to outside parties. Finally, if you are focusing the sensor you
may wish to collect all of the contents of the packets for more
complete analysis.
The analysis station is located inside your firewall. Its job is
to download and evaluate the data collected by the sensor. It
uses filters to collect events of interest (such as probable
attacks). This information will then be displayed on a web page.
The web page also has a supporting Graphical User Interface (GUI)
that allows the analyst to examine information from other
sensors, other time periods, to query the analysis system for
more information about a host or a pattern and also to create a
report suitable for sending to a CIRT.
The information that is displayed on the web page is the result
of tcpdump filters matching conditions that we are looking for in
the data. This booklet contains a large number of example filters
to help you flag the events in your data that you are interested
in. Some patterns cannot be detected using the tcpdump filter
language. Also, some attacks that are low and slow (such as a
scan that uses one packet per day) probably will not be detected
with a one hour interval. An analysis tool written in perl has
been included to handle these kinds of problems.
The computers may be any Unix systems that can compile libpcap
and tcpdump. You will need root access on these systems. You will
also need a recent version of perl, gzip, the Apache web server
and secure shell installed on the analysis system (the sensor
needs gzip and secure shell).
NOTE: Intrusion detection requires a very large amount
of disk space. More than half of your cost may be tied
up in mass storage. Our reference system uses a
twenty-three gigabyte (SCSI Seagate Elite 23) on the
sensor and two twenty-three gigabyte Seagate Elite 23s
on the analysis station. Any large disk drive
compatible with your Unix systems should work. Several
reviewers have pointed out that organizations can save
money by substituting non SCSI drives. One reviewer
provided the following information: On 7/3/98 Fry's
Electronics in N. California was selling 7.2GB
MaxtorUltra-DMA/33 drives for $189. Four of these could
be attached to an inexpensive Pentium-class machine
providing ~28GB for < $800.
Step 1: Acquiring the software for the SHADOW System
The software is available at no cost from the Lawrence Berkeley
Research Laboratory the Naval Surface Warfare Center, and the
SANS Institute.
Action 1.1. Download tcpdump and related software from:
ftp://ftp.ee.lbl.gov
The main program will be labeled tcpdump.tar.Z Make sure you also
get libpcap, (libpcap.tar.Z), since that is how the Unix system
gets the network information from its kernel. You will also want
tcpslice (tcpslice.tar.Z). These software packages have been made
available by the Network Research Group at the Lawrence Berkeley
Laboratory. Both the sensor and the analysis station require
tcpdump and libpcap.
Action 1.2 Acquire secure shell.
If you are not already using secure shell, visit the "getting
started with secure shell" home page at
http://ns.uoregon.edu/pgpssh/sshstart.html to obtain the
software.
Action 1.3 Obtain the SHADOW code. The remaining intrusion
detection software is available at:
http://www.nswc.navy.mil/ISSEC/CID/step.tar.gz
Action 1.3 Download and build Apache
The Apache web server code is included in many Unix
distributions, but it is probably a good idea to download the
latest code from the Apache software distribution.
http://www.apache.org
=======
IMPORTANT NOTE: Appendix A has Red Hat 5.1 installation instructions for
either a sensor or analysis system. If you need greater detail than
provided in step's 2 - 4 you may wish to consult Appendix A.
======
Step 2. Build the sensor.
The first system to build is the hardware/software facility that will
serve as the sensor.
Action 2.1 Obtain a computer running Unix. Tcpdump compiles on
multiple Unix systems, if the network you are protecting doesn't
have a great deal of traffic, older "obsolete" workstations may
be ideal. One group responsible for protecting multiple sites was
able to collect a large number of Sun sparc II "pizza boxes".
They fit nicely in the equipment racks and were able to handle up
to full T1 speeds quite well. However if you have a high traffic
network you should consider using FreeBSD or BSDI Unix. FreeBSD
is considered the fastest, or most efficient, free platform for a
SHADOW sensor due to FreeBSD's implementation of BPF (Berkeley
Packet Filter). You should have at least 32MB of RAM and 9 GB of
disk space. The machine should be dedicated to running the
intrusion detection software.
Action 2.2 Partition a large disk.
Since the primary purpose of the system is to collect data you
want the /LOG partition to be as large as possible.
Action 2.3 Build libpcap.
We start with libpcap. The Unix command to unpack the software
is:
$ uncompress libpcap.tar.Z
The result of this operation should be a file called libpcap.tar.
Next, type:
$ tar xf libpcap.tar
Unpack all the files that came in the software distribution and
place them in the directory you are working in. Then compile
libpcap. Read the documentation that came with the software. The
high points are to run configure and make.
Action 2.4 Compile tcpdump. Follow the same steps as in Action
2.3 to uncompress, extract the tar file, configure and make to
build tcpdump.
Step 3. Configuring the Sensor
In this section, we describe the actions necessary to install programs that
support tcpdump and allow it to run. They include cron, tcpdump.driver.sh,
start_logger.sh and stop_logger.sh.
Action 3.1 Partition a large disk. Since the primary purpose of
the system is to collect data you want the /LOG partition to be
as large as possible.
Action 3.2 Unpack the SHADOW tar file. The files in the sensor
directory are needed to build the sensor. Copy these files to the
sensor's /usr/local/logger directory.
Action 3.3 Secure the sensor. The sensor will be located outside
the firewall and the sensor programs run as root. Disable all
network services except secure shell. The SHADOW programs expect
secure shell to be present for secure file transfer. Make sure
all available security patches for your system are installed. If
you are unsure about what security patches are needed, a good
reference site to check is http://www.iss.net/xforce/ The sensor
should only be reachable by the analysis system and perhaps a
backup trusted system! The sensor operating system should have
the minimal possible information about the internal network.
Check /etc/hosts, /etc/hosts.equiv, /etc/networks, /.rhosts
yellow pages and any other files that might have configuration
information about your network.
Action 3.4 Configure cron. The Unix scheduling system, named
cron, automatically runs programs , often shell or perl scripts,
at a certain time.
Cron requires a particular format; the line we added to our cron
file is below:
0 * * * * /usr/local/bin/tcpdump.driver.sh > /dev/null 2>&1
This entry tells cron to execute a shell script named
tcpdump.driver.sh at the zero minute of every hour.
Action 3.5 Review the activities of tcpdump.driver.sh. This is
the program, run every hour by cron, that sets up some variables,
calls one shell script to stop the current tcpdump, run and then
calls another shell script to start tcpdump again.
Stop_logger.sh is the shell script that stops the old tcpdump
file and starts a new one every hour. Experience shows that one
hour is a long time to run tcpdump. Now that we have stopped the
logger, we must start it again. This is what start_logger.sh
does.
The other program, Cleanup.pl, runs on the analysis station
rather than the sensor. It is called by cron and its job is to
keep the sensor's disk from filling up by deleting older tcpdump
files. We store three days worth of data on the sensor, but this
is configurable.
Action 3.6 Test the tcpdump.driver.sh script. Run the script from
the command line. Check the process table (ps ax on linux, ps -ax
on SUNOS, ps -ef on ATT variants of Unix). Check to see if
tcpdump is running. Go to the /LOG directory and check to see
whether the log file is there and growing. If it is, you are
probably in good shape.
Action 3.7 Final legal review. Deploying a sensor should only be
done with approval from your organization's legal staff. All the
computer systems will need warning banners and all users should
be notified before the system is placed in operation. An SOP
detailing appropriate and inappropriate uses of the data is
recommended.
That is all there is to it. A spare Unix platform, tcpdump, a
large disk and a couple shell scripts and you have an intrusion
detection sensor.
Step 4. Set up the Analysis Station
In this section we describe methods for getting the data from the sensor to
the analysis station securely
Action 4.1 Configure the firewall to allow secure shell initiated
from the analysis station inside the firewall to copy the data
from the sensor to the analysis station. No other system except
the sensor should be allowed to access the analysis station
through the firewall.
Action 4.2 Edit cron to call fetchem.pl
Here is a cron example
7 * * * * /usr/local/bin/fetchem.pl > /dev/null 2>&1
Fetchem.pl's job is to pull from the sensor machine the raw
tcpdump file, store it in a descriptive directory, and run it
through tcpdump again with a filter file called "bad_events"
which will detect those events you suspect may be intrusion
attempts. The output from this tcpdump filter run is written to a
file that happens to be within the domain of an http web server.
Consequently, pointing a web browser to this domain allows an
analyst to quickly review any findings of the tcpdump filter run.
We will describe the format and usage of tcpdump filters in the
next section.
Action 4.3 Install libpcap, tcpdump and tcpslice on the analysis
station.
Action 4.4 Create a test bad_events file. Section 5 discusses
filters including a complex filter called bad_events. For test
purposes you may want to temporarily use a simple inclusive
filter for testing. A file named bad_events with the following
contents:
tcp and (tcp[13] & 2 != 0)
The first tcp in the example above means match TCP packets (as
opposed to UDP, ICMP, IGRP, IGMP etc). The "tcp[13]" refers to
the 13th byte of the tcpheader. This is the location of the TCP
"CODE BITs" field. The "& 2 != 0" is how we mask this field to
test for SYN. So what this does is causes all the TCP packets
with SYN set to be written to a file. Several reviewers felt that
just SYN and FIN packets could be used to "define" TCP
connections. This could be accomplished by: tcp and (tcp[13] & 3
!= 0).
============
(Begin Sidebar - To run alongside paragraph above)
TCP CODE BITS
Often called flags, the six bits in this field are used to interpret how to
handle a TCP packet. They are:
URG Urgent pointer field is valid
ACK Acknowledgement field
PSH Push requested
RST Reset (break) connection
SYN Synchronize (start) connection
FIN Finished, all done
(End Sidebar)
============
Action 4.5 Test fetchem.pl. Run fetchem interactively and see
whether the files are correctly being copied from the sensor into
the /LOG directory.
Step 5. Use tcpdump and its filters
This section is a step-by-step guide for learning the language you will use
to control tcpdump's filtering activities. The man page for tcpdump is an
excellent resource. To get the most out of your system you will want to be
able to customize your filters. This section is organized to teach the
filter language using simple filters that can actually be used for
intrusion detection, next we show a more complex filter developed by
stringing simple filters together. Finally, we present a complete list of
events one might wish to monitor for. A sample complete filter is included
with the SAHDOW software which is ready to be customized for use at your
site.
Tcpdump can operate in two modes: raw packet collection and packet
filtering. The sensor merely reads the raw packets from the network and
saves them into a file. The analysis station reads the packet files and
applies filters to look for specific patterns. Lets take a look at examples
of simple filters that we have found to be useful. Simple filters can be
strung together with "and", "or" and so forth to make a more complex
filter. We will also have an example of a complex filter.
NOTE: This is a good point to advise readers that a beginning intrusion
detection analyst should be very slow to panic! IMAP, PORTMAP and so forth
are common attacks, but are also commonly used services. Take some time to
learn how your site does business network wise. Of course, once you start
looking, it is probable that you will detect some attempted attacks.
Example 5.1: Simple filter to detect telnets.
Since the files are so large, especially on busy networks, the
files are kept compressed. So the first thing we will do in
constructing our filter is call GNU Unzip. Here is the whole
command:
$ gunzip -c yourlogfile.gz | tcpdump -r - "tcp and dst port 23"
tcpdump -r - tells tcpdump to read from a file instead of a
network interface and the trailing dash means standard input (the
output of gunziping the logfile). Our filter is in quotes. For
the remainder of these filter examples, we will provide only the
filter itself rather than the whole command line.
Example 5.2 A filter for IMAP.
The following filter scans packets looking for tcp SYN packets
sent to destination port 143, the IMAP service. IMAP is a very
common attack.
tcp and (tcp[13] & 2 != 0) and (dst port 143)
This filter above is pretty simple. The term "dst port" refers to
the destination port. Most Unix computers have a file called
/etc/services which provide names for these numeric ports. For
instance email (smtp) is TCP port 25, telnet is TCP port 23 and
so forth. IMAP as we mentioned earlier is port 143.
============
(Begin Sidebar - Please run alongside filter above)
More about SYN and FIN
Many sites block IMAP (TCP Port 143) because a vulnerability in this
service was one of the most successful attack exploits ever. If the service
is blocked all that will be detected is the connection request (active
open), which is a packet with the SYN flag set and no ACK. One variety of
IMAP exploit script has been detected in which the packets have both the
SYN and FIN flags set. The purpose of this may be to avoid completing the
TCP three way handshake and therefor possibly avoid detection. By masking
tcp[13] with & 3 != 0 you get both the SYN and FIN, if you use & 2 != 0 you
will only get SYN.
(End Sidebar)
============
If the filter finds the target pattern in a data file, tcpdump
will display the results. Let's begin to look at some sample
output!
First here is our file-naming format.
98 = The year, 1998, we aren't doing date mathematics so 2000
isn't an issue
01 = The month, January
21 = The day
16 = The hour in military time. Remember, we chose to roll the
files over hourly.
In the files created by the filters there will be a lot of lines
with a lot of information. As we continue to work through the
examples in the book, this format will become very familiar. The
primary fields are listed below :
Time Sourcehost.sourceport > Desthost.destport
Here is an example from a file named 98012116.txt:
16:33:14.296403 linux.zsagvari.c3.hu.35486 > 204.34.256.1.imap: S
0:0(0) win 512
Example 5.3 ICMP filter
Here is another simple filter from the tcpdump man page to look for all
ICMP packets that are not echo requests/replies (i.e., not ping packets).
icmp and icmp[0] != 8 and icmp[0] != 0
ICMP message types are not listed in /etc/services. However you
may find a helpful file in /usr/include/netinet/ip_icmp.h that
defines the names of the ICMP message numbers. ICMP, the Internet
Control Message Protocol, is not exploited to break in to your
site's computer systems. However, it is being used, for numerous
denial of service attacks. ICMP was designed as a network health
indicator and if you were to suddenly see a lot of "TIMEXs" Time
exceeded, or UNREACH, (net, host, protocol ... unreachable),
SOURCEQUENCH this could indicate network problems.
NOTE: Time exceeded and port unreachable messages are also a
potential result of running the traceroute program from one of
your site's host computers. Traceroute sends out packets to probe
for the identities of all the routers along a network path and
gathers the data it needs from the ICMP messages.
Example 5.4 Construct a filter to detect broadcasts:
ip and ip[19] = 0xff
One of the classic attacks with ICMP, in which very large ping
packets are fragmented, is often combined with a broadcast. After
all, why go for a system when you can go for a subnet? Also many
automated scanners send probes to broadcast addresses, this
filter will detect these as well. A filter like the one above
will detect the broadcast ICMPs, or any other broadcasts. Here is
sample output from a filter match for ICMP:
16:00:03.828071 fraggg.org > 128.256.15.255: (frag
27392:548@1480)
16:00:03.896593 fraggg.org > 128.256.1.255: (frag 21248:548@1480)
16:00:06.118729 fraggg.org > 128.256.15.255: icmp: echo request
(frag
52480:1480@0+)
16:00:06.250349 fraggg.org > 128.256.15.255: (frag
52480:548@1480)
(Fraggg.org could well be spoofed. When you send an ICMP echo
request (ping) to a subnet broadcast address e.g.172.16.1.255,
all the systems on 172.16.1 are supposed to answer the computer
that "pinged" them. If many subnet addresses are pinged and the
source address is spoofed, the ensuing flood of ICMP echo replies
can flood fraggg.org's internet connection.)
============
(Begin Sidebar - Please run alongside material above)
Directed broadcast.
If you see this type of broadcast on your network, it is likely
that your router, or your internet service provider's router, is
configured to allow the "directed broadcast" option. We recommend
that you change the router configuration to disable this. For
further information please see:
http://www.quadrunner.com/~chuegen/smurf.txt
(End Sidebar)
============
Example 5.5 Land attack filter
Speaking of denial of service, it turns out that some computer systems will
freeze if they receive a packet in which the source address is spoofed to
be the same as the destination address. Here is a filter to detect this:
ip and ip[12:4] = ip[16:4]
Example 5.6 A filter designed to detect SNMP
(udp port 161 or udp port 162) and not src net 172.17
SNMP was developed for network management. However, adversaries
of your organization can use SNMP to collect a lot of information
about your networks and computer systems. Network appliances,
such as routers, hubs and bridges, devices often have SNMP agents
built in but, in addition, many other devices, such as print
servers and X terminals, have built-in SNMP agents. These devices
use a "community string" to control access. Many SNMP agents
default to a community string called public, which means just
that. Please note that the example about lists an exception, if
the source network is 172.17.anything it will not be matched. It
is common for a site to allow connections of a certain type from
another trusted network, but not to allow them in general.
Example 5.7 A filter to watch for the r-utilities:
ip and ( tcp dst port 512 or tcp dst port 513 or tcp dst port 514)
The r-utilities, rlogin, rshell and so forth, allow two trusted
systems to exchange files and commands without authentication.
Ideally, systems that need to do this type of trusted exchange
across the internet will change to secure shell or some other
more secure mechanism. Two problems arise with the use of
r-utilities: (1) /etc/hosts.equiv and (2) individual user's
.rhosts files. If a system has a "+ +" in its host.equiv, that
means it trusts all users from all systems. Needless to say, it
is wise to look out for packets with r-utilities from unknown
sites.
Example 5.8 A filter to detect access to portmapper:
ip and dst port 111
Many /etc/services file call this sunrpc. RPCs tend to connect to
portmapper at either TCP or UDP 111 to find other services. This
is a very old (and effective) gateway to a series of attacks. In
late 1997, we started to see an increase in portmap attempts.
From 98012403.txt:
03:20:42.579548 netgate.srn.com.829 > ns2.nnnn.navy.mil.sunrpc: S
3648872793:3648872793(0) win 512 <mss 1460>
03:20:45.547040 netgate.srn.com.829 > ns2.nnnn.navy.mil.sunrpc: S
3648872793:3648872793(0) win 31744 <mss 1460>
Note the `S', or SYN packet flag. A firewall screens the DNS
server ns2.nswc.navy.mil so that the sunrpc attempt never
actually reaches it.
Example 5.9 An NFS filter
NFS is a good service to keep an eye on, and here is a filter to do it:
ip and udp port 2049
05:17:50.562188 j.K.EDU.885 > dorad.nnnn.navy.mil.nfs: 40 null
17:52.553265 j.K.EDU.885 > dorad.nnnn.navy.mil.nfs: 40 null
05:17:56.551772 j.K.EDU.885 > dorad.nnnn.navy.mil.nfs: 40 null
Not all "hits" are intrusion attempts. Our assessment of the data
series above is that it shows the results from an automated
process with a typographical error that happens to be dorad's
internet address. Also NFS can run on TCP so you might wish to
modify the filter to say:
ip and udp port 2049 or ((tcp[13] & 2 != 0) and (dst port 2049))
or ip and udp port 2049 or tcp[2:2] = 2049
NOTE: 2049 may be NFS and it may be used for something else.
Don't panic the first time you run this!
Example 5.10 A NetBIOS filter.
Microsoft Workgroup for Windows, Windows 95, Windows NT and SAMBA all use a
protocol called NetBIOS to communicate over the internet. This protocol
uses ports 137, 138, 139 of both TCP and UDP. Let's build this filter one
part at a time:
ip and
Will match both TCP and UDP Now we need to match the port
numbers: 137, 138, 139
port 137 or port 138 or port 139
Add parentheses for the precedence challenged
ip and (port 137 or port 138 or port 139)
If the destination is NetBIOS nameservice (port 137) and source
port is not 137 the connecting computer is probably SAMBA, under
most circumstances you probably don't care, but if you do:
ip and dst port 137 and not src port 137
Example 5.11 An X11 filter. X11 is a protocol that allows one to run a
program on one Unix host and display the results in a window on a separate
Unix workstation. If configured incorrectly, it is possible for an attacker
to obtain a screen dump of what is currently shown on your workstation,
monitor all of your keystrokes, and even insert their own keystrokes into
some software that you may be running. X11 uses TCP port 6000 and sometimes
the nearby higher ports. An example filter for this would be:
tcp and (port 6000 or port 6001 or port 6002)
or, perhaps it would be better to write it this way:
tcp and (tcp[13] & 2 != 0) and (dst port 6000 or dst port 6001 or dst port
6002)
Example 5.12 An IRC filter.
IRC is a real-time chat system that runs over the Internet usually using
TCP port 6667. When a Unix system is compromised by an attacker, it is very
common for the attacker to install an IRC server on that computer --
sometimes they will then use the server to chat with their cohorts about
other attacks. So, even if your organization makes absolutely no use of
IRC, it is useful to monitor for it since it can be a warning sign of a
compromised machine. A filter for this would be
tcp and port 6667
Example 5.13 An NNTP filter.
NNTP is the protocol used for exchanging the messages that appear in Usenet
newsgroups. If you have an NNTP server on your network that communicates
with an external server (often the NNTP server of your Internet provider)
you will undoubtedly see a huge amount of traffic for its TCP port number,
119. If you do not exchange messages with the outside world via NNTP, you
still may see occasional packets, possibly from attackers that want to
break into your server, but possibly also from attackers seeking sensitive
information from your private internal newsgroups.
A filter for NNTP would be
tcp and port 119
Example 5.14 A filter to detect fragments.
Fragmentation is sometimes used to evade intrusion detection systems.
Excessive fragmentation can also indicate problems with network
configuration. NOTE: sites that do NFS over the internet probably do not
want to run this filter as NFS likes large data chunks.
A filter to detect fragments would be
ip[6:2] & 0x2000 != 0
Example 5.15 A filter to detect socks.
Socks runs on TCP port 1080. We have detected several probes to 1080. One
CERT advises that there are exploits that allow an attacker to establish a
connection to port 1080 and then bounce out from the system to attack or
probe another computer. This way the attack would look like it comes from
your network. It is important to detect SYN attempts to 1080 since it is a
commonly used port for FTP and HTTP port counting.
A filter to detect socks would be
tcp and (tcp[13] & 2 != 0) and (dst port 1080)
Example 5.16 Construct a bad_events filter.
Sophisticated filters can be constructed to scan for any set of events you
want to detect. For example, here's a portion of a script called
"bad_events" that is used to detect any packets that could indicate
suspicious activity that might warrant further attention. In the next
section we provide all the filter "atoms" we were using the last time we
froze the code. The purpose for the example filter shown here is to
illustrate how the atomic filters we have been learning to use can be
strung together with ands and ors.
If you find the filter syntax to be a bit tricky, you'll need access to a
book called Internetworking with TCP/IP Volume I by Douglas E. Comer,
before you can do anything really fancy. The book will be a good investment
for your organization. TCP/IP Illustrated, Volume 1 by Richard Stevens is
another excellent resource which gives examples using tcpdump output. In
fact someone you know probably already has these, so try to borrow one
until your copy comes in. Here is an example filter using atomic filters
strung together:
(tcp and (tcp[13] & 3 != 0) and ((dst port 143) or (dst port 111)
or
(tcp[13] & 3 != 0 and tcp[13] & 0x10 = 0 and dst net 172.16 and
dst port 1080) or
(dst port 512 or dst port 513 or dst port 514) or
((ip[19] = 0xff) and not (net 172.16/16 or net 192.168/16)) or
(ip[12:4] = ip[16:4]))) or
(not tcp and not igrp and not dst port 520 and
((dst port 111) or (udp port 2049) or
((ip[19] = 0xff) and not (net 172.16/16 or net 192.168/16)) or
(ip[12:4] = ip[16:4])))
Example 5.17: Putting the pieces together.
Now that we have reviewed the syntax of the individual filter pieces and
seen an example of a composite bad events filter, lets take a closer look
at building a real system to use for automatic network monitoring.
The first problem we have to address is that several of the tcp filters
above will give a lot of false alarms if users on your network are doing
http, ftp, or exchanging email with the outside world and these are normal
activities! For example, the socks filter looks for tcp packets having
either source or destination port 1080 where the syn flag is set. If the
composite filter given in the previous example is used, the filter will
grab tcp packets involving port 1080 where either (or both) the syn and fin
flags are set. Usually, we are interested in "active open" connections,
i.e. the first connection in the tcp 3-way handshake. If we use the filters
as given above we will false alarm on "passive open" tcp connections. For
example, say a user on your network accesses a web page on some remote
server. They might connect (the active open) to port 80 on the webserver,
and use a source port of 1080. When the webserver responds back
acknowledging the syn it received and sending its own syn (the passive
open) it will connect to the client machine on port 1080 and use a source
port of 80. Notice that the passive open connection will meet the criteria
of the filter we have discussed - the destination port is 1080 and the syn
flag is set; however, this connection is clearly not the result of an
attacker trying to execute an exploit on tcp port 1080. Further, if we do
not account for passive open connections, any attempt to look for "unknown"
tcp port accesses (by defining what is "known" and looking for the rest)
will be foiled by false alarms as the source port for an active open is
typically chosen at random.
The solution to this problem is simple - we look for tcp packets where the
syn flag is set but the ack flag is not:
tcp and (tcp[13] & 2 != 0) and (tcp[13] & 0x10 = 0)
and note that this filter will also pick up connections where the syn and
fin flags are set simultaneously.
If we wanted to write a filter that specifically looks for tcp connections
where the syn and the fin flags are set simultaneously, we could use the
following.
tcp and (tcp[13] & 2 != 0) and (tcp[13] & 1 != 0)
In fact, we have recently seen connections coming to our networks having
the syn, fin, ack, push, reset, and urgent flags set at the same time. If
you are looking for new attack patterns it could be educational to screen
for inbound packets having weird combinations of these flags set.
The list below gives many possible "events of interest" that one might
screen for. These components can be logically grouped together to yield a
sophisticated filtering capability. At our site, we build several filters
up from these components and then use a shell script loop to call the
filters in sequence. This is done because there is a finite number of
components that tcp dump can handle in a single filter. Of course, some of
the filters may overlap and a given packet might be extracted from the data
by more than one filter, so the shell script loop is followed by a call to
a perl program that sorts and groups the filter outputs, throwing away
duplicate connections. The filters we have built are available at the
website, as are perl programs we use in concert with the tcpdump output to
provide more advanced capability. Some advanced techniques are discussed in
Step 7.
IP filters
-------------
#land attacks: source ip = dest ip
ip[12:4] = ip[16:4]
#broadcasts to x.x.x.255
ip[19] = 0xff
#broadcasts to x.x.x.0
ip[19] = 0x00
#fragmented ip packets with more fragments coming
ip[6:1] & 0x20 != 0
#fragmented ip packets with zero offset (first in sequence)
ip[6:2] & 0x1fff = 0
#unroutable ip addresses: 0.x.x.x, 127.x.x.x, 1.x.x.x, 2.x.x.x, 5.x.x.x,
# 240.x.x.x thru 255.x.x.x
net 0
net 127
net 1
net 2
net 5
ip[12] > 239
#ip packets with options set (header size is bigger than 5 32-bit words)
(ip[0:1] & 0x0f) > 5
# if want to check for specific options, check ip[20:1] (given
that (ip[0:1] & 0x0f > 5) is satisfied):
option ip[20:1]
------- ---------
record route 7
timestamp 0x44
loose source routing 0x83
strict source routing 0x89
TCP filters
--------------
#active open packets (syn is set, ack is not)
(tcp[13] & 2 != 0) and (tcp[13] & 0x10 = 0)
#destination ports less than 20
tcp[2:2] < 20
#source ports less than 20
tcp[0:2] < 20
#telnet (look for these from undesirable sources or to machines or
# subnets that shouldn't be receiving telnets)
dst port 23
#whois
dst port 43
#dns zone transfer
dst port 53
#gopher
dst port 70
#finger
dst port 79
#link (hacker signature port)
(dst port 87) or (src port 87)
#supdup (hacker signature port)
(dst port 95) or (src port 95)
#sunrpc (portmapper)
dst port 111
#nntp (look for these to non-news servers)
dst port 119
#epmap - DCE Endpoint resolution
dst port 135
#profile - PROFILE naming system
dst port 136
#netbios name service
dst port 137
#netbios datagram service
dst port 138
#netbios session service
dst port 139
#imap
dst port 143
#NeWS window system
dst port 144
#exec
dst port 512
#login
dst port 513
#shell
dst port 514
#printer spooler (look for unusual activity)
dst port 515
#uucp
dst port 540
#doom
dst port 666
#kerberos admin
dst port 749
#loadav
dst port 750
#pump
dst port 751
#socks
dst port 1080
#openwin (like X11)
dst port 2000
#nfs
dst port 2049
#listen - the System V listener
dst port 2766
#internet relay chat (sometimes 6668, 6669 and so forth are also used for
IRC
# especially with busy servers)
dst port 6667
#X11 ports
(tcp[2:2] >= 6000) and (tcp[2:2] < 7000)
#eleet (hacker signature port)
(dst port 31337) or (src port 31337)
#syn and fin flags are set simultaneously
(tcp[13] & 2 != 0) and (tcp[13] & 1 !=0)
#urgent flag is set
tcp[13] & 0x20 !=0
#look for anything unknown, please notice the "not"
not ((tcp[2:2] < 20) or
dst port 21 or
dst port 23 or
dst port 22 or
dst port 25 or
dst port 37 or
dst port 43 or
dst port 53 or
dst port 70 or
dst port 79 or
dst port 87 or
dst port 95 or
dst port 109 or
dst port 110 or
dst port 111 or
dst port 113 or
dst port 119 or
dst port 135 or
dst port 136 or
dst port 137 or
dst port 138 or
dst port 139 or
dst port 143 or
dst port 144 or
dst port 443 or
dst port 512 or
dst port 513 or
dst port 514 or
dst port 515 or
dst port 540 or
dst port 563 or
dst port 666 or
dst port 749 or
dst port 750 or
dst port 751 or
dst port 1080 or
dst port 1352 or
dst port 1452 or
dst port 1494 or
dst port 1521 or
dst port 1526 or
dst port 2000 or
dst port 2766 or
dst port 6667 or
dst port 31337 or
dst port 80 or
dst port 8000 or
dst port 8080)
UDP filters
-------------
#destination ports less than 20
udp[2:2] < 20
#source ports less than 20
udp[0:2] < 20
#bootps - Bootstrap Protocol Server
dst port 67
#bootpc - Bootstrap Protocol Client
dst port 68
#tftp
dst port 69
#kerberos
dst port 88
#sunrpc (portmapper)
dst port 111
#epmap - DCE Endpoint Resolution
dst port 135
#profile - Profile Naming System
dst port 136
#netbios datagram service
dst port 138
#netbios session service
dst port 139
#imap
dst port 143
#snmp
dst port 161
#snmp-trap
dst port 162
#xdmcp - X11 logins
dst port 177
#biff
dst port 512
#who
dst port 513
#syslog
dst port 514
#talk
dst port 517
#ntalk
dst port 518
#doom
dst port 666
#socks
dst port 1080
#nfs
dst port 2049
#eleet (hacker signature port)
(dst port 31337) or (src port 31337)
#traceroutes (dest port between 33000 and 33999, analyst should
#look for unchanging source port)
(udp[2:2] >= 33000) and (udp[2:2] <= 33999)
ICMP filters
--------------
#error messages: fragmentation needed but DF flag set
# (look for outbound)
(icmp[0] = 3) and (icmp[1] = 4)
#error messages: source route failed (look for outbound)
(icmp[0] = 3) and (icmp[1] = 5)
#fragmented icmp packets (look for inbound)
icmp and (ip[6:1] & 0x20 != 0)
Step 6. Display the information for maximum analytical value
Let's consider what you have done so far. You have set up the sensor in the
DMZ. You have collected data; yes, a lot of data. You have examined this
data with filters.
Sadly, when the files get very large, it takes increasingly large amounts
of time to parse the files with filters. And when tasks take time and
effort, many people just quit doing those tasks. For this reason, how we
display the information is every bit as important as what we filter for.
One of the advantages to the SHADOW architecture is that a single analysis
station can support multiple sensors (make sure you stagger the file
downloads in the cron files so that you aren't filtering the sensor data
from multiple sensors at the same time).
Action 6.1 Introduction to the display system
Fetchem.pl writes the information from filter matches into html
format into a directory that is served by the web server. People
responsible for watching the network simply use their web browser
to see what is going on. The web files with events flagged by the
bad_events filter are chained by forward and back arrows, so the
analyst can go from hour to hour easily.
Some additional notes on display of information:
Dir-it is the program that creates and updates the home page which is an
index to all the data. This was written by Robert Niles and we have
included it in the source distribution.
There are many web servers available. The one that we have used for
displaying intrusion detection information for several years is called the
Apache web server. It is available from http://www.apache.org.
Action 6.2 Using a "bad_events" filter to survey your network
Intrusions are detected exactly one way. By one technique or
another they are isolated from the traffic stream and found to be
interesting (abnormal), non-business related traffic. The
"gimmes" in intrusion detection for a traffic analysis approach,
are the services that could not possibly be part of your
organization's normal business related traffic. As we discussed
in the filter section, in 1997 the IMAP attack was a very
successful exploit. What if your site did not support an IMAP
service and yet you kept seeing incoming packets to TCP port 143?
So then, we need to determine what services we do support so that
we can flag attempts to other services as suspect. How do we do
this?
A great way to start is to locate your organization's firewall
policy. This way you can determine the things your organization
has decided to allow. Remember the great axiom of firewall
policy: That which is not specifically allowed is prohibited.
Therefore all TCP and UDP ports below 1024 that are not allowed
should be flagged with the bad events filter. All TCP ports above
1024 that are not allowed can be flagged if there is a SYN packet
to that port and all UDP ports above 1024 to ports that are not
allowed should be flagged.
If your organization does not have a firewall, or the firewall is
extremely permissive you will need to write a set of filters that
will let you examine all traffic and the determine statistically
which ports are normal business traffic and then monitor for
exceptions. For TCP:
((tcp and (tcp[13] & 2 != 0) and dst net 192.168 and dst port <
1024)
This filter will list all SYN packets, or attempts to open a
connection to a port below 1024 for the destination network
192.168 (Please put your network here). If you want to get all
incoming connections, simply modify the filter to say:
((tcp and (tcp[13] & 2 != 0) and dst net 192.168)
=============
(Begin Sidebar - Please run alongside the two paragraphs above)
TCP and UDP services can be divided into two categories, below
1024 and above. The reasons for this are rapidly becoming
historical. On Unix systems, root level access was required to
operate a service below port 1024. The idea was that such a
service could be trusted since anyone with root access was
presumably trustworthy. Further, the ports below are fairly well
defined and used appropriately, that is TCP port 25 is very
likely to be sendmail, 23 telnet and so on. Above 1024 it gets
pretty dicey. For instance TCP 1080 may be for socks, but if you
want to detect a probe on socks you will need to look for a SYN
packet (active open) to TCP 1080, otherwise you will detect a lot
of perfectly normal FTP and HTTP file transfers.
(End Sidebar)
=============
Action 6.3 Beginning intrusion detection
Now that you have completed surveying your network traffic
and created a bad_events filter to satisfy your needs, you can
begin using this tool for intrusion detection. The software will
place the files in a directory arranged by hour. As you click on
each hour, you will see the events that were flagged during that
time period. If you want further information on a particular event,
you may want to run one of the analysis scripts supplied with your
software. For example, one of the most common and useful tools is
the "one_day_pat.pl". You can execute the "one_day_pat.pl",
providing it a date and a pattern, and it will check all the hourly
files for the pattern over the entire day specified and output this
information appropriately.
NOTE: one_day_pat.pl can be run through the web gui or from a
command prompt shell. If you run it from the web interface the
date (-d) and the sensor or logger (-l) information is filled in
for you. If you choose to run one_day_pat.pl from the command
prompt you will need to supply these command line options, or the
software will make a guess as to what you want which may or may
no be correct. There is one additional option that can be
provided -n will avoid name lookups which can be much faster.
For instance, if you check the hourly files and see an entry
concerning srn.badguy.org and want more information about
the actions involving srn.badguy.org for the day, try:
$ one_day_pat.pl -d 980520 -l NNNN -p 'host srn.badguy.org'
and all traffic for that day to or from srn.badguy.org will be
displayed. This script is also helpful to test for events that
are not in your bad_events pattern.
NOTE: The web page only displays the events
matched by the bad_events filter or the additional perl scripts
will be displayed. From time to time, we recommend that you try
experimental filters to see
what you can detect. Networks are dynamic and it is critical to constantly
monitor for change.
What are some common attacks to detect, you might ask? It is
sometimes difficult to tell, however, some attacks are quite
noticeable. For example, an attacker might try a subnet, host,
or port scan in order to map parts of your network. This type
of attack can be observed by noting the changes in subnet, host,
or port portion of the destination IP address. A port scan may
look something like this:
12:00:01.276556 srn.badguy.org > 128.256.15.42.21: S
3201421942:3201421942(0) win 512 <mss 1460)
12:00:01.277237 srn.badguy.org > 128.256.15.42.22: S
1720311009:1720311009(0) win 512 <mss 1460) 12:00:01.294142 srn.badguy.org
> 128.256.15.42.23: S 1104712552:1104712552(0) win 512 <mss 1460)
12:00:01.315076 srn.badguy.org > 128.256.15.42.24: S
3049097370:3049097370(0) win 512 <mss 1460)
12:00:01.371799 srn.badguy.org > 128.256.15.42.25: S 58837740:58837740(0)
win 512 <mss 1460)
12:00:01.372513 srn.badguy.org > 128.256.15.42.26: S
1186269549:1186269549(0) win 512 <mss 1460)
12:00:01.373306 srn.badguy.org > 128.256.15.42.27: S
2264340599:2264340599(0) win 512 <mss 1460)
This attack is easy to notice since the scan proceeds step by
step with an increment in the destination IP address port number
for each call. However, attacks may not be this easy to spot.
Attackers may use different host names, attempt to collect
information at different time intervals, change their method of
attack, etc. in order to try to avoid detection. It is useful to
know which subnets, hosts, and ports are important to your work,
and pay careful attention to users attempting to access these
systems. For instance, attackers attempting to connect to ports
such as FTP, TELNET, DOMAIN, SUNRPC, NETBIOS, IMAP, and other
important ports that you implement should be observed and reported.
Lets look at the following example:
12:00:01.614233 srn.badguy.org.0 > 128.256.15.0.53: SF
2617442304:2617442304(0) win 512
12:00:01.629325 srn.badguy.org.0 > 128.256.15.1.53: SF
2617442304:2617442304(0) win 512
12:00:01.653388 srn.badguy.org.0 > 128.256.15.2.53: SF
2617442304:2617442304(0) win 512
12:00:01.670736 srn.badguy.org.0 > 128.256.15.3.53: SF
2617442304:2617442304(0) win 512
12:00:01.692327 srn.badguy.org.0 > 128.256.15.4.53: SF
2617442304:2617442304(0) win 512
12:00:01.711127 srn.badguy.org.0 > 128.256.15.5.53: SF
2617442304:2617442304(0) win 512
12:00:01.729200 srn.badguy.org.0 > 128.256.15.6.53: SF
2617442304:2617442304(0) win 512
12:00:01.749778 srn.badguy.org.0 > 128.256.15.7.53: SF
2617442304:2617442304(0) win 512
12:00:01.769174 srn.badguy.org.0 > 128.256.15.8.53: SF
2617442304:2617442304(0) win 512
In the example above the attacker is sending packets to incrementing (0, 1,
2, ...) hosts on the subnet 128.256.15.x. He is attempting to connect to the
DOMAIN port (TCP port 53) on each of these hosts. In this case, the
attacker is trying to find dns zones (UDP 53 is a name lookup, TCP 53 is a
zone transfer) Also notice the attacker is sending packets with both the
SYN and FIN flags set. Please refer to Example 5.2 for more information
about this type of packet. This is an attack to report, even if the
attacker is not getting through your firewall. The fact that an attacker is
attempting to find information or weaknesses is sufficient cause to file a
report. CIRTs appreciate knowing who is probing. They can often add your
information with other information collected on attacks and probes to
determine trends. Your report may be the "missing puzzle piece". Another
example probe to report might look something like this:
12:00:43.55 srn.badguy.org.13902 > 128.256.1.1.23: S
3494448575:3494448575(0) win 512 <mss 1460>
12:00:43.55 128.256.1.1.23 > srn.badguy.org.13902: S
1160640001:1160640001(0) ack 3494448576 win 32768 <mss 1460> (DF)
12:00:43.63 srn.badguy.org.13903 > 128.256.1.105.23: S
3466349419:3466349419(0) win 512 <mss 1460>
12:00:43.63 128.256.1.105.23 > srn.badguy.org.13903: S
2737045410:2737045410(0) ack 3466349420 win 8760 <mss 1460> (DF)
12:00:45.74 128.256.1.1.23 > srn.badguy.org.13902: S
1160640001:1160640001(0) ack 3494448576 win 32768 <mss 1460> (DF)
12:00:46.54 srn.badguy.org.13902 > 128.256.1.1.23: S
3494448575:3494448575(0) win 32120 <mss 1460>
12:00:46.54 128.256.1.1.23 > srn.badguy.org.13902: S
1160640001:1160640001(0) ack 3494448576 win 32768 <mss 1460> (DF)
12:00:46.63 srn.badguy.org.13903 > 128.256.1.105.23: S
3466349419:3466349419(0) win 32120 <mss 1460>
12:00:46.63 128.256.1.105.23 > srn.badguy.org.13903: S
2737045410:2737045410(0) ack 3466349420 win 8760 <mss 1460> (DF)
12:00:46.91 128.256.1.105.23 > srn.badguy.org.13903: S
2737045410:2737045410(0) ack 3466349420 win 8760 <mss 1460> (DF)
12:00:48.54 srn.badguy.org.14429 > 128.256.1.1.143: S
810840681:810840681(0) win 512 <mss 1460>
12:00:48.63 srn.badguy.org.14430 > 128.256.1.105.143: S
183573626:183573626(0) win 512 <mss 1460>
12:00:51.54 srn.badguy.org.14429 > 128.256.1.1.143: S
810840681:810840681(0) win 32120 <mss 1460>
12:00:51.63 srn.badguy.org.14430 > 128.256.1.105.143: S
183573626:183573626(0) win 32120 <mss 1460>
12:00:53.48 128.256.1.105.23 > srn.badguy.org.13903: S
2737045410:2737045410(0) ack 3466349420 win 8760 <mss 1460> (DF)
12:00:55.74 128.256.1.1.23 > srn.badguy.org.13902: S
1160640001:1160640001(0) ack 3494448576 win 32768 <mss 1460> (DF)
12:01:06.61 128.256.1.105.23 > srn.badguy.org.13903: S
2737045410:2737045410(0) ack 3466349420 win 8760 <mss 1460> (DF)
In this case, the attacker is trying to find out if connections are
possible through
the firewall to TCP ports 23 and 143, TELNET and IMAP respectively. Observe
the packets
returned by the attacker's target addresses. These packets indicate to the
attacker
that the target hosts support connection to port 23, TELNET, since the
returned
packets are sent to the attacker from port 23. Since no packets are
returned from
port 143, the target hosts do not support connection to the IMAP port.
It is very important to observe what you see often and
be able to detect the smallest deviance from the ordinary.
The examples given above are only a few of the many types of
Attacks or probes you may see as you track your network data. Although
you may have a lot of data to examine, you must
be careful to observe the indications of an attack and be able
to spot such an instance even if it is hiding in the middle of
a huge amount of data. Again, this is where the "one_day_pat.pl"
may become quite useful. Remember, attackers are persistent and
always trying new methods to gain access to your systems, therefore
it is vital that you be aware of anything unusual.
Action 6.4 Responding to a detect - So what if you find something?
If you are checking your hourly files and find something that looks
suspicious, but not really scary, the next logical step is to run
"one_day_pat.pl" as described above to get more information. It is
important
to keep an accurate audit trail of everything that you discover in case the
information is needed later to support recovery, or even prosecution. A
better way to run "one_day_pat.pl" is to collect its results into a file.
This can be done as follows:
$ one_day_pat.pl -d 980520 -l NNNN -p 'host srn.badguy.org' > host.time &
Then you can run
$ tail -f host.time
to view the results as they are created. The advantage of adopting this
approach is that you will have a record of every action that you took.
One helpful option you may want to include when using "one_day_pat.pl" is
the -n option. This option causes the results to be displayed as only IP
addresses, without wasting time matching IP addresses with names. If you
are only concerned with IP addresses, this option may be useful to you.
If you are in a hurry, because what you saw on your screen was fairly
scary,
then you may want to run tcpdump directly on the hourly file (without name
lookups using -n) for faster results. The following command will accomplish
this:
$ tcpdump -n -r 98051512 'host 192.168.1.1' > host.time &
$ tail -f host.time
Whether you run one_day_pat.pl or tcpdump directly on
an hourly file, now you can examine the data and decide whether you have a
problem. There are several things you may want to notice. If the packets
are only incoming without replies, then your firewall and system
protections
may be holding. If there are replies to the sending host, but they are ICMP
and
they say "port unreachable", then again you may be OK. Not great, but OK,
this is an
indicator that either the packets are getting through your firewall, or the
external interface of your firewall has knowledge of your internal network.
If you see other replies, you probably want to alert your organization's
computer incident response team. Again, it is also a good idea to set up
some sort of reporting system in which the details of an incident are
recorded for future use, you
never know when this information will be useful, whether it be evidence for
prosecution or a record for comparison with future incidents. NOTE: for
further
information on how to establish an incident handling team or how to respond
to incidents, see SANS Computer Incident Handling Step-by-Step.
Many sensors have enough power that you can run an additional session of
tcpdump with a filter to watch the attacker. NOTE: there have been cases
where an attack has come from several systems on a network, so you may want
to collect information on the entire net. The following command sequence
helps to illustrate this:
$ ssh -l root sensor
# df
# cd /LOG
# mkdir host.date; cd host.date
# tcpdump -n 'net 192.168.1' > time &
#tail -f time &
This way you will have information to provide the incident handling team.
NOTE: we strongly advise that you try this before an incident. It is
recommended that you establish a secure shell key relationship with your
incident handling team in advance of an incident so that you can transfer
incident information in an encrypted manner to protect against any
potential
network sniffers.
Enhancing your Intrusion Detection Capability
Step 7. Implement more advanced analytical capabilities.
There is a lot more to intrusion detection than looking for a few bad
events. So how can we improve the capability of our system?
Example 7.1 A filter to detect all non-smtp or domain accesses to your mail
and name server (DNS) hosts:
ip and (host mailhost.your.org or
host nameserver.your.org)
and not (dst port smtp or dst port domain)
At most sites, the systems that draw the most attacks are your DNS and
e-mail servers. They become prime targets because their IP addresses are
well advertised and because, if an attacker can gain control of these
systems, the attacker can probably control your whole site fairly soon
thereafter. In the case of these core hosts, don't just watch incoming
traffic. You should also be concerned if your nameserver or mailserver
starts establishing connections with outside hosts. This could be a sign
that your system has been compromised.
A sample filter for monitoring inbound traffic to a webserver, a
nameserver, a mailserver are given below.
One might construct similar filters for monitoring outbound traffic.
#ignore source nets that belong to protected network:
# 192.168.x.x my network
#
#
# look for unauthorized traffic to machines of interest:
#
# 192.168.1.1 webserver
# should only recieve traffic to tcp port 80 (syn only)
# ignore udp with dst port 53 or 137
# ignore icmp echo requests (8), echo replies (0),
# destination unreachable (3), and
# time exceeded (11) error messages
#
#
# 192.168.2.1 nameserver
# should only receive traffic to udp port 53
# ignore udp with source port 53
# ignore udp with dst port 137
# ignore icmp as for webserver
#
# 192.168.3.1 mailserver (smtp)
# should only recieve traffic to tcp ports 25 and 113 (syn)
# ignore udp with dst port 53 or 137
# ignore udp with src port 53
# ignore icmp as for webserver
#
#
ip and (not src net 192.168)
and
(
(dst host 192.168.1.1 and
(
(tcp and ((tcp[13] & 2 != 0) and (tcp[13] & 0x10 = 0))
and (not dst port 80))
or
(udp and not dst port 53 and not dst port 137)
or
(icmp and (icmp[0] != 8) and (icmp[0] != 0)
and (icmp[0] != 3) and (icmp[0] != 11))
or
(not (tcp or udp or icmp))
))
or
(dst host 192.168.2.1 and
(
(tcp and ((tcp[13] & 2 != 0) and (tcp[13] & 0x10 = 0)))
or
(udp and not dst port 53
and not src port 53
and not dst port 137)
or
(icmp and (icmp[0] != 8) and (icmp[0] != 0)
and (icmp[0] != 3) and (icmp[0] != 11))
or
(not (tcp or udp or icmp))
))
or
(dst host 192.168.3.1 and
(
(tcp and ((tcp[13] & 2 != 0) and (tcp[13] & 0x10 = 0))
and (not dst port 25)
and (not dst port 113))
or
(udp and not dst port 53
and not src port 53
and not dst port 137)
or
(icmp and (icmp[0] != 8) and (icmp[0] != 0)
and (icmp[0] != 3) and (icmp[0] != 11))
or
(not (tcp or udp or icmp))
))
)
Example 7.2 A bad hosts filter.
After you detect an attack from a host, you will want to add it to a bad
hosts list. Working through the SANS Institute, you may be able to find
organizations that are in similar, or complementary lines of work as yours,
and with whom you can share attack information.
Here is a filter that can help you do this, of course you have to add your
own bad hosts.
ip and (host bad.guy.org or host 192.168.1.200 or
net 192.168.4 or net 176.16.41 or net 10.1
)
Action 7.3 Sort the data for display and resolve hostnames from IP
addresses
In order to display the traffic dumped by the filters you have constructed
it is preferable to sort the traffic in some meaningful way. Tcpdump will
present the traffic to you as it occurred in time. If one is looking for
scans or single line connections amidst a torrent of denial-of-service ping
broadcasts (for example), the traffic must be sorted and grouped
intelligently to make subtleties stand out. Our current solution is to
group the connections by source IP address and insert a blank line in the
display between groups of connections originating from the same source. In
this way scans become immediately obvious, and very noisy / overwhelming
attack connections can be clustered together and set aside so that the
analyst can see the more subtle connection attempts.
The first requirement to do this is to run tcpdump with the -n option set.
This option setting tells tcpdump not to resolve the hostnames, but rather
to leave the IP addresses in dot notation. The resulting traffic may then
be parsed and sorted according to source IP (and then by time). As the
traffic is written to the output, we simply check for duplicate lines in
the sorted data and throw them away, and when the source IP changes we
insert a blank line. This simple script is available from the website.
However, being human, we would prefer to read host names as opposed to IP
addresses. Thus we may run another script that will take our sorted output
and resolve the host names from the IP addresses and re-insert them as
tcpdump would have. A short perl snippet that does the name resolution is
given below.
#! /usr/local/bin/perl
$ip = $ARGV[0];
@octets = split(/\./, $ip);
$binary_ip = pack "c4", $octets[0], $octets[2], $octets[1], $octets[3];
@hostinfo = gethostbyaddr($binary_ip, 2);
$hostname = $hostinfo[0];
print $ip, " ",$hostname,"\n";
This snippet should be executed with a single command line argument that is
the IP address in standard 255.255.255.255 type notation. The resulting
hostname is printed to the screen provided the DNS server is able to
resolve the IP address.
Action 7.4 Looking for host scans independent of protocol or attack type
One way to detect unusual activity on a network is to search for source IP
addresses that are attempting to connect with many destination hosts over a
given time period. We can find such hosts of interest independent of any
particular protocol or attack type; in fact, looking for scans in this
manner is a good way to detect new attack patterns.
For example, we have recently been observing scans of our network using tcp
reset packets. It is both time-consuming and uninteresting to monitor all
reset packets sent to all hosts on a network as this is a normal occurrence
in tcp communication. However, these connections become more interesting
when one observes a single host from the outside world sending reset
packets to many machines on your network when no other dialogue has taken
place. The best way to screen for these connections is to look for inbound
reset connections where no other traffic between the source and destination
hosts has occurred. However, such a program will have difficulty executing
fast enough to perform hour-by-hour analysis and the attack type will be
limited to resets only. The next best thing in terms of detecting this
attack is to screen for one-to-many correlations between source and
destination IP addresses; and this is a great thing to do to find new
probing or mapping techniques. Another application that readily comes to
mind is in finding scans that are embedded in a great deal of normal
traffic. For example, monitoring all udp 53 (domain name server) and udp
137 (netbios name service) connections would be at worst impossible and at
best overwhelming on large networks. However, sites are commonly scanned on
these ports where hackers are looking for nameservers or machines running
netbios. A scan-finder as discussed above is able to extract these scans
out of the noise.
The next question is how does one write a program to do such a thing that
does not take infinite time to execute. For example, we might write a
program to make a list of all the source IP addresses that are sending
connections to our protected network (tcp, udp, icmp,... everything). Then
for each source IP in the list we would make another list of all the
destination hosts that are connected to from the source. Using this scheme,
each time a new connection is considered, we must search the list of source
IPs to see if the source has been added to the list already. Then, if the
source host is already listed, we must search the corresponding list of
destination IPs to find if the current destination host has already been
recorded. In the beginning, while the lists are short, this process
proceeds quickly. But, as the list of source addresses grows the search
becomes slower; similarly, in the case of a large network scan, the list of
destination addresses (corresponding to a given host) grows very large thus
slowing the search.
Our solution to this problem is given at the website.
The basic idea is the following:
First we filter the raw data using tcp dump. We grab all inbound traffic
except tcp connections having source port 80. We consider only inbound
connections in order to reduce the amount of traffic that we have to
process. We exclude tcp connections having source port 80 to avoid
mistaking the connections back from a commonly accessed webserver for a
scan. The tcpdump filter is simple:
(not src net 192.168) and not (tcp and src port 80)
This filter is then run with the -n and -t options set on tcpdump. The -n
option tells tcpdump not to resolve the IP addresses into hostnames and the
-t option suppresses the timestamp in the printed output. The result is
piped to a file which is then sent to a sort routine. The sort command used
is sort -u. On most unix systems this will sort the connections according
to source IP address and throw away duplicate connections. We are able to
sort by source IP address because we have thrown away the timestamp and
hence the source IP is the first thing on the data line. Further, since we
have ignored the timestamp, connections between hosts that are identical
except for when they occurred will not be considered, since we are using
the -u (for unique) option on the sort. In this way, we are able to exploit
the efficient coding of the sort routine (which is much faster than perl)
and reduce the data further by throwing away duplicate connections.
The final step is then to have a perl program that takes the filtered
sorted unique output and determine which source hosts contacted many
destination hosts. Because we have sorted the data according to source IP
address we are able to sidestep searching for the source IP in the list we
construct each time a new data line is considered.
The final challenge then is to provide for the case where many, many (read
thousands) of destination hosts are contacted by a single source IP. To
handle this case we attempted to reduce the search space by using an
embedded hash table structure. Under this scheme, for a given source IP, we
search for the first octet of the destination IP in a hash that only
consists of previously found first octet numbers (max number of elements is
256), then once the first octet is found it points to another hash
containing recorded second octet numbers (corresponding to the first
octet). The hash of second octet numbers is searched (again max search
space is 256 elements) and a match here points to a hash of third octet
numbers and so on. The final value of the fourth octet hash is the number
of times this destination host was accessed by this source. Again, the code
is available at the web site.
Action 7.5 Reduce the data volume
We have found there is a practical limit to the analysis that can be
performed with filters. Also, tcpdump collects a LOT of data. In order to
perform more advanced analysis, it is necessary to reduce the data. We do
this in two steps.
first we use filters to separate the data by protocol.
Then, we further reduce the data to a common format that we use to
store information from any sensor.
The reduced data is comma delimited so it can easily be imported
into databases or spreadsheets and contains the following fields:
date, time, src, srcport, dst, dstport, protocol. We refer to this
format as bcp, since it is Bulk CoPied from all analysis stations
to a huge database for storage, or historical analysis.
Our first step is to convert from hourly to daily files. We do this
because,
in comparing historical information, delays are created by the overhead
involved in opening and closing hourly files. Here's how we do conversion
and
reduction.
Use tcpslice to concatenate hourly files into daily files. We can
put a whole day's worth of files together using tcpslice. From our
/LOG/date, the directory holding the day's data we could type:
$ tcpslice *.9802* > tcpdump.980202
tcpslice would cat all the hourly files into a daily file
tcpdump.980202, the daily file for February 02, 1998.
Next, we extract the various protocols we are interested in as part of the
data reduction. At our site the primary protocols in use are: TCP, UDP,
ICMP, IGMP and IGRP. To extract UDP into an ascii file for further
processing:
$ tcpdump -r tcpdump.980202 udp > udp.980202
We repeat this operation for protocol we are interested in keeping
for long term analysis. We run some cursory tests for ICMP looking
for routing updates that are from external addresses, but do not
archive IGRP or IGMP. We do reduce and archive, UDP, TCP, and
ICMP.
Then we run a simple test on the data, from time to time we find some
interesting IP traffic that is not one of the protocols.
$ tcpdump -r tcpdump.980202 (not udp and not tcp and not icmp and
not igrp and not igmp) > other.980202
If other is not an empty file then it would pay to invest some
time tracking down the source and destination addresses to sort
out what is going on. There are a variety of services including
hardware encryption units that use IP datagrams of various types
in order to communicate.
While it is important to know how to do all these things, it does get
monotonous, so we have included consolidate.pl to take all the hourly files
and produce daily files from them.
Finally, in order to compare results over time, we convert the protocol
files to bcp format using translate.pl. Another advantage of converting
files to this format is that its universal format allows us to integrate
data from different kinds of sensors.
Appendix A
Building an Intrusion Detection System using RedHat Linux 5.1 and Gateway
2000 Pentiums
This documentation shows how we built an intrusion detection system (sensor
and analysis station) using RedHat Linux 5.1.
We came across several complications that took a little bit of research,
experimentation, and perseverance to overcome. We have included tips where
we could, however, every site and system is different and will have its own
unique idiosyncracies.
The following is a list of Hardware and Software that we used:
HW: SW:
Pentium II 350 MHZ RedHat Linux 5.1
128MB RAM RedHat Linux Patches
9GB SCSI Hard Drive CID Code
STB Velocity 128bit 8MB Video Card Libpcap
Generic Multi-Sync Monitor Tcpdump
Tcpslice
Secure Shell
Note: The Hardware was ordered from Gateway with Win95 pre-loaded.
Step 1: Information Gathering
A. Make list of Hardware specifications. You will need this information
later for Linux configuration.
B. Boot machine under Windows and check and record IRQ settings for all
network peripherals.
C. You may run into IRQ conflicts as Windows allows for IRQ sharing
however Linux does not.
D. If you do happen to find a shared IRQ one solution is to physically
move the network card or the SCSI card to another slot.
Step 2: RedHat Linux 5.1 Installation
Note: Refer to the RedHat Linux manual for further information.
A. Boot machine with RedHat Linux 5.1 boot disk.
B. Select install from CDROM.
C. Partition Disk. (9GB)
1. Select Disk Druid.
2. Delete existing partition.
3. For the sensor make 3 partitions using the add option as follows:
mount point size type
127MB Linux Swap
/ 2000MB Linux Native
/LOG Rest of disk Linux Native
1. For the analysis station make 5 partitions using the add option
as follows:
mount point size type
127MB Linux Swap
/ 2000MB Linux Native
/LOG 5059MB Linux Native
/tmp 512MB Linux Native
/usr/local 1000MB Linux Native
Note: The Linux Boot partition can't be > 2GB (2000MB). Also, you
might need to play with the sizes to make them fit.
1. Once done with disk partitions select OK to move on.
D. Check for bad blocks on swap partition only as the other partitions
are too big and will take too much time.
A. Select the components that you will need. This will depend on your
site's particular needs.
B. Select type of mouse (ps/2).
C. Select video card (STB Velocity 128).
D. Select monitor (Generic Multi-sync) and don't probe.
E. Select Video memory (8MB).
F. Select no clockchip setting.
G. Select Video modes: 8bit 16bit 24bit
(x)1280x1024 (x)1280x1024 (x)1280x1024
L. Configure LAN settings:
1. Enter IP, Netmask, Primary Name Server, Default Gateway,
Hostname, Domain Name, Secondary Name Server, Tertiary Name
Server
M. Format Machine time.
A. Select services. Depends on site.
B. Create Boot disk.
C. Download and Install latest patches.
1. Download patches from: http://www.redhat.com to directory
/usr/local/archives/rpm
2. The glint patch is needed before the package manager can be used.
3. From /usr/local/archives/rpm enter: rpm -Uvh glint*
4. From Control Panel Select Package Manager.
5. Select configure and enter path to look.
6. Select Available to see which packages are not installed.
7. Select packages to install or upgrade.
Q. To use the screensaver and Xlock functions the following packages need
to be installed:
1. xscreensaver*
2. fortune-mod*
3. xlockmore*
R. Reboot
A. Build Custom Kernel:
1. Instructions are also in RedHat Linux Manual.
2. Make sure the kernel-headers* and kernel-source* rpm files are
upgraded and installed.
3. cd /usr/src/linux
4. make mrproper
5. make xconfig
+ Select options based on what your requirements are.
+ Enter information about firewall and ping ignores here.
1. make dep
2. make clean
3. make boot
4. make modules
5. rm -rf /lib/modules/2.0.34-0.6-old
6. mv /lib/modules/2.0.34-0.6 /lib/modules/2.0.34-0.6-old
7. make modules_install
8. mv /boot/vmlinuz-2.0.34-0.6 /boot/vmlinuz-2.0.34-0.6.old
9. cp /usr/src/linux/arch/i386/boot/zImage /boot/vmlinuz-2.0.34-0.6
10. edit /etc/lilo.conf - enter kernel data
11. /sbin/lilo
12. reboot
13. The machine should boot up in the new customized kernel. If you
want a choice of booting up in the new kernel or the old kernel
press the Tab button when the machine gets to the Linux: prompt.
14. Linux Installation is now complete.
1. every password change.
2. Disable all inetd.conf services.
3. The following directories need to be created:
+ directory1
+ directory2
1. Configure SHADOW code for your site.
+ crontab
1. Configure SSH for your site.
B. Configure Analysis Station
1. Run pwconv to incorporate shadow passwords (you might need to run
this after every password change.
2. Disable all inetd.conf services.
3. The following directories need to be created:
+ directory1
+ directory2
1. Configure SHADOW code for your site.
2. Configure SSH for your site.
C. Test fetchem
A. Edit and install crontab for your site.
B. To use the analyzer, point your browser to:
http://your.analysis.station
Rob for:
Alan Paller, Director of Research 301-951-0102
The SANS Institute www.sans.org sans@clark.net
--------------------------------------------------------------------------
Upcoming SANS Conferences: | SANS Publications:
Intrusion Detection Conference and | SANS Network Security Digest
Network Security 98 (Oct., Orlando)| Incident Handling: Step-by-Step
UC Davis Workshop on | Windows NT Security: Step-by-Step
Intrusion Detection | The SANS NT Digest