If you run a server, you will undoubtedly get to a point where you need to nail down some network-related problems. Of course it would be easy to just shoot a mail to the support department, but sometimes you need to get your hands dirty. In this case,
tcpdump is the tool for that job. Tcpdump is a network packet analyzer that runs under the command line.
This article will be split into three parts:
Filtering based on certain traffic characteristics.
A short snippet of the more advanced features (such as logical expressions, filtering by TCP flags).
Since tcpdump is not included with most base systems, you will need to install it. However, nearly all Linux distributions have tcpdump in their core repositories. For Debian based distributions, the command to install tcpdump is:
apt-get install tcpdump
For CentOS/RedHat, use the following command:
yum install tcpdump
FreeBSD offers a pre-built package which can be installed by issuing:
pkg install tcpdump
There's also a port available,
net/tcpdump which can be installed via:
cd /usr/ports/net/tcpdump make install clean
If you run
tcpdump without any arguments, you'll be be battered with results. Running it on a freshly spinned up instance here on Vultr for less than five seconds gives the following results:
2661 packets captured 2663 packets received by filter 0 packets dropped by kernel
Before going into more details on how to filter input, you should take a look at some parameters that can be passed to tcpdump:
-i - Specifies the interface you want to listen on, for example:
tcpdump -i eth0.
-n - Do not try to do reverse lookups on IP addresses, for example:
tcpdump -n (if you add another
n tcpdump will show you port numbers instead of names).
-X - Show the content of the collected packets:
-c - Only capture
x being an arbitrary number, for example
tcpdump -c 10 captures exactly 10 packets.
-v - Increase the amount of packet information you are shown, more
vs add more verbosity.
Each of those parameters mentioned here can be combined together. If you wanted to capture 100 packets, but only on your VPN interface tun0, then the tcpdump command would look like this:
tcpdump -i tun0 -c 100 -X
There are dozens (if not hundreds) of options in addition to those few, but they are the most common ones. Feel free to read tcpdump's manpage on your system.
Now that you have a basic understanding of tcpdump, it's time to look at one of tcpdump's most awesome features: expressions. Expressions will make your life a lot easier. They are also known as BPF or Berkeley Packet Filters. Using expressions allows you to selectively display (or ignore) packets based on certain characteristics - such as origin, destination, size, or even TCP sequence number.
So far you've managed to limit your search to a certain amount of packets on a certain interface, but let's be honest here: that still leaves too much background noise to effectively work with the collected data. That's where expressions come into play. The concept is pretty straightforward, so we'll leave out the dry theory here and support the understanding with some practical examples.
The expressions that you'll probably be using the most are:
host - Look for traffic based on hostnames or IP addresses.
dst - Look for traffic from or to a specific host.
proto - Look for traffic of a certain protocol. Works for tcp, udp, icmp, and others. Omitting the
proto keyword is also possible.
net - Look for traffic to / from a certain range of IP addresses.
port - Look for traffic to / from a certain port.
less - Look for traffic bigger or smaller than a certain amount of bytes.
While the manpage for
tcpdump just contains a few examples, the manpage for
pcap-filter has very detailed explanations on how each filter works and can be applied.
If you want to see how your communication with a certain server is going, then you can use the
host keyword, for example (including some of the parameters from above):
tcpdump -i eth0 host vultr.com
Sometimes there are computers on the network that don't honor the MTU or spam you with large packets; filtering them out can be difficult sometimes. Expressions allow you to filter out packages that are bigger or smaller than a certain number of bytes:
tcpdump -i eth0 -nn greater 128 or tcpdump -i eth0 -nn less 32
Maybe only a certain port is of interest for you. In this case, use the
tcpdump -i eth0 -X port 21
You can also look out for port ranges:
tcdump -i eth0 -X portrange 22-25
Since NAT gateways are pretty common, you may only look for destination ports:
tcpdump dst port 80
If you are watching traffic to your web server, you may only want to look at TCP traffic to port 80:
tcpdump tcp and dst port 80
You are probably asking yourself what the keyword
and is doing there. Good question. That brings us to the last part of this article.
tcpdump offers basic support for logical expressions, more specifically:
&& - Logical "and".
|| - Logical "or".
! - Logical "not".
Together with the ability to group expressions together, this allows you to create very powerful searches for incoming and outgoing traffic. So let's filter out traffic coming from vultr.com on port 22 or 443:
tcpdump -i eth0 src host vultr.com and (dst port 22 or 443)
Running this on the command line will give you the following error:
bash: syntax error near unexpected token `('
That's because there is a caveat:
bash tries to evaluate every character it can. This includes the
) characters. In order to avoid that error, you should use single quotes around the combined expression:
tcpdump -i eth0 'src host vultr.com and (dst port 22 or 443)'
Another useful example: When debugging SSH issues with one of your users, you may want to ignore everything that's related to your SSH session:
tcpdump '!(host $youripaddress) && port 22)'
Again, the use cases are endless, and you can specify into extreme depths what kind of traffic that you want to see. The following command would show you only SYNACK packets of a TCP handshake:
tcpdump -i eth0 'tcp=18'
This works by looking at the thirteenth offset of the TCP header and the eighteenth byte within it.
If you made it all the way here, then you are ready for most use cases that will arise. I can barely touch the surface without going into too many details. I highly recommend that you experiment with the different options and expressions a bit further; and as usual: reference the manpage when you get lost.
Last but not least - a quick look back. Remember the beginning of this article? With the thousands of packets captured in a matter of seconds? The power of
tcpdump can trim that down a whole lot:
tcpdump -i eth0 tcp port 22
The result is now:
81 packets captured 114 packets received by filter 0 packets dropped by kerne
This is much saner and easier to debug. Happy networking!