Introduction
Setting up a firewall for your infrastructure is a great way to provide some basic security for your services. Once you’ve developed a policy you are happy with, the next step is to test your firewall rules. It is important to get a good idea of whether your firewall rules are doing what you think they are doing and to get an impression of what your infrastructure looks like to the outside world.
In this guide, we’ll go over some simple tools and techniques that you can use to validate your firewall rules. These are some of the same tools that malicious users may use, so you will be able to see what information they can find by making requests of your servers.
Prerequisites
In this guide, we will assume that you have a firewall configured on at least one server. You can get started building your firewall policy by following one or more of these guides:
- Iptables
- UFW
- FirewallD
In this guide, we will call the server containing the firewall policies you wish to test the target. In addition to your target, you will also need to have access to a server to test from, located outside of the network that your firewall protects. In this guide, we will use an Ubuntu 14.04 server as our auditing machine.
Once you have a server to test from and the targets you wish to evaluate, you can continue with this guide.
You should only perform the activities outlined in this guide on infrastructure that you control, for the purpose of security auditing. The laws surrounding port scanning are uncertain in many jurisdictions. ISPs and other providers have been known to ban users who are found port scanning.
The Tools We Will Use to Test Firewall Policies
There are quite a few different tools that we can use to test our firewall policies. Some of them have overlapping functionality. We will not cover every possible tool. Instead, we will cover some general categories of auditing tools and go over the tools we will be using in this guide.
Packet Analyzers
Packet analyzers can be used to watch all of the network traffic that goes over an interface in great detail. Most packet analyzers have the option of operating in real time, displaying the packets as they are sent or received, or of writing packet information to a file and processing it at a later time. Packet analysis gives us the ability to see, at a granular level, what types of responses our target machines are sending back to hosts on the open network.
For the purposes of our guide, we will be using the tcpdump
tool. This is a good option because it is powerful, flexible, and rather ubiquitous on Linux systems. We will use it to capture the raw packets as we run our tests in case we need the transcript for later analysis. Some other popular options are Wireshark (or tshark
, its command line cousin) and tcpflow
which can piece together entire TCP conversations in an organized fashion.
Port Scanners
In order to generate the traffic and responses for our packet analyzer to capture, we will use a port scanner. Port scanners can be used to craft and send various types of packets to remote hosts in order to discover type of traffic the server accepts. Malicious users often use this as a discovery tool to try to find vulnerable services to exploit (part of the reason to use a firewall in the first place), so we will use this to try to see what an attacker could discover.
For this guide, we will use the nmap
network mapping and port scanning tool. We can use nmap
to send packets of different types to try to figure out which services are on our target machine and what firewall rules protect it.
Setting Up the Auditing Machine
Before we get started, we should make sure we have the tools discussed above. We can get these from Ubuntu’s default package repositories.
Update the local package index and install the software if it is not already available:
- sudo apt-get update
- sudo apt-get install nmap tcpdump
The nmap
package that we downloaded above is good enough for our purposes in most ways. However, its databases for operating system detection and service versioning routines is probably not current.
We can get some updated versions of these files from the nmap
SVN repositories to use for our scans. Start by creating a directory for these files in your home directory and moving inside:
- mkdir ~/nmap_updated_data
- cd ~/nmap_updated_data
Next, in your web browser, navigate to the nmap
releases page here. Click on the latest stable version (nmap-6.47
at the time of this writing) to see the files in the directory.
We will use wget
to download seven files onto our auditing server, all of which begin with nmap-
and have no suffix. Change the version in the URL below to match the latest stable version:
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-service-probes
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-payloads
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-services
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-protocols
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-rpc
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-mac-prefixes
- wget https://svn.nmap.org/nmap-releases/nmap-6.47/nmap-os-db
We can tell nmap
to use the files in this directory during our scans so that it has a better chance of recognizing more recent operating systems and service versions.
Next, we will create a directory where we can store our scan results:
- mkdir ~/scan_results
To make sure that you get clean results, exit out of any sessions you might have open between your auditing system and the target system. This includes SSH sessions, any HTTP(S) connections you may have established in a web browser, etc.
Scan your Target for Open TCP Ports
Now that we have our server and files ready, we will begin by scanning our target host for open TCP ports.
There are actually a few TCP scans that nmap
knows how to do. The best one to usually start off with is a SYN scan, also known as a “half-open scan” because it never actually negotiates a full TCP connection. This is often used by attackers because it fails to register on some intrusion detection systems because it never completes a full handshake.
Setting Up the Packet Capture
Before we scan, we will set up tcpdump
to capture the traffic generated by the test. This will help us analyze the packets sent and received in more depth later on if we need to. Let’s create a directory within ~/scan_results
so that we can keep the files related to our SYN scan together:
- mkdir ~/scan_results/syn_scan
We can start a tcpdump
capture and write the results to a file in our ~/scan_results/syn_scan
directory with the following command:
- sudo tcpdump host target_ip_addr -w ~/scan_results/syn_scan/packets
By default, tcpdump
will run in the foreground. In order to run our nmap
scan in the same window, we’ll need to pause the tcpdump
process and then restart it in the background.
We can pause the running process by hitting CTRL-Z
:
CTRL-Z
This will pause the running process:
Output^Z
[1]+ Stopped sudo tcpdump host target_ip_addr -w ~/scan_results/syn_scan/packets
Now, you can restart the job in the background by typing bg
:
- bg
You should see a similar line of output, this time without the “Stopped” label and with an ampersand at the end to indicate that the process will be run in the background:
Output[1]+ sudo tcpdump host target_ip_addr -w ~/scan_results/syn_scan/packets &
The command is now running in the background, watching for any packets going between our audit and target machines. We can now run our SYN scan.
Run the SYN Scan
With tcpdump
recording our traffic to the target machine, we are ready to run nmap
. We will use the following flags to get nmap
to perform the actions we require:
-sS
: This starts a SYN scan. This is technically the default scan thatnmap
will perform if no scan type is given, but we will include it here to be explicit.-Pn
: This tellsnmap
to skip the host discovery step, which would abort the test early if the host doesn’t respond to a ping. Since we know that the target is online, we can skip this.-oN
: This writes the results to a file that we can use for later analysis.-vv
: This increases the verbosity of the output.--reason
: This tellsnmap
to provide the reason that a port’s state was reported a certain way.--datadir
: This specifies the directory with our updatednmap
files.
Together, the command will look something like this:
- sudo nmap -sS -Pn -oN ~/scan_results/syn_scan/nmap.results -vv --reason --datadir ~/nmap_updated_data target_ip_addr
You will likely see results that look something like this:
OutputStarting Nmap 6.40 ( http://nmap.org ) at 2015-08-21 15:58 EDT
Initiating ARP Ping Scan at 15:58
Scanning 198.51.100.15 [1 port]
Completed ARP Ping Scan at 15:58, 0.20s elapsed (1 total hosts)
Initiating Parallel DNS resolution of 1 host. at 15:58
Completed Parallel DNS resolution of 1 host. at 15:58, 0.29s elapsed
Initiating SYN Stealth Scan at 15:58
Scanning 198.51.100.15 [1000 ports]
Discovered open port 22/tcp on 198.51.100.15
Discovered open port 80/tcp on 198.51.100.15
Increasing send delay for 198.51.100.15 from 0 to 5 due to 36 out of 119 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 5 to 10 due to 12 out of 38 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 10 to 20 due to 11 out of 34 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 20 to 40 due to 11 out of 29 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 40 to 80 due to 11 out of 31 dropped probes since last increase.
Completed SYN Stealth Scan at 15:58, 14.06s elapsed (1000 total ports)
Nmap scan report for 198.51.100.15
Host is up, received arp-response (0.00059s latency).
Scanned at 2015-08-21 15:58:27 EDT for 14s
Not shown: 998 closed ports
Reason: 998 resets
PORT STATE SERVICE REASON
22/tcp open ssh syn-ack
80/tcp open http syn-ack
Read data files from: /home/user/nmap_updated_data
Nmap done: 1 IP address (1 host up) scanned in 14.64 seconds
Raw packets sent: 1126 (49.528KB) | Rcvd: 1126 (45.036KB)
Stop the tcpdump Packet Capture
Now that the scan is complete, we can bring our tcpdump
process back into the foreground and stop it.
Bring it out of the background by typing:
- fg
Stop the process by holding the control key and hitting “c”:
CTRL-C
Analyzing the Results
You should now have two files in your ~/scan_results/syn_scan
directory. One called packets
, generated by the tcpdump
run, and one generated by nmap
called nmap.results
.
Let’s look at the nmap.results
file first:
- less ~/scan_results/syn_scan/nmap.results
# Nmap 6.40 scan initiated Fri Aug 21 16:21:36 2015 as: nmap -sS -Pn -oN /home/user/scan_results/syn_scan/scan_results -vv --reason --datadir /home/user/nmap_updated_data 198.51.100.15
Nmap scan report for 198.51.100.15
Host is up, received arp-response (0.0096s latency).
Scanned at 2015-08-21 16:21:36 EDT for 2s
Not shown: 918 closed ports, 80 filtered ports
Reason: 918 resets and 80 no-responses
PORT STATE SERVICE REASON
22/tcp open ssh syn-ack
80/tcp open http syn-ack
Read data files from: /home/user/nmap_updated_data
# Nmap done at Fri Aug 21 16:21:38 2015 -- 1 IP address (1 host up) scanned in 1.70 seconds
The highlighted area above contains the main results of the scan. We can see that port 22 and port 80 are open on the scanned host in order to allow SSH and HTTP traffic. We can also see that 918 ports were closed, and 80 ports were “filtered”. Filtered means that these ports were identified as being stopped by something along the network path. It could be a firewall on the target, but it could also be filtering rules on any of the intermediate hosts between the audit and target machines.
If we want to see the actual packet traffic that was sent to and received from the target, we can read the packets
file back into tcpdump
, like this:
- sudo tcpdump -nn -r ~/scan_results/syn_scan/packets | less
This file contains the entire conversation that took place between the two hosts. You can filter in a number of ways.
For instance, to view only the traffic sent to the target, you can type:
- sudo tcpdump -nn -r ~/scan_results/syn_scan/packets 'dst target_ip_addr' | less
Likewise, to view only the response traffic, you can change the “dst” to “src”:
- sudo tcpdump -nn -r ~/scan_results/syn_scan/packets 'src target_ip_addr' | less
Open TCP ports would respond to these requests with a SYN packet. We can search directly for responses for this type with a filter like this:
- sudo tcpdump -nn -r ~/scan_results/syn_scan/packets 'src target_ip_addr and tcp[tcpflags] & tcp-syn != 0' | less
This will show you only the successful SYN responses, and should match the ports that you saw in the nmap
run:
Outputreading from file packets, link-type EN10MB (Ethernet)
16:21:37.749444 IP 198.51.100.15.80 > 198.51.100.2.40092: Flags [S.], seq 986505647, ack 3171548275, win 29200, options [mss 1460], length 0
16:21:37.749481 IP 198.51.100.15.22 > 198.51.100.2.40092: Flags [S.], seq 3684808250, ack 3171548275, win 29200, options [mss 1460], length 0
You can do more analysis of the data as you see fit. It has all been captured for later processing.
Scan your Target for Open UDP Ports
Now that you have a good handle on how to run these tests, we can complete a similar process to scan for open UDP ports.
Setting Up the Packet Capture
Once again, let’s create a directory to hold our results:
- mkdir ~/scan_results/udp_scan
Start a tcpdump
capture again. This time, write the file to the new ~/scan_results/udp_scan
directory:
- sudo tcpdump host target_ip_addr -w ~/scan_results/udp_scan/packets
Pause the process and put it into the background:
CTRL-Z
- bg
Run the UDP Scan
Now, we are ready to run the UDP scan. Due to the nature of the UDP protocol, this scan usually takes significantly longer than the SYN scan. UDP is a connectionless protocol, so receiving no response could mean that the target’s port is blocked, that it was accepted, or that the packet was lost. To try to distinguish between these, nmap
must retransmit additional packets to try to get a response.
Most of the flags will be the same as we used for the SYN scan. In fact, the only new flag is:
-sU
: This tellsnmap
to perform a UDP scan.
Also, be sure to write the results to the ~/scan_results/udp_scan
directory. All together, the command should look like this:
- sudo nmap -sU -Pn -oN ~/scan_results/udp_scan/nmap.results -vv --reason --datadir ~/nmap_updated_data target_ip_addr
When you are finished, make sure to stop your tcpdump
run.
Stop the tcpdump Packet Capture
Bring the tcpdump
process back into the foreground by typing:
- fg
Stop the packet capture by holding control and hitting “c”:
CTRL-c
Analyzing the Results
Now, we can take a look at the generated files.
The resulting nmap.results
file should look fairly similar to the one we saw before:
- less ~/scan_results/udp_scan/nmap.results
# Nmap 6.40 scan initiated Fri Aug 21 17:00:57 2015 as: nmap -sU -Pn -oN /home/user/scan_results/udp_scan/nmap.results -vv --reason --datadir /home/user/nmap_updated_data 45.55.56.133
Increasing send delay for 198.51.100.15 from 400 to 800 due to 11 out of 12 dropped probes since last increase.
Nmap scan report for 198.51.100.15
Host is up, received arp-response (0.00084s latency).
Scanned at 2015-08-21 17:00:57 EDT for 1090s
Not shown: 997 closed ports
Reason: 997 port-unreaches
PORT STATE SERVICE REASON
22/udp open|filtered ssh no-response
80/udp open|filtered http no-response
443/udp open|filtered https no-response
MAC Address: 04:01:67:0E:04:01 (Unknown)
Read data files from: /home/user/nmap_updated_data
# Nmap done at Fri Aug 21 17:19:07 2015 -- 1 IP address (1 host up) scanned in 1090.49 seconds
A key difference will likely be the amount of ports marked open|filtered
. This means that nmap
couldn’t determine whether the lack of a response meant that a service accepted the traffic or whether it was dropped by some firewall or filtering mechanism along the delivery path.
Analyzing the tcpdump
output is also significantly more difficult since there are no connection flags and because we must match up ICMP responses to UDP requests.
We can see how nmap
had to send out many packets to the ports that were reported as open|filtered
by asking to see the UDP traffic to one of the reported ports:
- sudo tcpdump -nn -P out -r ~/scan_results/udp_scan/packets 'udp and port 22'
You will likely see something that looks like this:
Outputreading from file packets, link-type EN10MB (Ethernet)
17:00:59.640934 IP 198.51.100.2.61403 > 198.51.100.15.22: UDP, length 0
17:00:59.741114 IP 198.51.100.2.61404 > 198.51.100.15.22: UDP, length 0
17:02:15.731259 IP 198.51.100.2.61405 > 198.51.100.15.22: UDP, length 0
17:02:16.532531 IP 198.51.100.2.61406 > 198.51.100.15.22: UDP, length 0
17:02:18.134395 IP 198.51.100.2.61407 > 198.51.100.15.22: UDP, length 0
17:02:18.935328 IP 198.51.100.2.61408 > 198.51.100.15.22: UDP, length 0
17:02:19.736238 IP 198.51.100.2.61409 > 198.51.100.15.22: UDP, length 0
17:02:20.537204 IP 198.51.100.2.61410 > 198.51.100.15.22: UDP, length 0
17:02:22.339179 IP 198.51.100.2.61411 > 198.51.100.15.22: UDP, length 0
Compare this to the results we see from one of the scanned ports that was marked as “closed”:
- sudo tcpdump -nn -P out -r ~/scan_results/udp_scan/packets 'udp and port 53'
Outputreading from file packets, link-type EN10MB (Ethernet)
17:13:33.609528 IP 198.51.100.2.61403 > 198.51.100.15.53: 0 stat [0q] (12)
We can try to manually reconstruct the process that nmap
goes through by first compiling a list of all of the ports that we’re sending UDP packets to using something like this:
- sudo tcpdump -nn -P in -r ~/scan_results/udp_scan/packets "udp" | awk '{print $5;}' | awk 'BEGIN { FS = "." } ; { print $5 +0}' | sort -u | tee outgoing
Then, we can see which ICMP packets we received back saying the port was unreachable:
- sudo tcpdump -nn -P in -r ~/scan_results/udp_scan/packets "icmp" | awk '{print $10,$11}' | grep unreachable | awk '{print $1}' | sort -u | tee response
We can see then take these two responses and see which UDP packets never received an ICMP response back by typing:
- comm -3 outgoing response
This should mostly match the list of ports that nmap
reported.
Host and Service Discovery
We can run some additional tests on our target to see if it is possible for nmap
to identify the operating system running or any of the service versions.
Let’s make a directory to hold our versioning results:
- mkdir ~/scan_results/versions
Discovering the Versions of Services on the Server
We can attempt to guess the versions of services running on the target through a process known as fingerprinting. We retrieve information from the server and compare it to known versions in our database.
A tcpdump
wouldn’t be too useful in this scenario, so we can skip it. If you want to capture it anyways, follow the process we used last time.
The nmap
scan we need to use is triggered by the -sV
flag. This is the full command:
- sudo nmap -sV -Pn -oN ~/scan_results/versions/service_versions.nmap -vv --reason --datadir ~/nmap_updated_data target_ip_addr
If you view the file that results, you may get information about the service running, depending on how “chatty” or even how unique the service’s response is. Often, you will not receive definitive results:
- less ~/scan_results/versions/service_versions.nmap
# Nmap 6.40 scan initiated Fri Aug 21 18:33:53 2015 as: nmap -sV -Pn -oN /home/user/scan_results/version_scan/nmap.results -vv --reason --datadir /home/user/nmap_updated_data 198.51.100.15
Increasing send delay for 198.51.100.15 from 0 to 5 due to 36 out of 119 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 5 to 10 due to 12 out of 38 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 10 to 20 due to 11 out of 34 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 20 to 40 due to 11 out of 29 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 40 to 80 due to 11 out of 31 dropped probes since last increase.
Nmap scan report for 198.51.100.15
Host is up, received arp-response (0.0033s latency).
Scanned at 2015-08-21 18:33:53 EDT for 12s
Not shown: 998 closed ports
Reason: 998 resets
PORT STATE SERVICE REASON VERSION
22/tcp open ssh syn-ack (protocol 2.0)
80/tcp open http syn-ack nginx 1.4.6 (Ubuntu)
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at http://www.insecure.org/cgi-bin/servicefp-submit.cgi :
SF-Port22-TCP:V=6.40%I=7%D=8/21%Time=55D7A75D%P=x86_64-pc-linux-gnu%r(NULL
SF:,29,"SSH-2.0-OpenSSH_6.6.1p1x20Ubuntu-2ubuntu2rn");
MAC Address: 04:01:67:0E:04:01 (Unknown)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel
Read data files from: /home/user/nmap_updated_data
Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
# Nmap done at Fri Aug 21 18:34:05 2015 -- 1 IP address (1 host up) scanned in 11.97 seconds
You can see that the test was able to identify the SSH protocol version accepted and that the Ubuntu version of Nginx was installed. It had trouble getting more information about the SSH server as it did not recognize the fingerprint.
Discovering the Host Operating System
We can try to have nmap
guess the host operating system based on characteristics of its software and responses as well. This works much in the same way as service versioning. Once again, we will omit the tcpdump
run from this test, but you can perform it if you’d like.
The flag we need in order to perform operating system detection is -O
(the capitalized letter “O”). A full command may look something like this:
- sudo nmap -O -Pn -oN ~/scan_results/versions/os_version.nmap -vv --reason --datadir ~/nmap_updated_data target_ip_addr
If you view the output file, you might see something that looks like this:
- less ~/scan_results/versions/os_version.nmap
# Nmap 6.40 scan initiated Mon Aug 24 15:49:50 2015 as: nmap -O -Pn -oN /home/user/scan_results/versions/os_version.nmap -vv --reason --datadir /home/user/nmap_updated_data 198.51.100.15
Increasing send delay for 198.51.100.15 from 0 to 5 due to 13 out of 43 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 5 to 10 due to 39 out of 129 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 10 to 20 due to 11 out of 25 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 20 to 40 due to 11 out of 24 dropped probes since last increase.
Increasing send delay for 198.51.100.15 from 40 to 80 due to 11 out of 30 dropped probes since last increase.
Nmap scan report for 198.51.100.15
Host is up, received user-set (0.00064s latency).
Scanned at 2015-08-24 15:49:50 EDT for 32s
Not shown: 998 closed ports
Reason: 998 resets
PORT STATE SERVICE REASON
22/tcp open ssh syn-ack
80/tcp open http syn-ack
No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).
TCP/IP fingerprint:
OS:SCAN(V=6.40%E=4%D=8/24%OT=22%CT=1%CU=40659%PV=N%DS=2%DC=I%G=Y%TM=55DB757
OS:E%P=x86_64-pc-linux-gnu)SEQ(SP=104%GCD=1%ISR=10C%TI=Z%CI=Z%TS=8)OPS(O1=M
OS:5B4ST11NW8%O2=M5B4ST11NW8%O3=M5B4NNT11NW8%O4=M5B4ST11NW8%O5=M5B4ST11NW8%
OS:O6=M5B4ST11)WIN(W1=7120%W2=7120%W3=7120%W4=7120%W5=7120%W6=7120)ECN(R=Y%
OS:DF=Y%T=40%W=7210%O=M5B4NNSNW8%CC=Y%Q=)T1(R=Y%DF=Y%T=40%S=O%A=S+%F=AS%RD=
OS:0%Q=)T2(R=N)T3(R=N)T4(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T5(R=Y%DF
OS:=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)T6(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=
OS:%RD=0%Q=)T7(R=N)U1(R=Y%DF=N%T=40%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=
OS:G%RUD=G)U1(R=N)IE(R=N)
Uptime guess: 0.028 days (since Mon Aug 24 15:09:48 2015)
Network Distance: 2 hops
TCP Sequence Prediction: Difficulty=260 (Good luck!)
IP ID Sequence Generation: All zeros
Read data files from: /home/user/nmap_updated_data
OS detection performed. Please report any incorrect results at http://nmap.org/submit/ .
# Nmap done at Mon Aug 24 15:50:22 2015 -- 1 IP address (1 host up) scanned in 32.63 seconds
We can see that in this case, nmap
has no guesses for the operating system based on the signature it saw. If it had received more information, it would likely show various percentages which indicate how much of the operating system signature in its databases match target machine. You can see the fingerprint signature that nmap
received from the target below the TCP/IP fingerprint:
line.
Operating system identification can help an attacker determine which exploits may be useful on the system. Configuring your firewall to respond to fewer inquiries can help to hinder the accuracy of some of these detection methods.
Conclusion
Testing your firewall and building an awareness of what your internal network looks like to an outside attacker can help minimize your risk. The information you find from probing your own infrastructure may open up a conversation about whether any of your policy decisions need to be revisited in order to increase security. It may also illuminate any gaps in your security that may have occurred due to incorrect rule ordering or forgotten test policies. It is recommended that you test your policies with the latest scanning databases regularity in order improve, or at least maintain, your current level of security.
To get an idea of some policy improvements for your firewall, check out this guide.
Source: DigitalOcean News