Firewall Tester

Firewall Tester is an utility to test firewalls into laboratory environments or into real networks.
It use lxc containers to generate the required network setup to run each defined test, and generates a result report.
It’s developed in python3 fully OOP and modular by design.
Its design allows to extend the functionalities by simple or complex modules to be run in client and/or server side container. Each module is a fully isolated python module, this way it can be extended to suppor any protocol for 3 and 7 OSI layers.

Overview

It allows you to define test rules in a classic firewall rule format:

action, source ip, source port, destinantion ip, destination port
Analyzing the firewall configuration, and the test rule, the system is able to define and setup the required network topolgy to run the test from client to server container, using a router container to manage traffic beyond the firewall’s networks, if required.
It covers these 4 possible scenarios:

  1. Client and server connects directly to the firewall:

    Client <----> Firewall <----> Server

  2. Clients connects to Firewall by intermediate router, server connects directly to firewall:

    Client <----> Router <----> Firewall <----> Server

  3. Clients connects directly to Firewall, server connects by intermediate router to firewall:

    Client <----> Firewall <----> Router <----> Server

  4. Clients and server connects by intermediate router to Firewall:

    Client <----> Router <----> Firewall <----> Router <----> Server

Just 3 LXC containers covers this 4 scenarios:

  • Client container: Runs the client side of each test, it works by modules, so can be extended.

  • Server container: Runs the server side of each test, it works by modules, so can be extended. It’s execution can be avoided to test connections to real servers.

  • Router container: Runs both client and server side router as required by each test. The traffic is isolated by Linux policy routing, so traffic is not routed fron client to server side by router without flowing through the firewall. It’s execution can be avoided to test connections to real servers.

How it works

The system requires 3 csv files as input data:

  • fwnics.csv: Defines the firewall’s neworking configuration.

  • routes.csv: Defines the firewall’s routing table.

  • tests.csv: Defines the tests to run.

Fields defined in fwnics.csv:

Field

Required

Description

Name

Yes

Interface name

IP

Yes

Interface IP address

Mask

Yes

Interface netmask

Vlan

Yes

Interface VLAN or VLAN where the interface connects to

Values allowed per field in fwnics.csv:

Field

Allowed values

Name

String

IP

IP address

Mask

Dotted decimal or CIRD

Vlan

Valid VLAN ID (Integer)

Fields defined in routes.csv:

Field

Required

Description

Interface

Yes

Interface name. Same as defined in fwnics.csv

IP

Yes

Route IP address

Mask

Yes

Route netmask

Gateway

Yes

Route gateway IP address

Metric

No

If needed metric for the route

Values allowed per field in routes.csv:

Field

Allowed values

Interface

String

IP

IP address

Mask

Dotted decimal or CIRD

Gateway

IP address

Metric

Integer

Fields defined in tests.csv:

Field

Required

Description

Rule number

Yes

Line rule number in the firewall ACL, there can be more than one test per ACL rule (more than one line with the same line number) it is not an unique ID type field

Permission

Yes

Action defined in the firewall ACL rule

Protocol

No

Protocol defined in the firewall ACL rule, if any

Source

No

Source address defined in the firewall ACL rule

Source Port

No

Source port defined in the firewall ACL rule

Destination

No

Destination address defined in the firewall ACL rule

Destination Port

No

Destination port defined in the firewall ACL rule

application

No

Application to simulate (Layer 7 application: DHCP, HTTP, HTTPS, NTP…)

Source zone

No

Source zone or interface defined in the firewall ACL

Destination zone

No

Destination zone or interface defined in the firewall ACL rule

Only Client Container

No

Enables or disables the server side and router (client and server side) containers in test. To run a test to servers outside the containers system, set it to True

Values allowed per field in tests.csv:

Field

Allowed values

Rule number

Integer

Protocol

IP protocol name or number or “IP” or “Any” or empty String

Examples:

  • tcp or udp or icmp or TCP or UDP or ICMP

  • IP or ip or Any or any or empty string which are different ways to write the same

Source and destination

Any or empty string or an space separated list of network and/or host IP address

Examples:

  • Any or any or empty string which are different ways to write the same

  • A single item: 10.10.10.20 or 10.0.0.0/8

  • A list of values: “192.168.0.0/26 10.10.10.10 172.16.30.1/32 10.1.1.0/255.255.255.0”

Source and destination port
Any or empty string or an space separated list of any of the following values:
Possible values for TCP and UDP:

  • An integer or Any or empty string. A single integer fallback to TCP if there is not a protocol defined in Protocol field of the test line

  • A protocol:port_number string

  • A protocol:operand:port_number string where operand can be one of: “eq”, “ne”, “lt”, “gt”, “=”, “!=”, “<>”, “<”, “>”

Possible values for ICMP

  • An icmp:message string

  • A message:code string, this format requires to set to ICMP the rule’s protocol field

Examples:

  • Any or any or empty string which are different ways to write the same

  • A single item: https or 443

  • A list of values: “tcp:21 tcp:http udp:69 icmp:echo-request”

  • A list of values: “ftp 22 http ntp” (If there is a protocol defined in Protocol field it will be used falling back to TCP, if ther is not protocol defined)

Application

A single application name or empty (i.e: DHCP NTP DNS… the support of applications depends on developed modules). If an application name is specified, the test runs as a L7 test, an empty value runs a L3 test

Source zone

String

Destination zone

String

Only Client Container

Boolean (True/yes or False/no/) True by default

It is possible to setup some parameters through the config file, config.yml, a yaml format file which defines the following options:

Name

Type

Possible Values

Description

max_retries_for_timeouts

Integer

1 to N

Defines how many times an accept test should be run if it timesout. An accept test should connect so the system retry if it doesn’t, before mark the test as failed.

one_test_per_line

Boolean

True or False

For tests with a lot of networks, you can tune the deepness of the tests per line. This option runs only one test per test line, randomize the selection of src, srcport, dst and dstport from the values in the test line.

max_tests_per_line

Integer

0 to N

For tests with a lot of networks, you can tune the deepness of the tests per line. This options sets a maximum number of tests to be run for test line, set it to 0 for no limit.

test_one_port_per_network_pair

Boolean

True or False

For tests with a lot of networks, you can tune the deepness of the tests per line. This option test only one port per src and dst pair. The port is choosen randomly.

debug_level

Integer

0 to 100

Enable debug and sets the required level. Debug messages are witten to output/logs/fwtester.log file.

Once the csv files are provided and the config file stablished you can run the tests by going to the fwtester folder and running the command ./fwtester.py or python fwtester.py

The system begins reading the tests.csv file, and do the following per each line:

  1. Load the LXC containers, if not loaded (It should occurs once in the fist test).

  2. Check if the test line is stored in the MongoDB database as done, if its on database skip the test, this allows to stop and resume a long number of tests.

  3. Gets de fields defined by the test line.

  4. Determines the test’s topology, how client and server connect to the firewall (basically if there is a need of intermediate routers) and it it’s required or not to run the server side

  5. Configures the required containers.

  6. Chek if the protocol or application are supported by any protocol/application module.

  7. Execute the test and get the result.

  8. Write the test result to database.

  9. Depending of the deepness limits it skip to the next line or gets new values from the same test line. Becareful with wide networks and port test with no deepness limits it can get very very long time to finish.

  10. Once all defined tests are finished a csv format report is generated with all the results in output/reports folder

Modules

The system is designed to be extensible by modules, to increase the supported protocols, and also to be able to run application tests. The module system is very independent from the core and flexible. This way the coder have fully freedom to code. The current modules are based on python scapy but any Python code can be used freely.

There are 3 tipes of modules: Layer 2, Layer 3 and Layer 7. Currently supoort for L3 and L7 modules is implemented.

Here is a list of modules already developed:

Name

Layer

Description

tcp

3

Module to run test for TCP connections by running a simple tcp session.

udp

3

Module to run test for UDP connections by running a simple udp connection. As UDP is not session oriented this module rely on sending a message tha should receive back, is intended for testing in lab environments.

icmp

3

Module to run test for ICMP traffic by sending an echo-request and expecting an echo-replay as response.

dhcp

7

Module to test DHCP protocol by sending a dhcp-request and expecting a dhcp-offer. It support multicast and unicast dhcp-requests, take in mind that to cross a fw it must be a dhcp-rely on the network, otherrwise test will fail.

dns

7

Module to test DNS protocol by sending a dns request and waiting for a response. It supports TCP and UDP as transport protocols

ntp

7

Module to test NTP protocol by sending a ntp request and waiting for a response. It supports TCP and UDP as transport protocols

Installation

Install from scratch (physical or virtual device)

This method is intended to be used on a clean installed machine, if you are running it on a non clean install machine, be aware that it can break the system. Specifically it can break network configuration since it change the logic of the names to old style ethX, and do changes to /etc/network/interfaces with the assumption that it is a fresh intall.

  1. Download install script

  1. login as root or any user, if the loggedin user is not root run command su -

  2. wget https://gitlab.com/abdulet/fwtester/-/raw/1.2/install-debian.sh?ref_type=heads&inline=false

  1. Run the install script as root

  • bash install-debian.sh

  1. Machine will reboot.

  2. After reboot login again and if there is a DHCP server on the network eth0 should be configured by DHCP. If there is not DHCP server on the network, set the ip address by hand following the Debian standard instructions. Edit /etc/network/interfaces.

  1. sudo vi /etc/network/interfaces

  2. Set management IP address on eth0 interface to an static known one, if desired

  1. Edit confguration files:

  1. cd csvdata/

  2. Copy files

  1. fwnics.csv.template fwnic.csv

  2. routes.csv.template routes.csv

  3. tests.csv.template tests.csv

  1. Edit fwnics.csv and fill it with real firewall NICs, one line per NIC:

  1. Name: A descriptive name for the NIC, usually the security zone

  2. IP: The IP address of the NIC

  3. Mask: The mask of the NIC

  4. Gateway: The gateway of the network for the NIC

  1. Edit routes.csv and fill it with the routing information from the FW:

  1. Interface: The route’s interface (exit interface)

  2. IP: The ip address of the route

  3. Mask: The mask for the route

  4. Gateway: The next hop’s IP address

  1. Edit tests.csv and add the tests to run