Getting started with DAQ
In this tutorial you will download and install DAQ. You will then run the first test with virtual ethernet ports and subsequently move to testing a single device and multiple devices in parallel.
The pre-requisites for this tutorial are a Debian based Linux distribution (Debian GNU/Linux 9.4 Stretch is the recommended one) and the git tool. If you don't have it yet, open a terminal window and install it with the following command:
sudo apt install git
The first step to install DAQ is to download its source code from the DAQ github repository:
git clone https://github.com/faucetsdn/daq.git
Once the software is downloaded, go to the DAQ folder
cd daq
and run the installation with this command:
bin/setup_base
You will be asked to enter your administration password.
The setup_base command installs a minimum set of basic packages, including docker and Open vSwitch.
bin/setup_dev
The setup_dev command installs the development environment dependencies, including python3, various network tools, a Java development kit and specific versions of Mininet and Faucet. The setup of the development environment is required to work with mudacl, the MUD ACL genenerator.
Once the base and development software packages are installed, build DAQ with the following command:
sudo cmd/build
This program is likely to take a long time to execute, so go get yourself a cup of your favourite hot or cool beverage.
If you want to monitor the build progress, look inside the build folder. The build command outputs several log files that you can show in the command line, for instance with the following command:
tail -f build/docker_build.aardvark
Once DAQ has been built, execute this command to run the DAQ tests for the first time.
cmd/run -s
This command will ask you for the administration password, and on first run creates the local directory with a base system configuration file local/system.yaml
It then activates Open vSwitch, configures Faucet with a single virtual ethernet port (faux) and executes the tests. The option -s stands for "single shot" and executes the tests only once.
If all goes well, the console will show something similar to the log below. In this case we have installed DAQ in the /opt/daq directory.
opt@ubuntu:~/opt/daq$ cmd/run -s
Activating venv
Flattening config from local/system.yaml into inst/config/system.conf
Starting Mon 22 Feb 17:38:57 GMT 2021
Clearing previous state...
Activating venv
Flattening config from local/system.yaml into inst/config/system.conf
HEAD is now at 778ed989 Merge pull request #3726 from gizmoguy/test-base-8.0.6
HEAD is now at 5131c8f0 Include assigned vlan in grpc (#274)
Release version
Cleaning bridge ctrl-br...
ovsdb-server is running with pid 362
ovs-vswitchd is running with pid 431
No external switch model specified.
daq-usi
iptables: Bad rule (does a matching rule exist in that chain?).
DAQ autostart gcp_cred= cmd/usi
Starting USI in debug mode
4452ebbfac4ab566a73e00bba7591ae4a8adba39f4d5e3cb73b07ada18f70f5a
DAQ autoclean docker cp daq-usi:/root/logs.txt inst/cmdusi.log
DAQ autoclean docker kill daq-usi
Activating venv
Flattening config from local/system.yaml into inst/config/system.conf
Autostarting system components...
DAQ autostart cmd/faux
Activating venv
Flattening config from local/system.yaml into inst/config/system.conf
Launching faux ...
DAQ autoclean docker kill daq-faux
Removing old interface faux
Adding new interface to 24981...
Done with faux device launch.
Activating venv
Flattening config from local/system.yaml into inst/config/system.conf
No LSB modules are available.
Entering virtual python environment...
Using python3 at /opt/daq/venv/bin/python3
Prepending /opt/daq/binhack to PATH
Executing: python3 daq/daq.py local/system.yaml -s usi_setup.url=172.17.0.1:5000
/opt/daq/mininet/mininet/cli.py:147: SyntaxWarning: "is" with a literal. Did you mean "=="?
if line is '':
/opt/daq/mininet/mininet/cli.py:447: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if key is not '':
processing arg: local/system.yaml
Including config file /opt/daq/config/system/base.yaml
Including config file /opt/daq/config/system/default.yaml
processing arg: -s
processing arg: usi_setup.url=172.17.0.1:5000
base_conf=/opt/daq/resources/setups/baseline/base_config.json
default_timeout_sec=350
dhcp_lease_time=500s
finish_hook=/opt/daq/bin/dump_network
initial_dhcp_lease_time=120s
interfaces.faux.opts=None
internal_subnet.subnet=10.20.0.0/16
long_dhcp_response_sec=105
monitor_scan_sec=30
port_flap_timeout_sec=5
settle_sec=5
single_shot=True
site_description="Baseline Configuration"
site_path=local/site/
switch_setup.of_dpid=2
switch_setup.uplink_port=7
topology_hook=/opt/daq/bin/dump_network
usi_setup.rpc_timeout_sec=20
usi_setup.url=172.17.0.1:5000
Feb 22 17:39:02 daq INFO pid is 25143
Feb 22 17:39:02 gcp INFO No gcp_cred file specified in config, disabling gcp use.
Feb 22 17:39:02 runner INFO Loading base config from /opt/daq/resources/setups/baseline/base_config.json
Feb 22 17:39:02 config INFO Including config file /opt/daq/resources/setups/baseline/../common/base_config.json
Feb 22 17:39:02 runner INFO Loading site config from local/site/site_config.json
Feb 22 17:39:02 config INFO Skipping missing config file local/site/site_config.json
Feb 22 17:39:02 topology INFO No device_specs file specified, skipping...
Feb 22 17:39:02 runner INFO Reading test definition file /opt/daq/config/modules/host.conf
Feb 22 17:39:02 runner INFO Reading test definition file /opt/daq/subset/pentests/build.conf
Feb 22 17:39:02 runner INFO Reading test definition file /opt/daq/usi/build.conf
Feb 22 17:39:02 runner INFO Reading test definition file /opt/daq/subset/ipaddr/build.conf
Feb 22 17:39:02 runner INFO DAQ RUN id: 6ad97612-0362-4617-a795-91ec9d6ab33c
Feb 22 17:39:02 runner INFO Configured with tests pass, fail, ping, bacnet, mudgee, nmap, discover, ipaddr
Feb 22 17:39:02 runner INFO DAQ version 1.10.2-13-gd02ca8c
Feb 22 17:39:02 runner INFO LSB release Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic
Feb 22 17:39:02 runner INFO system uname Linux ubun2 5.4.0-65-generic #73~18.04.1-Ubuntu SMP Tue Jan 19 09:02:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Feb 22 17:39:02 network INFO Activating faucet topology...
Feb 22 17:39:02 topology INFO Starting faucet...
Feb 22 17:39:13 network INFO Initializing faucitizer...
Feb 22 17:39:13 network INFO Waiting 5s for network to settle
Feb 22 17:39:18 network INFO Creating ovs sec with dpid/port 2/7
Feb 22 17:39:18 network INFO Added switch link pri-eth1 <-> sec-eth7
Feb 22 17:39:18 network INFO Attaching device interface faux on port 1.
Feb 22 17:39:18 network INFO Attaching device interface sec-2 on port 2.
Feb 22 17:39:18 network INFO Attaching device interface sec-3 on port 3.
Feb 22 17:39:18 network INFO Attaching device interface sec-4 on port 4.
Feb 22 17:39:18 network INFO Attaching device interface sec-5 on port 5.
Feb 22 17:39:18 network INFO Attaching device interface sec-6 on port 6.
Feb 22 17:39:18 network INFO Starting mininet...
Feb 22 17:39:18 mininet INFO *** Configuring hosts
Feb 22 17:39:18 mininet INFO *** Starting controller
Feb 22 17:39:18 mininet INFO controller
Feb 22 17:39:18 mininet INFO *** Starting 2 switches
Feb 22 17:39:18 mininet INFO pri
Feb 22 17:39:18 mininet INFO sec
Feb 22 17:39:18 mininet INFO ...
Feb 22 17:39:18 runner INFO Waiting for system to settle...
Feb 22 17:39:21 runner INFO Entering main event loop.
Feb 22 17:39:21 runner INFO See docs/troubleshooting.md if this blocks for more than a few minutes.
Feb 22 17:39:21 runner INFO System port 1 on dpid 1 is active True
Feb 22 17:39:21 runner INFO Port 1 dpid 2 is now active
Feb 22 17:39:23 runner INFO Port 1 dpid 2 learned 9a:02:57:1e:8f:00
Feb 22 17:39:23 runner INFO Gateway for device group 9a02571e8f00 not found, initializing base 1...
Feb 22 17:39:23 gateway INFO Initializing gateway 9a02571e8f00 as gw01/10
Feb 22 17:39:24 gateway INFO Added networking host gw01 on port 10 at 10.20.0.1
Feb 22 17:39:25 gateway INFO Added dummy target dummy01 on port 11 at 10.20.0.2
Feb 22 17:39:25 runner INFO Test ping gw01->dummy01
Feb 22 17:39:36 runner INFO Test ping gw01->dummy01
Feb 22 17:39:37 runner INFO Test ping dummy01->gw01
Feb 22 17:39:38 runner INFO Test ping dummy01->192.168.84.1
Feb 22 17:39:39 runner INFO Test ping gw01->dummy01 from 192.168.84.1
Feb 22 17:39:40 gateway INFO Gateway 1 change lease time to 120s
Feb 22 17:39:40 gateway INFO Gateway 1 startup capture gw01-eth0 in container's /tmp/gateway.pcap
Feb 22 17:39:40 dhcp INFO DHCP monitor gw01 waiting for replies...
Feb 22 17:39:40 gateway INFO Attaching target 9a02571e8f00 to gateway group 9a02571e8f00
Feb 22 17:39:40 host WARNING Creating new device dir: /opt/daq/local/site/mac_addrs/9a02571e8f00
Feb 22 17:39:40 host WARNING Skipping defaults since no device_template found
Feb 22 17:39:40 host INFO Loading device module config from /opt/daq/local/site/mac_addrs/9a02571e8f00/device_config.json
Feb 22 17:39:40 host INFO Device config reloaded: True 9a02571e8f00 on port 1
Feb 22 17:39:40 config INFO Writing config to /opt/daq/local/site/mac_addrs/9a02571e8f00/aux/module_config.json
Feb 22 17:39:40 host INFO Host 9a:02:57:1e:8f:00 running with enabled tests ['pass', 'fail', 'ping', 'nmap']
Feb 22 17:39:40 report INFO Writing report to inst/reports/report_9a02571e8f00_2021-02-22T173940.*
Feb 22 17:39:40 report INFO Writing alternate report to local/site/mac_addrs/9a02571e8f00/report.*
Feb 22 17:39:40 host INFO Target device 9a02571e8f00 on port 1 initializing...
Feb 22 17:39:42 host INFO Loading base module config from /opt/daq/local/site/mac_addrs/9a02571e8f00/base_config.json
Feb 22 17:39:42 network INFO Creating mirror pair mirror-1 <-> mirror-1-ext at 1001
Feb 22 17:39:42 host INFO Executing topology_hook: /opt/daq/bin/dump_network inst/network
Feb 22 17:39:42 host INFO Target device 9a02571e8f00 on port 1 startup pcap capture
Feb 22 17:39:42 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for infinite seconds output in run-9a02571e8f00/scans/startup.pcap
Feb 22 17:39:42 network INFO Directing traffic for 9a:02:57:1e:8f:00 on port 1 to 1
Feb 22 17:39:42 network INFO Waiting 5s for network to settle
Feb 22 17:39:47 runner INFO Port 10 dpid 1 learned 5e:8f:b6:22:f9:40 (ignored)
Feb 22 17:39:47 runner INFO Port 7 dpid 2 learned 5e:8f:b6:22:f9:40 (ignored)
Feb 22 17:39:47 runner INFO Port 11 dpid 1 learned 7a:dd:ec:e5:d6:a4 (ignored)
Feb 22 17:39:47 runner INFO Port 7 dpid 2 learned 7a:dd:ec:e5:d6:a4 (ignored)
Feb 22 17:39:47 host INFO Target device 9a02571e8f00 on port 1 waiting for ip
Feb 22 17:39:47 host INFO Target device 9a02571e8f00 on port 1 using NORMAL DHCP mode, wait 0
Feb 22 17:39:51 runner INFO Port 1 dpid 2 learned 9a:02:57:1e:8f:00
Feb 22 17:39:51 runner INFO Port 7 dpid 2 learned 5e:8f:b6:22:f9:40 (ignored)
Feb 22 17:39:51 runner INFO Port 1 dpid 1 learned 9a:02:57:1e:8f:00 (ignored)
Feb 22 17:39:51 runner INFO Port 10 dpid 1 learned 5e:8f:b6:22:f9:40 (ignored)
Feb 22 17:39:54 dhcp INFO DHCP monitor gw01 received Offer reply after 14s: 10.20.76.163/9a:02:57:1e:8f:00
Feb 22 17:39:54 runner INFO IP notify Offer 9a:02:57:1e:8f:00 is 10.20.76.163 on Gateway group 9a02571e8f00 set 1 (done/14)
Feb 22 17:39:54 dhcp INFO DHCP monitor gw01 received ACK reply after 14s: 10.20.76.163/9a:02:57:1e:8f:00
Feb 22 17:39:54 runner INFO IP notify ACK 9a:02:57:1e:8f:00 is 10.20.76.163 on Gateway group 9a02571e8f00 set 1 (done/14)
Feb 22 17:39:54 gateway INFO Ready target 9a02571e8f00 from gateway group 9a02571e8f00
Feb 22 17:39:54 gateway INFO Gateway 1 change lease time to 500s
Feb 22 17:39:54 runner INFO IP activating target 9a02571e8f00
Feb 22 17:39:54 host INFO Target device 9a02571e8f00 on port 1 triggered as 10.20.76.163
Feb 22 17:39:54 runner INFO Test ping gw01->10.20.76.163
Feb 22 17:39:55 runner INFO Test ping gw01->10.20.76.163
Feb 22 17:39:56 runner INFO Test ping gw01->10.20.76.163 from 192.168.84.1
Feb 22 17:39:57 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:39:57 host INFO Target device 9a02571e8f00 on port 1 done with base.
Feb 22 17:39:57 host INFO Target device 9a02571e8f00 on port 1 background pcap for 30s
Feb 22 17:39:57 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for 30 seconds output in run-9a02571e8f00/scans/monitor.pcap
Feb 22 17:39:58 runner INFO Port 11 dpid 1 learned 7a:dd:ec:e5:d6:a4 (ignored)
Feb 22 17:39:58 runner INFO Port 7 dpid 2 learned 7a:dd:ec:e5:d6:a4 (ignored)
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 pcap complete
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 start pass01
Feb 22 17:40:28 config INFO Writing config to inst/run-9a02571e8f00/nodes/pass01/tmp/module_config.json
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for infinite seconds output in run-9a02571e8f00/scans/test_pass.pcap
Feb 22 17:40:28 exmodule INFO Target subnet 10.20.76.163/32 overlaps with runner subnet 10.20.0.0/16.
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test pass running
Feb 22 17:40:28 host INFO Executing finish_hook: /opt/daq/bin/dump_network inst/run-9a02571e8f00/finish/pass01
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test pass test host finalize 0
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test pass passed 0.274756s
Feb 22 17:40:28 host INFO Host callback pass/pass01 was 0 with None
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 start fail01
Feb 22 17:40:28 config INFO Writing config to inst/run-9a02571e8f00/nodes/fail01/tmp/module_config.json
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for infinite seconds output in run-9a02571e8f00/scans/test_fail.pcap
Feb 22 17:40:28 exmodule INFO Target subnet 10.20.76.163/32 overlaps with runner subnet 10.20.0.0/16.
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test fail running
Feb 22 17:40:28 host INFO Executing finish_hook: /opt/daq/bin/dump_network inst/run-9a02571e8f00/finish/fail01
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test fail test host finalize 1
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test fail failed 0.254763s: 1 None
Feb 22 17:40:28 host INFO Host callback fail/fail01 was 1 with None
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 start ping01
Feb 22 17:40:28 config INFO Writing config to inst/run-9a02571e8f00/nodes/ping01/tmp/module_config.json
Feb 22 17:40:28 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for infinite seconds output in run-9a02571e8f00/scans/test_ping.pcap
Feb 22 17:40:28 exmodule INFO Target subnet 10.20.76.163/32 overlaps with runner subnet 10.20.0.0/16.
Feb 22 17:40:28 exmodule INFO Target device 9a02571e8f00 test ping running
Feb 22 17:40:28 runner INFO Port 7 dpid 2 learned 42:8d:c4:70:ee:0e (ignored)
Feb 22 17:40:28 runner INFO Port 12 dpid 1 learned 42:8d:c4:70:ee:0e (ignored)
Feb 22 17:40:38 host INFO Executing finish_hook: /opt/daq/bin/dump_network inst/run-9a02571e8f00/finish/ping01
Feb 22 17:40:38 exmodule INFO Target device 9a02571e8f00 test ping test host finalize 0
Feb 22 17:40:38 exmodule INFO Target device 9a02571e8f00 test ping passed 9.493804s
Feb 22 17:40:38 host INFO Host callback ping/ping01 was 0 with None
Feb 22 17:40:38 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:40:38 host INFO Target device 9a02571e8f00 on port 1 start nmap01
Feb 22 17:40:38 config INFO Writing config to inst/run-9a02571e8f00/nodes/nmap01/tmp/module_config.json
Feb 22 17:40:38 host INFO Target device 9a02571e8f00 on port 1 pcap intf mirror-1 for infinite seconds output in run-9a02571e8f00/scans/test_nmap.pcap
Feb 22 17:40:38 exmodule INFO Target subnet 10.20.76.163/32 overlaps with runner subnet 10.20.0.0/16.
Feb 22 17:40:38 exmodule INFO Target device 9a02571e8f00 test nmap running
Feb 22 17:40:38 runner INFO Port 7 dpid 2 learned c6:7b:93:2d:2a:c4 (ignored)
Feb 22 17:40:38 runner INFO Port 12 dpid 1 learned c6:7b:93:2d:2a:c4 (ignored)
Feb 22 17:40:47 dhcp INFO DHCP monitor gw01 received ACK reply after 53s: 10.20.76.163/9a:02:57:1e:8f:00
Feb 22 17:40:47 runner INFO IP notify ACK 9a:02:57:1e:8f:00 is 10.20.76.163 on Gateway group 9a02571e8f00 set 1 (done/53)
Feb 22 17:40:47 runner INFO DHCP activation group 9a02571e8f00 already activated
Feb 22 17:41:29 host INFO Executing finish_hook: /opt/daq/bin/dump_network inst/run-9a02571e8f00/finish/nmap01
Feb 22 17:41:29 exmodule INFO Target device 9a02571e8f00 test nmap test host finalize 0
Feb 22 17:41:29 exmodule INFO Target device 9a02571e8f00 test nmap passed 51.586738s
Feb 22 17:41:29 host INFO Host callback nmap/nmap01 was 0 with None
Feb 22 17:41:29 host INFO Target device 9a02571e8f00 on port 1 network pcap complete
Feb 22 17:41:29 host INFO Target device 9a02571e8f00 on port 1 no more tests remaining
Feb 22 17:41:29 runner INFO Target device 9a:02:57:1e:8f:00 cancel (#1/0).
Feb 22 17:41:29 network INFO Directing traffic for 9a:02:57:1e:8f:00 on port 1 to None
Feb 22 17:41:29 network INFO Waiting 5s for network to settle
Feb 22 17:41:34 gateway INFO Detach target 9a02571e8f00 from gateway group 9a02571e8f00: ['9a:02:57:1e:8f:00']
Feb 22 17:41:34 runner INFO Retiring Gateway group 9a02571e8f00 set 1. Last device: 9a02571e8f00
Feb 22 17:41:34 gateway INFO Terminating gateway 1/9a02571e8f00
Feb 22 17:41:35 host INFO Target device 9a02571e8f00 on port 1 terminate, running unknown, trigger False: _target_set_cancel
Feb 22 17:41:35 network INFO Deleting mirror pair mirror-1 <-> mirror-1-ext
Feb 22 17:41:35 report INFO Finalizing report_9a02571e8f00_2021-02-22T173940
Feb 22 17:41:35 report INFO Skipping missing report header template local/site/report_template.md
Feb 22 17:41:35 report INFO Copying test report inst/run-9a02571e8f00/nodes/ping01/tmp/report.txt
Feb 22 17:41:35 report INFO Copying test report inst/run-9a02571e8f00/nodes/nmap01/tmp/report.txt
Feb 22 17:41:35 report INFO Generating HTML for writing pdf report...
Feb 22 17:41:35 report INFO Metamorphosising HTML to PDF...
Feb 22 17:41:35 weasyprint WARNING Expected a media type, got only/**/screen/**/and/**/(min-width: 480px)
Feb 22 17:41:35 weasyprint WARNING Invalid media type " only screen and (min-width: 480px) " the whole @media rule was ignored at 231:1.
Feb 22 17:41:35 weasyprint WARNING Expected a media type, got only/**/screen/**/and/**/(min-width: 768px)
Feb 22 17:41:35 weasyprint WARNING Invalid media type " only screen and (min-width: 768px) " the whole @media rule was ignored at 236:1.
Feb 22 17:41:36 report INFO Copying reports to local/site/mac_addrs/9a02571e8f00/report.*
Feb 22 17:41:36 host INFO Finalized with reports ['report_path.md', 'report_path.pdf', 'report_path.json', 'trigger_path']
Feb 22 17:41:36 runner INFO Target device 9a02571e8f00 finalize: [] (target set not active)
Feb 22 17:41:36 runner WARNING Suppressing future tests because test done in single shot.
Feb 22 17:41:36 runner INFO Port 1 dpid 2 learned 9a:02:57:1e:8f:00
Feb 22 17:41:36 runner INFO Remaining target sets: []
Feb 22 17:41:36 stream INFO Monitoring 0 fds
Feb 22 17:41:36 runner WARNING No active ports remaining (0 monitors), ending test run.
Feb 22 17:41:36 runner INFO Stopping network...
Feb 22 17:41:36 mininet INFO *** Stopping 1 controllers
Feb 22 17:41:36 mininet INFO controller
Feb 22 17:41:36 mininet INFO *** Stopping 1 links
Feb 22 17:41:36 mininet INFO .
Feb 22 17:41:36 mininet INFO *** Stopping 2 switches
Feb 22 17:41:37 mininet INFO pri
Feb 22 17:41:37 mininet INFO sec
Feb 22 17:41:37 mininet INFO *** Stopping 1 hosts
Feb 22 17:41:37 mininet INFO gw01
Feb 22 17:41:37 mininet INFO *** Done
Feb 22 17:41:37 runner INFO Done with runner.
Feb 22 17:41:37 daq INFO DAQ runner returned 0
Cleanup docker cp daq-usi:/root/logs.txt inst/cmdusi.log
Cleanup docker kill daq-usi
daq-usi
Cleanup docker kill daq-faux
daq-faux
Done with run, exit 0
While running the test, switch to a different console terminal window and check which docker containers are running:
sudo docker ps
The output will be similar to the following:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 40125f7e070b daq/networking "env TERM=dumb PS1=\u007f…" 23 seconds ago Up 21 seconds daq-gw01 bd1114d18b2c daq/faucet "/usr/local/bin/entr…" 32 seconds ago Up 29 seconds 0.0.0.0:6653->6653/tcp daq-faucet a5fd5cc321d4 daq/faux "bin/start_faux" 34 seconds ago Up 32 seconds daq-faux
After successfully exiting the test, DAQ will have created a report in the inst folder. In this case the report is named inst/reports/report_9a02571e8f00_2018-11-24T22:32:58.txt, showing the ethernet MAC address and report date in the name.
The local/system.yaml is the configuration file to use to set up the DAQ network topology and some of the run time configuration variables without having to put them in the command line.
DAQ supports many topologies categories, as documented in the reference page on Network Topologies: emulation, adapter, test lab, tiered and production.
In the previous section, DAQ configured and executed the tests on an emulated network interface (named faux), as indicated by this output line:
INFO:network:Attaching device interface faux on port 1.
We'll now proceed to configure DAQ to use a physical USB ethernet network adapter.
Once the adapter is plugged into the USB port, it will add a network interface.
Identify it using the ifconfig command:
sudo ifconfig
See below an example of the output, showing the network interface has been registered as enx000ec6c32a6f.
...
enx000ec6c32a6f: flags=4163 mtu 1500
inet6 fe80::20e:c6ff:fec3:2a6f prefixlen 64 scopeid 0x20
ether 00:0e:c6:c3:2a:6f txqueuelen 1000 (Ethernet)
RX packets 1233820 bytes 83057697 (79.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2698368 bytes 204281978 (194.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
...
In local/system.yaml, replace the faux entry under interfaces with the network interface name, which in this example is enx000ec6c32a6f . Unlike faux devices, physical interfaces do not have an opts key, but are rather have a port key, the value of which must correspond to a virtual port the deviced is attatched to. The corresponding entry looks as follows:
interfaces:
enx000ec6c32a6f:
port: 1
The default configuration file in a fresh setup of DAQ is as follows.
# Base configuration file for DAQ.
#
# This setup will create and test one faux device. To specify a real USB
# adapter, modify the interfaces entry with the appripriate interface name(s).
# Load defaults.
include: default.yaml
# Description for dashboard.
site_description: "Baseline Configuration"
# Faux interface devices
interfaces:
faux:
opts:
With the ammendments to replace the faux device with a phyiscal intercace, the configuration file looks as follows.
# Base configuration file for DAQ.
#
# This setup will create and test one faux device. To specify a real USB
# adapter, modify the interfaces entry with the appripriate interface name(s).
# Load defaults.
include: default.yaml
# Description for dashboard.
site_description: "Baseline Configuration"
# Faux interface devices
interfaces:
enx000ec6c32a6f:
port: 1
Now connect a device to the USB ethernet adapter, and execute again the "single shot" tests:
cmd/run -s
The tests are designed to run in parallel on multiple devices. Get at least another ethernet adapter, find its ID:
sudo ifconfig
...
enx9cebe840a74c: flags=4163 mtu 1500
ether 9c:eb:e8:40:a7:4c txqueuelen 1000 (Ethernet)
RX packets 62563 bytes 20983980 (20.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 29 bytes 5141 (5.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
...
And include it inside the local/system.yaml file following the same structure for a phyiscal device mentioned earlier, however the second physical interface will have a port of 2.
Then, for a fresh setup of DAQ, the local/system.yaml for two phyiscal devices looks as follows:
# Base configuration file for DAQ.
#
# This setup will create and test one faux device. To specify a real USB
# adapter, modify the interfaces entry with the appripriate interface name(s).
# Load defaults.
include: default.yaml
# Description for dashboard.
site_description: "Baseline Configuration"
# Faux interface devices
interfaces:
enx000ec6c32a6f:
port: 1
enx9cebe840a74c:
port: 2
Now execute again the "single shot" tests:
sudo cmd/run -s
Check which docker containers have been created by DAQ:
sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 48ab066371e2 daq/networking "env TERM=dumb PS1=\u007f…" 25 seconds ago Up 23 seconds daq-gw01 d198e6b78742 daq/networking "env TERM=dumb PS1=\u007f…" About a minute ago Up About a minute daq-gw02 3b60351f2e8d daq/faucet "/usr/local/bin/entr…" About a minute ago Up About a minute 0.0.0.0:6653->6653/tcp daq-faucet
The DAQ command line output will confirm that the two USB ethernet adapters have been connected.
Loading config from local/system.conf
Starting Fri 21 Dec 14:19:46 GMT 2018, run_mode is local
Clearing previous reports...
Running as root...
Loading config from local/system.conf
Release version 0.9.0
cleanup='echo cleanup'
intf_names=enx000ec6c32a6f,enx9cebe840a74c
run_mode=local
site_description='Two devices'
skip_autostart=
ovsdb-server is running with pid 528
ovs-vswitchd is running with pid 20151
Entering virtual python environment...
Using python3 at /opt/daq/venv/bin/python3
Executing: python3 -u daq/daq.py /opt/daq/local/system.conf -s
processing arg: /opt/daq/local/system.conf
Reading config from /opt/daq/local/system.conf
Reading config from misc/system.conf
processing arg: -s
INFO:daq:configuration map: {'single_shot': True, 'site_description': 'Two devices', 'intf_names': 'enx000ec6c32a6f,enx9cebe840a74c', 'run_mode': 'local'}
INFO:daq:pid is 20586
INFO:gcp:No gcp_cred credential specified in config
INFO:runner:Reading test definition file misc/host_tests.conf
INFO:runner:Reading test definition file subset/pentests/pentests.conf
INFO:runner:Configured with tests ['pass', 'fail', 'ping', 'bacnet', 'mudgee', 'nmap', 'brute']
INFO:network:Activating faucet topology...
INFO:topology:No device_specs file specified, skipping...
INFO:topology:Writing network config to inst/faucet.yaml
INFO:topology:Starting faucet...
INFO:network:Creating ovs sec with dpid/port 2/7
INFO:network:Added switch link pri-eth1 <-> sec-eth7
INFO:network:Attaching device interface enx000ec6c32a6f on port 1.
INFO:network:Attaching device interface enx9cebe840a74c on port 2.
INFO:network:Attaching device interface sec-3 on port 3.
INFO:network:Attaching device interface sec-4 on port 4.
INFO:network:Attaching device interface sec-5 on port 5.
INFO:network:Attaching device interface sec-6 on port 6.
INFO:network:Starting mininet...
INFO:mininet:*** Configuring hosts
INFO:mininet:*** Starting controller
INFO:mininet:controller
INFO:mininet:*** Starting 2 switches
INFO:mininet:pri
INFO:mininet:sec
INFO:mininet:...
INFO:runner:Waiting for system to settle...
INFO:runner:Entering main event loop.
INFO:runner:If this blocks for too long, check inst/faucet.log for errors
INFO:runner:Port 1 dpid 2 is now active True
INFO:runner:Port 2 dpid 2 is now active True
INFO:runner:System port 7 on dpid 2 is active True
INFO:runner:Port 2 dpid 2 learned 38:d1:35:02:00:50
INFO:runner:Gateway for device group 38d135020050 not found, initializing base 2...
INFO:gateway:Initializing gateway 38d135020050 as gw02/20
INFO:gateway:Adding networking host gw02 on port 20 at 10.0.0.1
INFO:gateway:Added dummy target dummy02 on port 21 at 10.0.0.2
INFO:dhcp:DHCP monitor gw02 waiting for replies...
INFO:runner:Test ping gw02->dummy02
INFO:runner:Test ping gw02->dummy02
INFO:runner:Test ping gw02->dummy02
INFO:runner:Test ping dummy02->gw02
INFO:runner:Test ping dummy02->192.168.84.2
INFO:runner:Test ping gw02->dummy02 from 192.168.84.2
INFO:gateway:Attaching target 2 to gateway group 38d135020050
INFO:host:Creating report as inst/report_38d135020050_2018-12-21T14:20:16.txt
INFO:runner:Target port 2 registered 38:d1:35:02:00:50
INFO:host:Target port 2 initializing...
INFO:network:Creating mirror pair mirror-02 <-> mirror-02-ext at 1002
INFO:host:Target port 2 startup pcap capture
INFO:network:Directing traffic for 38:d1:35:02:00:50 on port 2: True
INFO:host:Target port 2 waiting for dhcp as 38:d1:35:02:00:50
INFO:gateway:Ready target 38:d1:35:02:00:50 from gateway group 38d135020050
INFO:runner:DHCP activating target 38:d1:35:02:00:50
INFO:host:Target port 2 triggered as 10.20.33.179
INFO:runner:Test ping gw02->10.20.33.179
INFO:runner:Test ping gw02->10.20.33.179
INFO:runner:Test ping gw02->10.20.33.179 from 192.168.84.2
INFO:host:Target port 2 monitor scan complete
INFO:host:Target port 2 done with base.
INFO:host:Target port 2 background scan for 30 seconds...
INFO:runner:Port 1 dpid 2 learned 04:a3:16:ea:06:33
INFO:runner:Gateway for device group 04a316ea0633 not found, initializing base 1...
INFO:gateway:Initializing gateway 04a316ea0633 as gw01/10
INFO:gateway:Adding networking host gw01 on port 10 at 10.0.0.3
INFO:gateway:Added dummy target dummy01 on port 11 at 10.0.0.4
INFO:dhcp:DHCP monitor gw01 waiting for replies...
INFO:runner:Test ping gw01->dummy01
INFO:runner:Test ping gw01->dummy01
INFO:runner:Test ping gw01->dummy01
INFO:runner:Test ping dummy01->gw01
INFO:runner:Test ping dummy01->192.168.84.1
INFO:runner:Test ping gw01->dummy01 from 192.168.84.1
INFO:gateway:Attaching target 1 to gateway group 04a316ea0633
INFO:host:Creating report as inst/report_04a316ea0633_2018-12-21T14:20:58.txt
INFO:runner:Target port 1 registered 04:a3:16:ea:06:33
INFO:host:Target port 1 initializing...
INFO:network:Creating mirror pair mirror-01 <-> mirror-01-ext at 1001
INFO:host:Target port 1 startup pcap capture
INFO:network:Directing traffic for 04:a3:16:ea:06:33 on port 1: True
INFO:host:Target port 1 waiting for dhcp as 04:a3:16:ea:06:33
INFO:host:Target port 2 scan complete
INFO:host:Target port 2 monitor scan complete
INFO:docker:Target port 2 test pass running
INFO:docker:Target port 2 test pass passed
INFO:docker:Target port 2 test fail running
INFO:docker:Target port 2 test fail failed: 1 None
INFO:docker:Target port 2 test ping running
INFO:docker:Target port 2 test ping passed
INFO:docker:Target port 2 test bacnet running
INFO:docker:Target port 2 test bacnet passed
INFO:docker:Target port 2 test mudgee running
INFO:docker:Target port 2 test mudgee passed
INFO:docker:Target port 2 test nmap running
INFO:gateway:Ready target 04:a3:16:ea:06:33 from gateway group 04a316ea0633
INFO:runner:DHCP activating target 04:a3:16:ea:06:33
INFO:host:Target port 1 triggered as 10.20.16.91
INFO:runner:Test ping gw01->10.20.16.91
INFO:runner:Test ping gw01->10.20.16.91
INFO:runner:Test ping gw01->10.20.16.91 from 192.168.84.1
INFO:host:Target port 1 monitor scan complete
INFO:host:Target port 1 done with base.
INFO:host:Target port 1 background scan for 30 seconds...
INFO:runner:Port 1 dpid 2 learned 04:a3:16:ea:06:33
INFO:host:Target port 1 scan complete
INFO:host:Target port 1 monitor scan complete
INFO:docker:Target port 1 test pass running
INFO:docker:Target port 1 test pass passed
INFO:docker:Target port 1 test fail running
INFO:docker:Target port 1 test fail failed: 1 None
INFO:docker:Target port 1 test ping running
INFO:docker:Target port 1 test ping passed
INFO:docker:Target port 1 test bacnet running
INFO:docker:Target port 1 test bacnet passed
INFO:docker:Target port 1 test mudgee running
INFO:docker:Target port 1 test mudgee passed
INFO:docker:Target port 1 test nmap running
INFO:runner:Port 1 dpid 2 learned 04:a3:16:ea:06:33
INFO:docker:Target port 2 test nmap failed: 1 None
INFO:docker:Target port 2 test brute running
INFO:docker:Target port 2 test brute passed
INFO:host:Target port 2 no more tests remaining
INFO:host:Finalizing report inst/report_38d135020050_2018-12-21T14:20:16.txt
INFO:gcp:Ignoring report upload: not configured
INFO:runner:Target port 2 finalize: ['02:nmap:1'] (target set not active)
INFO:runner:Target port 2 cancel 38:d1:35:02:00:50 (#2/0).
INFO:network:Directing traffic for 38:d1:35:02:00:50 on port 2: False
INFO:host:Target port 2 terminate, trigger False
INFO:network:Deleting mirror pair mirror-02 <-> mirror-02-ext
INFO:gateway:Detach target 2 from gateway group 38d135020050
INFO:runner:Retiring target gateway 2, 38:d1:35:02:00:50, 38d135020050, 2
INFO:gateway:Terminating gateway 38d135020050
WARNING:runner:Suppressing future tests because test done in single shot.
INFO:runner:Remaining target sets: [1]
INFO:runner:Port 1 dpid 2 learned 04:a3:16:ea:06:33
INFO:docker:Target port 1 test nmap failed: 1 None
INFO:docker:Target port 1 test brute running
INFO:docker:Target port 1 test brute passed
INFO:host:Target port 1 no more tests remaining
INFO:host:Finalizing report inst/report_04a316ea0633_2018-12-21T14:20:58.txt
INFO:gcp:Ignoring report upload: not configured
INFO:runner:Target port 1 finalize: ['01:nmap:1'] (target set not active)
INFO:runner:Target port 1 cancel 04:a3:16:ea:06:33 (#2/0).
INFO:network:Directing traffic for 04:a3:16:ea:06:33 on port 1: False
INFO:host:Target port 1 terminate, trigger False
INFO:network:Deleting mirror pair mirror-01 <-> mirror-01-ext
INFO:gateway:Detach target 1 from gateway group 04a316ea0633
INFO:runner:Retiring target gateway 1, 04:a3:16:ea:06:33, 04a316ea0633, 1
INFO:gateway:Terminating gateway 04a316ea0633
INFO:runner:Remaining target sets: []
WARNING:runner:No active ports remaining: ending test run.
INFO:mininet:*** Stopping 1 controllers
INFO:mininet:controller
INFO:mininet:*** Stopping 1 links
INFO:mininet:.
INFO:mininet:*** Stopping 2 switches
INFO:mininet:pri
INFO:mininet:sec
INFO:mininet:*** Stopping 1 hosts
INFO:mininet:gw02
INFO:mininet:*** Done
INFO:runner:Done with runner.
Sometimes using sudo can leave the files with the wrong ownership.
To revert them to the current user, use the following command:
sudo chown $USER -R /path/to/daq
Sometimes, corporate IT departments install software that rewrite SSL certificates. This could mean that you won't be able to download various packages necessary to build and run DAQ, e.g. parts of gradle, various debian packages etc etc from https:// URLs. This issue is most prevalent if you're running a virtual machine on your corporate laptop (e.g. via Parallels/VirtualBox/Hyper-V)
The generalised (but by no means complete) solution is to contact your IT department and install the root certificate into your debian flavoured VM and, most imporantly for DAQ, the open-jdk .truststore. This should fix any issues you're having navigating to URLs you can access just fine on your company approved operating system. The steps are as follows:
- Get the root certificate from your IT department, preferably in
.cerformat (this method has been tested with the Cisco Umbrella root cert) - Convert your
.cercert to a.crt - Add it to your operating system's certificate store
- Remember to refresh your OS's idea of the certificate store (OS dependent)
- Add the certificate to the Java truststore
- DAQ uses open-jdk-8, if you're using enterprise Java there will be a separate truststore
- NB: For a fresh install of open-jdk, the default truststore password should be
changeit
Sources:
- The cert needs to be converted to crt (not just renamed): https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate
- And then added to the ubuntu cert store
- Refresh the ubuntu cert store
- Then convert to .der and add to the open-jdk truststore: https://plone.lucidsolutions.co.nz/linux/java/how-to-add-a-certificate-authority-ca-certificate-to-the-openjdk-cacerts
Go to the next tutorial > Set up the DAQ dashboard