Linux cooked-mode capture (SLL) support

[showipintbri]

Before opening a new issue, please make sure you've reviewed the troubleshooting guide:
https://github.com/brimsec/brim/wiki/Troubleshooting

Describe the bug
Loading a *.pcap file yielded this error

To Reproduce
Try to load the *.pcap file again

Expected behavior
The summary data generated by zeek should be produced

Screenshots

Desktop Info
[Please complete the following information]

• OS: Windows 10
• Brim Version: 0.24.0

[showipintbri]

This was caused from a *.pcapng file which had Linux cooked-mode capture (SLL) layer-2 headers mixed with proper Ethernet headers. https://wiki.wireshark.org/SLL

I was able to run the *.pcapng through Tracewrangler (https://www.tracewrangler.com/), removing the 'sll' headers and replacing with false Ethernet headers so the *.pcapng can then be processed in Brim.

It works, I'm digging through it now. 👍

[philrz]

@showipintbri: Thanks for the quick follow-up with the workaround!

Now that you've figured out the root cause, I was hoping to get some example data to reproduce the problem. I know there's usually sensitive data in pcaps so I figured you couldn't share yours, so I followed the info in that Wireshark link. Per the tips found there, it looks like I was able to get something workable by taking a live capture on Wireshark after having selected the "Any" interface. That by itself still imported without complaint into Brim, but going off your tip about the SLL-header'ed traffic being "mixed" with proper traffic, I did a mergecap with a "regular" small capture, and now I've got the attached mixed.pcapng.gz that (when uncompressed) did trigger this error for me when I tried to import it:

It looks like the base pcap indexing & slice extraction don't have a problem with any of this. Examples:

$ pcap index -r mixed.pcapng -x mixed.idx

$ pcap ts -r mixed.pcapng 
1583768523.826851
1583768523.826857
1583768523.826968
1583768523.826968
1583768523.827279
1583768523.834115
1583774873.399273
1583774873.468752
1583774873.468875
1583774873.469441
1583774873.5385
1583774873.541443
1583774873.541448
1583774873.54145
1583774873.541452
1583774873.541538
1583774873.541538
1583774873.54182
1583774873.551155
1583774873.551321
1583774873.620491
1583774873.620522
1583774873.621642
1583774873.621677
1583774873.622985
1583774873.692631
1583774873.692766
1616548650.644569466
1616548650.644641131
1616548650.653302969
1616548650.653549125
1616548650.977907107
1616548650.977939753
1616548655.944718629
1616548655.944742861
1616548655.944848855
1616548655.944892435
1616548655.945103746
1616548655.94511556

$ pcap slice -x mixed.idx -r mixed.pcapng -from 1583768523.826851 -to 1583774873.692766 > /tmp/1.pcapng

$ pcap slice -x mixed.idx -r mixed.pcapng -from 1616548650.644569466 -to 1616548655.94511556 > /tmp/2.pcapng

$ pcap ts -r /tmp/1.pcapng 
1583768523.826851
1583768523.826857
1583768523.826968
1583768523.826968
1583768523.827279
1583768523.834115
1583774873.399273
1583774873.468752
1583774873.468875
1583774873.469441
1583774873.5385
1583774873.541443
1583774873.541448
1583774873.54145
1583774873.541452
1583774873.541538
1583774873.541538
1583774873.54182
1583774873.551155
1583774873.551321
1583774873.620491
1583774873.620522
1583774873.621642
1583774873.621677
1583774873.622985
1583774873.692631
1583774873.692766

$ pcap ts -r /tmp/2.pcapng 
1616548650.644569466
1616548650.644641131
1616548650.653302969
1616548650.653549125
1616548650.977907107
1616548650.977939753
1616548655.944718629
1616548655.944742861
1616548655.944848855
1616548655.944892435
1616548655.945103746
1616548655.94511556

However, both Zeek and Suricata choke on it, for sure. Running on my Mac with the artifacts bundled with Brim v0.24.0:

$ cat mixed.pcapng | /Applications/Brim.app/Contents/Resources/app/zdeps/zeek/zeekrunner 
WARNING: No Site::local_nets have been defined.  It's usually a good idea to define your local networks.
fatal error in <command line>, line 3: failed to read a packet from -: an interface has a type 113 different from the type of the first interface

$ cat mixed.pcapng | /Applications/Brim.app/Contents/Resources/app/zdeps/suricata/suricatarunner 
23/3/2021 -- 18:50:36 - <Notice> - This is Suricata version 5.0.3 RELEASE running in USER mode
23/3/2021 -- 18:50:36 - <Info> - CPUs/cores online: 12
23/3/2021 -- 18:50:36 - <Info> - No 'host-mode': suricata is in IDS mode, using default setting 'sniffer-only'
23/3/2021 -- 18:50:36 - <Info> - eve-log output device (regular) initialized: eve.json
23/3/2021 -- 18:50:37 - <Info> - 1 rule files processed. 21416 rules successfully loaded, 0 rules failed
23/3/2021 -- 18:50:37 - <Info> - Threshold config parsed: 0 rule(s) found
23/3/2021 -- 18:50:37 - <Info> - 21419 signatures processed. 1521 are IP-only rules, 3813 are inspecting packet payload, 15856 inspect application layer, 103 are decoder event only
23/3/2021 -- 18:50:42 - <Error> - [ERRCODE: SC_ERR_PCAP_OPEN_OFFLINE(26)] - failed to get first packet timestamp. pcap_next_ex(): -1
23/3/2021 -- 18:50:42 - <Warning> - [ERRCODE: SC_ERR_PCAP_DISPATCH(20)] - Failed to init pcap file -, skipping
23/3/2021 -- 18:50:42 - <Notice> - all 1 packet processing threads, 2 management threads initialized, engine started.
23/3/2021 -- 18:50:42 - <Error> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - pcap file reader thread failed to initialize
23/3/2021 -- 18:50:42 - <Notice> - Signal Received.  Stopping engine.
23/3/2021 -- 18:50:42 - <Info> - time elapsed 0.018s
23/3/2021 -- 18:50:42 - <Info> - Alerts: 0
23/3/2021 -- 18:50:42 - <Info> - cleaning up signature grouping structure... complete

We happen to be in the middle of a project to decouple the packet processing from the Brim app (brimdata/zui#1505), which should make it easier to debug, work-around, and/or fix these kinds of problems. Perhaps there's something that could be done at the base packet processing layer to detect & work around this similar to what you did with Tracewrangler, or at the very least identify the problem and present a better error message and a pointer to at workaround. Since we need to get the base project finished first I imagine we won't get to diving into this specific corner case right away, but I wanted to get the details written down for later while it's fresh in our minds.

Glad to hear you've managed to work around it in the interim! I'll hold this one open as a reminder for us to circle back to it and for others to find if they bump into this root cause.

[philrz]

Per the prior comment, I'm going to transfer this one to the Brimcap repo since in the future these kinds of pcap import problems will come up in that context. After we finish base Brimcap functionality perhaps someone can look into adding an optional "replace the headers" workaround as part of the pipeline.

[philrz]

It looks like another user has bumped into a variation of this problem as reported in a recent community Slack thread. In this case the user was able to share their test data traf2.pcap.gz which must be gunzip'ed before testing.

The attached video shows the user-facing symptom: The pcap can successfully dragged into Zui, but once we click on a connection and attempt to open its pcap slice in Wireshark, the "no packets found" error is shown.

Repro.mp4

Having looked closely at this repro, I can see conditions have changed slightly since this issue was first opened. In the past, the Zeek bundled with Brimcap/Zui was based on Zeek v3.2.1 which did not have support for cooked-mode/SLL. However, I can see support for this was added in Zeek v5.1.0, which explains why the Zeek summary logs are shown after the successful drag & drop. Meanwhile, Suricata quietly fails because they still lack support for this as tracked in their open issue #4951.

$ suricata -r ../traf2.pcap 
i: suricata: This is Suricata version 7.0.2 RELEASE running in USER mode
E: pcap: datalink type 276 not (yet) supported in module PcapFile.
W: pcap: Failed to init pcap file ../traf2.pcap, skipping
E: pcap: pcap file reader thread failed to initialize
i: threads: Threads created -> RX: 1 W: 12 FM: 1 FR: 1   Engine started.
i: suricata: Signal Received.  Stopping engine.

Back to Zui/Brimcap, in my local Zui checkout I had it print out the brimcap command line that it's using to attempt to extract the pcap slice.

$ brimcap search -dst.ip 10.14.3.50 -dst.port 49668 -duration 22.121ms -proto tcp -src.ip 10.14.4.13 -src.port 49182 -ts 2022-03-05T11:46:54.963118Z -w /var/folders/fl/2wdg4knx65jb4xw3bjygdjsw0000gn/T/packets-2022-03-05T11_46_54.963118Z.pcap -root /Users/phil/work/zui/apps/zui/run/plugins/brimcap/storage/root
{"type":"error","error":"no packets found"}

However, recreating the equivalent filter logic in Wireshark we can see it has no problem isolating the slice. Wireshark also shows the evidence that the pcap is of the "cooked" variety.

Finally, the Brimcap v1.6.0 that's bundled with current Zui releases does reflect that the link type is not currently supported.

$ ./brimcap -version
Version: v1.6.0

$ ./brimcap traf2.pcap 
Pcap type:         pcap
Pcap version:      2.4
Link type:         UnknownLinkType
Packet size limit: 262144
Number of packets: 95431

However, I do think I see a way to address this, so I will put up a Brimcap PR shortly.

[philrz]

Verified in Brimcap commit c74d4ff.

Using the test data from the last comment above, Brimcap is now able to detect the link type for this capture.

$ ./brimcap -version
Version: v1.6.0-13-gc74d4ff

$ ./brimcap info ~/Desktop/traf2.pcap 
Pcap type:         pcap
Pcap version:      2.4
Link type:         Linux SLL2
Packet size limit: 262144
Number of packets: 95431

Then with Zui 5d04fd3 that uses this Brimcap version, I can now successfully extract pcap slices from the user's "traf2.pcap" that failed in the past.

Verify.mp4

All that said, after delivering this fix to the user, we learned that there's still a remaining issue here that seems to be specific to the Zeek port on Windows, so I've opened a separate issue zeek/zeek#3609 to track that. There's also the problem cited above with how Suricata also lacks support for this link layer protocol, though that doesn't cause any failure to import the pcap into Zui, it just means no chance of seeing alerts. Ultimately it doesn't seem appropriate for us to maintain our own issues for every problem we encounter with one of the bundled analyzers, so I've opened #337 to track the idea of documenting somewhere (probably in the Brimcap wiki) what the known problems are that our users have hit with analyzers and link to the open issues for those other projects in the event that users want to pursue those.