You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now brimcap load seems dependent on the log outputs of analyzers being auto-detect-able. This works fine for the main cases of Zeek and Suricata since their default outputs are Zeek TSV and NDJSON, respectively. However, while working on #72 I happened to try working with a NetFlow analyzer's CSV output. Using these configs with Brimcap v0.0.3, we can see how it fails because we don't currently auto-detect CSV.
$ cat nfdump-wrapper-csv.sh
#!/bin/bash
TMPFILE=$(mktemp)
cat - > "$TMPFILE"
nfpcapd -r "$TMPFILE" -l .
rm "$TMPFILE"
for file in nfcapd.*
do
nfdump -r $file -o csv | ghead -n -3 > ${file}.csv
done
$ cat nfdump-csv.yml
analyzers:
- cmd: nfdump-wrapper-csv.sh
globs: ["*.csv"]
$ brimcap load -root "$HOME/Library/Application Support/Brim/data/brimcap-root" -config nfdump-csv.yml -p testpool ~/pcap/wrccdc.pcap
100.0% 500.0MB/500.0MB records=0
Post "http://localhost:9867/pool/1sArPVhV4gBbiH5B1E8NytXJ34G/log": format detection error
tzng: line 1: bad format
zeek: line 1: bad types/fields definition in zeek header
zjson: line 1: invalid character 's' in literal true (expecting 'r')
zson: identifier "ts" must be enum and requires decorator
zng: zng type ID out of range
parquet: auto-detection not supported
zst: auto-detection not supported
If we wanted to support this, I can think of two ways to proceed.
Allow the specification of input format in brimcap load (similar to the zq -i options)
Note to self: For now I've added a comment in the "Custom Brimcap Configuration" article linking to this as an open issue, so that way the reader understands the current limitation is temporary. If/when we address this issue, I should update the wiki article to remove the workaround and the comment.
Right now
brimcap load
seems dependent on the log outputs of analyzers being auto-detect-able. This works fine for the main cases of Zeek and Suricata since their default outputs are Zeek TSV and NDJSON, respectively. However, while working on #72 I happened to try working with a NetFlow analyzer's CSV output. Using these configs with Brimcapv0.0.3
, we can see how it fails because we don't currently auto-detect CSV.If we wanted to support this, I can think of two ways to proceed.
brimcap load
(similar to thezq -i
options)The text was updated successfully, but these errors were encountered: