living target for ECS

[neu5ron]

Logstash

• Document or note new ECS fields
  • tls.
  • threat.
  • rule.
  • alert.
  • vulnerability.
  • event.
  • error.
  • log.
  • dns.
  • domain.
  • package.
    • pending confirmation based on whether only should be "installed" software, or can include just metadata of extracted/seen/passive software information.. -- no
    • open github issue in ECS with notes NSM/sensor info on binaries/software crossing the wire NSM/sensor certain http header fields (ie: x-flash-version), rdp info/software stuff, ssh info/software stuff, etc some windows event logs i can think of, windows update log with info on the updates available OR when packages are removed/uninstalled ASA/VPN client software info from the router/firewall itself
  • client.
  • server.
  • destination.
  • source.
  • file.
    • file.system is os.something now
  • follow documentation for custom fields
• Document or note new ECS fields for Zeek
  • TLS
    • ssl.log
    • no other logs that seem to apply, maybe a field or two in rdp.log or smtp.log but they i would say no...
  • log/syslog/event > syslog.log
  • notice.log to rule.name
  • dns.
    • question.subdomain
    • question.top_level_domain
  • domain.
    • subdomain
    • top_level_domain
  • client.
    • subdomain
    • top_level_domain
  • server.
    • subdomain
    • top_level_domain
  • destination.
    • subdomain
    • top_level_domain
  • source.
    • subdomain
    • top_level_domain
  • related.hash
• Document or note new Suricata
  • TLS
    • tls log
      • generate logs with extended/custom
  • alert log to rule schema
• Update logstash config for Zeek
  • TLS
• Update logstash config for Suricata
• copy src/dst for suricata to client/server AND OR copy client/server for zeek to src/dst?
• dns.question.registered_domain is required field now for SIEM DNS tab

Kibana

• Update migration dashboards for Zeek
• Update migration dashboards for Suricata
• Tweak docket pivot
• Check any other dashboards
• Test / Save everything for 7.5.1 objects
• test docket pivot zeek
• test docket pivot suricata

Etc

• ntp elasticsearch mappings for zeek
• syslog ECS schema elasticsearch mappings

TAP DEVICE SETUP to support (major) jumbo MTU shenanigans

Create virtual interface

This doesn't appear to work, would drop my interfaces.
sudo ip tuntap add dev foo0 mode tap

This did work on my box though with out dropping my interface on a physical machine. Still drops on a VM.
sudo ip link add dev foo0 type dummy

set interface to largest mtu

sudo ip link set dev foo0 mtu 65521

Bring the interface up

sudo ip link set foo0 up

Configure zeek to ignore checksums

cat <<EOF | sudo tee -a /usr/share/zeek/site/local.zeek
redef ignore_checksums = T;
EOF

Configure Suricata to deal with this shenanigans

sudo vi /etc/suricata/rocknsm-overrides.yaml

...
af-packet:
  - interface: foo
    #threads: auto
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    mmap-locked: yes
    #rollover: yes
    block-size: 131072
    tpacket-v3: yes
    use-emergency-flush: yes
...

Redeploy ROCK

sudo /usr/sbin/deploy_rock.sh

Restart zeek, suricata, and stenographer

sudo systemctl enable stenographer@foo0 stenographer
sudo systemctl restart zeek suricata stenographer@foo0

Run tcpreplay (in another shell)

sudo tcpreplay -i -M 10 foobar pcaps/day1.pcap

If you need to time travel to separate out multiple pcap's into different days

Stop all the services

systemctl stop elasticsearch kafka zookeeper suricata stenographer zeek logstash

Copy any steno files you want to snapshot

cp -R /data/stenographer/foobar/thread0/ /home/admin/steno_snapshots/day1/.

Clean up the steno directory for next run

rm -rf /data/stenographer/foobar/thread0/*

Time travel to desired time

date -s "2 SEP 2018 11:00:00"

Start up the services

systemctl start elasticsearch zookeeper kafka suricata stenographer zeek logstash

Wait till the services are settled and running

Replay your next set of pcap's through the dummy interface.

tcpreplay -i foobar /home/admin/pcaps/day2.pcap

[ipninichuck]

I have a question about the alert( ex: alert.action) category that is being used. Should we be swapping this for the rule category? It would seem that with a little expansion the rule category could encompass all info relating to alerts triggered. Though it is interesting to note that the filebeat-* index has not done so with their Zeek or Suricata modules yet, though I suspect it might be coming.

[peasead]

I have a question about the alert( ex: alert.action) category that is being used. Should we be swapping this for the rule category? It would seem that with a little expansion the rule category could encompass all info relating to alerts triggered. Though it is interesting to note that the filebeat-* index has not done so with their Zeek or Suricata modules yet, though I suspect it might be coming.

I'll take a stab at this. I apologize if I'm missing your question.

filebeat-* does have suricata.eve.alert.category and suricata.eve.alert.action as Exported Fields (Note). I'm not sure it would make sense combined or expanded as they're providing different information, as you can see below.

suricata.eve.alert.action would be (Note):
• Pass
• Drop
• Reject
• Alert

suricata.eve.alert.category would be:
• PHISHING
• COINMINER
• JA3
• EXPLOIT_KIT
• HUNTING
• ADWARE_PUP
• MALWARE
• ...

I'm unfamiliar with rule.* in the Suricata module, though I'd assume it's suricata.eve.alert.*.

I'm not quite sure alert would make sense in a Zeek module though, outside of maybe the Intel Framework, but I'm not sure that is really an alert in the traditional sense. (Note).

Please correct me if I missed what you were asking.

[neu5ron]

yup, alert will be included. thanks for bringing it up! Im not sure the exact format of the fields yet - but im adding to the list of applicable fields

[ipninichuck]

I also noticed those fields in the filebeat modules. That is why I was confused when the ECS 1.4 docs had no reference to the categories. When I brought it up on the Elastic Slack someone brought up the new rules category. It will be interesting to see how it plays out. For now I'm highlighting the fields in my spreadsheet as possibly changing. Than you for the further information.

…

On Thu, Dec 26, 2019, 9:05 AM Andrew Pease ***@***.***> wrote: I have a question about the alert( ex: alert.action) category that is being used. Should we be swapping this for the rule category? It would seem that with a little expansion the rule category could encompass all info relating to alerts triggered. Though it is interesting to note that the filebeat-* index has not done so with their Zeek or Suricata modules yet, though I suspect it might be coming. I'll take a stab at this. I apologize if I'm missing your question. filebeat-* does have suricata.eve.alert.category and suricata.eve.alert.action as Exported Fields (Note <https://www.elastic.co/guide/en/beats/filebeat/master/exported-fields-suricata.html>). I'm not sure it would make sense combined or expanded as they're providing different information, as you can see below. suricata.eve.alert.action would be (Note <https://suricata.readthedocs.io/en/latest/configuration/suricata-yaml.html#suricata-yaml-action-order> ): • Pass • Drop • Reject • Alert suricata.eve.alert.category would be: • PHISHING • COINMINER • JA3 • EXPLOIT_KIT • HUNTING • ADWARE_PUP • MALWARE • ... I'm unfamiliar with rule.* in the Suricata module, though I'd assume it's suricata.eve.alert.*. I'm not quite sure alert would make sense in a Zeek module though, outside of *maybe* the Intel Framework <https://www.elastic.co/guide/en/beats/filebeat/master/exported-fields-zeek.html#_intel>, but I'm not sure that is really an alert in the traditional sense. (Note <https://www.elastic.co/guide/en/beats/filebeat/master/exported-fields-zeek.html> ). Please correct me if I missed what you were asking. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#46?email_source=notifications&email_token=AGI5DZ3EMEEUPWWD4ZRN733Q2TP5TA5CNFSM4J57NRH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHV2TYA#issuecomment-569092576>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGI5DZ6ZMLULQ6P6Q7BPIELQ2TP5TANCNFSM4J57NRHQ> .