Skip to content

Latest commit

 

History

History
51 lines (43 loc) · 1.54 KB

README.md

File metadata and controls

51 lines (43 loc) · 1.54 KB
Raw

netflow-analyzer

A complete netflow analysis setup, based on the following:

  • Orchestration - docker compose
  • Database - timescaledb
  • Netflow collection - pmacct
  • Web front end - tech not picked yet.

Setup:

Database:

  1. Copy the .env_example to .env 
    cp .env_example .env
  2. Edit .evn setting the following values:
    • Password entries ( All created using a secure password generator similar to this ).
      1. POSTGRES_PASSWORD
      2. POSTGRES_PMACCT_PW
      3. PGADMIN_DEFAULT_PASSWORD
  3. Start the database: 
    docker compose up -d db
    Ensure the database started successfully, using the following to check the logs (CTRL-C to exit the log view): 
    docker compose logs -f db
    You are looking for a line that says database system is ready to accept connections, if present then you are good to continue, otherwise the database needs to be troubleshooted.

Notes:

  • TimescaleDB:
    • IP address column type
    • Compress data after a certain point
    • Build schema and loading script
    • pgAdmin for management of backend
    • Report builder options
  • Kafka for queue:
    • How well would this work against RabbitMQ?
    • It still writes data to disk... Optane?
    • Protobuf for faster parsing
    • Possibly remove this layer entirely if use pmacct instead of goflow2
    • Trigger script for enriching with external data sources
    • Can use this to deal with creating aggregates
  • Mikrotik:
    • v9 / IPFIX does not work with goflow2