Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Documentation available? #40

Open
Henry3713 opened this issue Jan 16, 2023 · 1 comment
Open

Documentation available? #40

Henry3713 opened this issue Jan 16, 2023 · 1 comment

Comments

@Henry3713
Copy link

Henry3713 commented Jan 16, 2023 •
edited
Loading

Hi there,

Thanks for the provided Project.

I was wondering if there is any documentation out there where I can find more details about installation and the settings related stuff.

I tried the installation method provided via readme but it seems not working in my case. I like to troubleshoot but missing the knowledge how to check those things.

Using IPFIX flows.

/tmp/nfcollector-dump.log output looks like you see down below but I can't see if it gets to the database. Grafana shows N.A. :-)

  data set
    2 records:
      record 0:
        sourceIPv4Address: 8x.x.x.x
        destinationIPv4Address: 1x.x.x.x
        ipClassOfService: 0
        protocolIdentifier: 6
        sourceTransportPort: 47508
        destinationTransportPort: 22
        icmpTypeCodeIPv4: 0
        ingressInterface: 830
        vlanId: 266
        sourceIPv4PrefixLength: 31
        destinationIPv4PrefixLength: 16
        bgpSourceAsNumber: 65xxx
        bgpDestinationAsNumber: 24940
        ipNextHopIPv4Address: 8x.x.x.x
        tcpControlBits: 24
        egressInterface: 832
        octetDeltaCount: 772
        packetDeltaCount: 7
        minimumTTL: 61
        maximumTTL: 61
        flowStartMilliseconds: 2023-01-12 17:34:52.233 +0100 CET
        flowEndMilliseconds: 2023-01-12 17:37:17.309 +0100 CET
        flowEndReason: 1
        dot1qVlanId: 0
        dot1qCustomerVlanId: 0
      record 1:
        sourceIPv4Address: 2x.x.x.x
        destinationIPv4Address: 8x.x.x.x
        ipClassOfService: 0
        protocolIdentifier: 6
        sourceTransportPort: 443
        destinationTransportPort: 60971
        icmpTypeCodeIPv4: 0
        ingressInterface: 832
        vlanId: 0
        sourceIPv4PrefixLength: 10
        destinationIPv4PrefixLength: 29
        bgpSourceAsNumber: 8075
        bgpDestinationAsNumber: 65xxx
        ipNextHopIPv4Address: 8x.x.x.x
        tcpControlBits: 24
        egressInterface: 830
        octetDeltaCount: 1342
        packetDeltaCount: 1
        minimumTTL: 115
        maximumTTL: 115
        flowStartMilliseconds: 2023-01-12 17:37:16.737 +0100 CET
        flowEndMilliseconds: 2023-01-12 17:37:16.737 +0100 CET
        flowEndReason: 1
        dot1qVlanId: 0
        dot1qCustomerVlanId: 0
IPFIX message
@Henry3713
Copy link
Author 

root@xxx:~/oi24/nfcollector# docker ps
CONTAINER ID   IMAGE                        COMMAND                  CREATED          STATUS          PORTS                                                                                                                             NAMES
c6c5ae07b7a3   javadmohebbi/gonfcollector   "/opt/nfcollector/bi…"   22 minutes ago   Up 22 minutes   0.0.0.0:6859->6859/udp, :::6859->6859/udp                                                                                         oi24-nfcolector-1
52665d2e4dbe   influxdb:2.0.7               "/entrypoint.sh infl…"   22 minutes ago   Up 22 minutes   0.0.0.0:8082->8082/tcp, :::8082->8082/tcp, 0.0.0.0:8086->8086/tcp, :::8086->8086/tcp, 0.0.0.0:8089->8089/tcp, :::8089->8089/tcp   oi24-influxdb-1
42ea1b0074c4   grafana/grafana:7.5.0        "/run.sh"                22 minutes ago   Up 22 minutes   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                                                                                         oi24-grafana-1

docker logs c6c5ae07b7a3
2023/01/16 11:11:18 Reading config from "/opt/oi24/netflow-collector/etc/collector.yml"
2023/01/16 11:11:18 can not read config from:  /opt/oi24/netflow-collector/etc/collector.yml
2023/01/16 11:11:18 Minimal configuration will be read using OS environment
2023/01/16 11:11:18 Reading config from "/opt/oi24/netflow-collector/etc/trans.yml"
2023/01/16 11:11:18 can not read config from:  /opt/oi24/netflow-collector/etc/trans.yml
2023/01/16 11:11:18 Minimal configuration will be read using OS environment
set max CPU num to: 2
2023/01/16 11:11:18 Reading config from "/opt/oi24/netflow-collector/etc/ip2location.yml"
Can not open log file: /opt/nfcollector/var/log/collector.log. Logs will be displayed ONLY on standard output (stdout)
        open /opt/nfcollector/var/log/collector.log: no such file or directory
--- DEBUGGING IS ENABLED ---
2023/01/16 11:11:18 can not read config from:  /opt/oi24/netflow-collector/etc/ip2location.yml
2023/01/16 11:11:18 Minimal configuration will be read using OS environment
ERROR[2023-01-16T11:11:20Z] File: type.go Function: github.com/goNfCollector/location.New Line: 101 [14]-Can not open Local CSV DB: (open /opt/nfcollector/vendors/ip2location/local-db/local.csv: no such file or directory)  debug="File: type.go Function: github.com/goNfCollector/location.New Line: 101"
ERROR[2023-01-16T11:11:20Z] File: type.go Function: github.com/goNfCollector/location.New Line: 112 [15]-Can not read Local CSV DB: (invalid argument)  debug="File: type.go Function: github.com/goNfCollector/location.New Line: 112"
2023/01/16 11:11:20 Reading config from "/opt/oi24/netflow-collector/etc/socket.yml"
2023/01/16 11:11:20 can not read config from:  /opt/oi24/netflow-collector/etc/socket.yml
2023/01/16 11:11:20 Minimal configuration will be read using OS environment
2023/01/16 11:11:20 Reading config from "/opt/oi24/netflow-collector/etc/socket.yml"
2023/01/16 11:11:20 can not read config from:  /opt/oi24/netflow-collector/etc/socket.yml
2023/01/16 11:11:20 Minimal configuration will be read using OS environment
2023/01/16 11:11:20 new client connected:  /opt/nfcollector/var/socket/fw.socket
DEBUG[2023-01-16T11:11:20Z] File: type.go Function: github.com/goNfCollector/influxdb.New Line: 80 new influxDB exporter influxdb:8086 bucket:nfCollector org:OPENINTELLIGENCE is created  debug="File: type.go Function: github.com/goNfCollector/influxdb.New Line: 80"
DEBUG[2023-01-16T11:11:20Z] File: collector.go Function: github.com/goNfCollector/collector.(*Collector).listen Line: 236 listening on 0.0.0.0:6859  debug="File: collector.go Function: github.com/goNfCollector/collector.(*Collector).listen Line: 236"
INFO[2023-01-16T11:11:20Z] Server is now listening on 0.0.0.0:6859 (UDP)...!  debug="File: collector.go Function: github.com/goNfCollector/collector.(*Collector).listen Line: 254"

docker logs 52665d2e4dbe
2023-01-16T11:11:18.351119112Z  info    found existing boltdb file, skipping setup wrapper      {"system": "docker", "bolt_path": "/var/lib/influxdb2/influxd.bolt"}
ts=2023-01-16T11:11:25.194916Z lvl=info msg="Welcome to InfluxDB" log_id=0fQ_H42W000 version=2.0.7 commit=2a45f0c037 build_date=2021-06-04T19:17:40Z
ts=2023-01-16T11:11:25.196043Z lvl=info msg="Resources opened" log_id=0fQ_H42W000 service=bolt path=/var/lib/influxdb2/influxd.bolt
ts=2023-01-16T11:11:25.202155Z lvl=info msg="Checking InfluxDB metadata for prior version." log_id=0fQ_H42W000 bolt_path=/var/lib/influxdb2/influxd.bolt
ts=2023-01-16T11:11:25.202241Z lvl=info msg="Using data dir" log_id=0fQ_H42W000 service=storage-engine service=store path=/var/lib/influxdb2/engine/data
ts=2023-01-16T11:11:25.202261Z lvl=info msg="Compaction settings" log_id=0fQ_H42W000 service=storage-engine service=store max_concurrent_compactions=4 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
ts=2023-01-16T11:11:25.202272Z lvl=info msg="Open store (start)" log_id=0fQ_H42W000 service=storage-engine service=store op_name=tsdb_open op_event=start
ts=2023-01-16T11:11:25.202301Z lvl=info msg="Open store (end)" log_id=0fQ_H42W000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=0.029ms
ts=2023-01-16T11:11:25.202324Z lvl=info msg="Starting retention policy enforcement service" log_id=0fQ_H42W000 service=retention check_interval=30m
ts=2023-01-16T11:11:25.202332Z lvl=info msg="Starting precreation service" log_id=0fQ_H42W000 service=shard-precreation check_interval=10m advance_period=30m
ts=2023-01-16T11:11:25.202365Z lvl=info msg="Starting query controller" log_id=0fQ_H42W000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
ts=2023-01-16T11:11:25.205602Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0fQ_H42W000 max_select_point=0 max_select_series=0 max_select_buckets=0
ts=2023-01-16T11:11:25.495945Z lvl=info msg=Listening log_id=0fQ_H42W000 service=tcp-listener transport=http addr=:8086 port=8086
ts=2023-01-16T11:11:25.495968Z lvl=info msg=Starting log_id=0fQ_H42W000 service=telemetry interval=8h

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Henry3713