[Filebeat : v7.9.3] Invalid memory address or nil pointer dereference

[iamyeka]

Filebeat version: 7.9.3

I met some issue about "invalid memory address or nil pointer dereference" which caused a panic. But i cannot figure out why that happened. Below is the stack of the panic.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x402b74]

goroutine 182 [running]:
github.com/elastic/beats/v7/libbeat/monitoring/report/log.makeDeltaSnapshot(0xc04faaee10, 0xc04faaee40, 0xc04faaee70, 0xc04faaeea0, 0xc04faaeed0, 0xc00702b950, 0xc00702b980, 0xc00702b9b0, 0xc00702b9e0, 0xc00702ba10, ...)
	/go/src/netease.com/libbeat/monitoring/report/log/log.go:174 +0x452
github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).snapshotLoop(0xc000a9c660)
	/go/src/netease.com/libbeat/monitoring/report/log/log.go:136 +0x2b6
github.com/elastic/beats/v7/libbeat/monitoring/report/log.MakeReporter.func1(0xc000a9c660)
	/go/src/netease.com/libbeat/monitoring/report/log/log.go:107 +0x50
created by github.com/elastic/beats/v7/libbeat/monitoring/report/log.MakeReporter
	/go/src/netease.com/libbeat/monitoring/report/log/log.go:105 +0x15c

The key code is here:

From the code i don't under what circumstance the panic could happen. Could anyone help find out why the panic happen?

[lowryxiao]

My 2 cents, if you could provide the steps to reproduce the issue that would be better to dig further, like the configurations and the log contents that made heartbeat panic.

[iamyeka]

My 2 cents, if you could provide the steps to reproduce the issue that would be better to dig further, like the configurations and the log contents that made heartbeat panic.

The point is it happens occasionally. Not very often, maybe once a few weeks.

Here is the configuration:

filebeat.config:
  inputs:
    clean_inactive: 36h
    close_inactive: 5m
    ignore_older: 72h
    path: ${path.config}/inputs.d/*.yml
    reload:
      enabled: true
      period: 5s
http.enabled: true
http.host: 0.0.0.0
http.port: 5066
logging.files:
  interval: 24h
  keepfiles: 7
  name: filebeat
  path: /var/log/filebeat
  permissions: 420
  rotateeverybytes: 104857600
  rotateonstartup: false
logging.level: info
logging.to_files: true
output.kafka:
  bulk_max_size: 81920
  channel_buffer_size: 81920
  compression: snappy
  hosts: [...] // hidden because of secrity issue
  partition.round_robin:
    group_events: 10000
    reachable_only: false
  topic: '%{[fields.topic]}'
  version: 0.10.2.1
  worker: 3
processors:
- cloudmusic: null
queue.mem:
  events: 1638400
  flush.min_events: 81920
  flush.timeout: 1s

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!