Support the timestamp processor in packetbeat

[nkakouros]

Describe the enhancement:
The timestamp processor is missing from packetbeat (and auditbeat/metricbeat).

Describe a specific use case for the enhancement or feature:
In the logged events, I have an event.start and event.end time that both are less that the @timestamp field. I would like to make the event.start field the timestamp of the events.

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[nkakouros]

This is needed as it makes things complicated for someone who wants to replace the timestamp of the event.

As it stands now, one will need to create to e.g. an elasticsearch ingest node pipeline for that purpose. But if they want to have more than one ingest node pipelines, they either have to repeat the time-change processors in each and every pipeline or use the pipeline processor to jump to the time-change pipeline. This can quickly become complex.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nkakouros]

.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[adriansr]

The point of the timestamp processor is to set @timestamp from a string in other fields, potentially in different datetime formats.

If you just want to set @timestamp from a field that is already a valid date for Elasticsearch, such as event.start or event.end, you can do so already with the convert processor:

processors:
 - convert:
      fields:
         from: event.start
         to: "@timestamp"

This is a bit counterintuitive, as there's also a copy_fields processor, but that one doesn't allow to set metadata fields like @timestamp.

[adriansr]

@jamiehynds @andrewkroh given that we have a workaround, do you still think it's worth it to have the processor in all Beats?

[andrewkroh]

@adriansr Is using convert to copy the value a valid workaround? I didn't test it but I thought this was problematic because the value is not a time.Time or common.Time and you would get an error.

beats/libbeat/beat/event.go

Lines 69 to 77 in 2876cfb

	func (e *Event) PutValue(key string, v interface{}) (interface{}, error) {
	if key == "@timestamp" {
	switch ts := v.(type) {
	case time.Time:
	e.Timestamp = ts
	case common.Time:
	e.Timestamp = time.Time(ts)
	default:
	return nil, errNoTimestamp

[adriansr]

It worked for me when setting it from event.start in Packetbeat, but I wasn't aware of that limitation, that will cause it to break in some cases.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[elasticmachine]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)