Skip to content

Commit

Permalink
[filebeat][decode_cef] Allow hyphens in extension key (elastic#40427)
Browse files Browse the repository at this point in the history
This adds support for hyphens (`-`) in extension keys. The CEF spec says that extension keys alphanumeric. So this is a deviation, but a minor one that is inline with past deviations to allow dots in extension keys. 

I have also added .ri file to gitignore file as they are intermediate files generated by regel.

Closes elastic#40348
  • Loading branch information
vinit-chauhan authored Aug 12, 2024
1 parent d7ae68c commit e4012a2
Show file tree
Hide file tree
Showing 10 changed files with 450 additions and 208 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,12 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Implement Elastic Agent status and health reporting for Winlog Filebeat input. {pull}40163[40163]
- Fix filestream's registry GC: registry entries will never be removed if clean_inactive is set to "-1". {pull}40258[40258]
- Added `ignore_empty_values` flag in `decode_cef` Filebeat processor. {pull}40268[40268]
- Added support for hyphens in extension keys in `decode_cef` Filebeat processor. {pull}40427[40427]
- Journald: removed configuration options `include_matches.or`, `include_matches.and`, `backoff`, `max_backoff`, `cursor_seek_fallback`. {pull}40061[40061]
- Journald: `include_matches.match` now behaves in the same way as matchers in `journalctl`. Users should carefully update their input configuration. {pull}40061[40061]
- Journald: `seek` and `since` behaviour have been simplified, if there is a cursor (state) `seek` and `since` are ignored and the cursor is used. {pull}40061[40061]


*Heartbeat*


Expand Down
1 change: 1 addition & 0 deletions x-pack/filebeat/processors/decode_cef/cef/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
*.svg
*.dot
*.ri
2 changes: 1 addition & 1 deletion x-pack/filebeat/processors/decode_cef/cef/cef.rl
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
# Only alnum is defined in the CEF spec. The other characters allow
# non-conforming extension keys to be parsed.
extension_key_start_chars = alnum | '_';
extension_key_chars = extension_key_start_chars | '.' | ',' | '[' | ']';
extension_key_chars = extension_key_start_chars | '.' | ',' | '[' | ']' | '-';
extension_key_pattern = extension_key_start_chars extension_key_chars*;
extension_value_chars_nospace = extension_value_escapes | (any -- equal -- escape -- space);

Expand Down
21 changes: 21 additions & 0 deletions x-pack/filebeat/processors/decode_cef/cef/cef_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ const (

noValueInExtension = `CEF:26|security|threat=manager|1.0|100|trojan successfully stopped|10|src= dst=12.121.122.82 spt=`

hyphenInExtensionKey = `CEF:26|security|threatmanager|1.0|100|trojan successfully stopped|10|Some-Key=123456`

// Found by fuzzing but minimised by hand.
fuzz0 = `CEF:0|a=\\ b|`
fuzz1 = `CEF:0|\|a=|b=`
Expand Down Expand Up @@ -87,6 +89,7 @@ var testMessages = []string{
escapedMessage,
truncatedHeader,
noValueInExtension,
hyphenInExtensionKey,
fuzz0,
fuzz1,
fuzz2,
Expand Down Expand Up @@ -180,6 +183,24 @@ func TestEventUnpack(t *testing.T) {
}, e.Extensions)
})

t.Run("hyphenInExtensionKey", func(t *testing.T) {
var e Event
err := e.Unpack(hyphenInExtensionKey)
assert.NoError(t, err)
assert.Equal(t, 26, e.Version)
assert.Equal(t, "security", e.DeviceVendor)
assert.Equal(t, "threatmanager", e.DeviceProduct)
assert.Equal(t, "1.0", e.DeviceVersion)
assert.Equal(t, "100", e.DeviceEventClassID)
assert.Equal(t, "trojan successfully stopped", e.Name)
assert.Equal(t, "10", e.Severity)
assert.Equal(t, map[string]*Field{
"Some-Key": {
String: "123456",
},
}, e.Extensions)
})

t.Run("equalsSignInHeader", func(t *testing.T) {
var e Event
err := e.Unpack(equalsSignInHeader)
Expand Down
64 changes: 34 additions & 30 deletions x-pack/filebeat/processors/decode_cef/cef/parser.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit e4012a2

Please sign in to comment.