Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Aruba Documentation #11101

Closed
wants to merge 0 commits into from

Conversation

qcorporation
Copy link

Change Log

  • added manifest file with udp and tcp support
  • named ownership for aruba to the dnd team
  • documented all fields from the official aruba documentation and mapped them to either ecs or fields
  • put placeholders for stream.yml, base-fields and docker-compose
  • added example logs from CX 6300, 6000 and 8360

Checklist

  • [ x ] I have reviewed tips for building integrations and this pull request is aligned with them.
  • [ no ] I have verified that all data streams collect metrics or logs.
  • [ x ] I have added an entry to my package's changelog.yml file.
  • [ x ] I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • Reviewed that all ecs or newly created maps make sense in the absence of actual logs, just from the documentation
  • Review al fields data types, that they make sense
  • Validate that nothing was missed for message type for the Aruba CX lines between version v5200 -> v8214

How to test this PR locally

  • review the mappings. This will be a guide for additional contributions as we separate the work

@qcorporation qcorporation self-assigned this Sep 12, 2024
@qcorporation qcorporation added the Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices] label Sep 12, 2024
@elasticmachine
Copy link

elasticmachine commented Sep 13, 2024

🚀 Benchmarks report

Package 1password 👍(0) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
signin_attempts 6134.97 4219.41 -1915.56 (-31.22%) 💔

Package abnormal_security 👍(1) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 5154.64 4201.68 -952.96 (-18.49%) 💔
threat 2785.52 1492.54 -1292.98 (-46.42%) 💔

Package activemq 👍(3) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 8474.58 5714.29 -2760.29 (-32.57%) 💔
topic 111111.11 76923.08 -34188.03 (-30.77%) 💔

Package apache_tomcat 👍(2) 💚(0) 💔(7)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access 2531.65 1689.19 -842.46 (-33.28%) 💔
catalina 13333.33 10638.3 -2695.03 (-20.21%) 💔
localhost 25641.03 17857.14 -7783.89 (-30.36%) 💔
memory 33333.33 16393.44 -16939.89 (-50.82%) 💔
request 40000 25000 -15000 (-37.5%) 💔
session 24390.24 19607.84 -4782.4 (-19.61%) 💔
thread_pool 8403.36 6993.01 -1410.35 (-16.78%) 💔

Package auth0 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
logs 6578.95 3816.79 -2762.16 (-41.98%) 💔

Package authentik 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
group 4504.5 3333.33 -1171.17 (-26%) 💔

Package aws 👍(10) 💚(6) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
route53_public_logs 10000 8264.46 -1735.54 (-17.36%) 💔
vpcflow 8000 5747.13 -2252.87 (-28.16%) 💔
cloudfront_logs 2415.46 1416.43 -999.03 (-41.36%) 💔

Package azure 👍(6) 💚(2) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
activitylogs 1607.72 1107.42 -500.3 (-31.12%) 💔
identity_protection 4739.34 3731.34 -1008 (-21.27%) 💔
platformlogs 5434.78 4385.96 -1048.82 (-19.3%) 💔

Package azure_frontdoor 👍(0) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
waf 4149.38 3194.89 -954.49 (-23%) 💔

Package barracuda_cloudgen_firewall 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 11627.91 9615.38 -2012.53 (-17.31%) 💔

Package bitdefender 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
push_statistics 62500 38461.54 -24038.46 (-38.46%) 💔

Package bitwarden 👍(3) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
policy 8196.72 6535.95 -1660.77 (-20.26%) 💔

Package box_events 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
events 5405.41 3300.33 -2105.08 (-38.94%) 💔

Package carbon_black_cloud 👍(3) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
watchlist_hit 3378.38 2801.12 -577.26 (-17.09%) 💔

Package ceph 👍(1) 💚(3) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
cluster_health 30303.03 21739.13 -8563.9 (-28.26%) 💔
cluster_status 7462.69 5319.15 -2143.54 (-28.72%) 💔
osd_tree 25641.03 17241.38 -8399.65 (-32.76%) 💔

Package cisco_duo 👍(0) 💚(3) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
auth 2057.61 1485.88 -571.73 (-27.79%) 💔
offline_enrollment 32258.06 6329.11 -25928.95 (-80.38%) 💔

Package cisco_ftd 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 841.75 644.75 -197 (-23.4%) 💔

Package cisco_meraki 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
events 500000 333333.33 -166666.67 (-33.33%) 💔

Package citrix_adc 👍(4) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
system 6410.26 4830.92 -1579.34 (-24.64%) 💔

Package claroty_ctd 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
event 1394.7 1162.79 -231.91 (-16.63%) 💔

Package cloudflare_logpush 👍(6) 💚(5) 💔(7)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
gateway_network 7194.24 5376.34 -1817.9 (-25.27%) 💔
sinkhole_http 6134.97 3424.66 -2710.31 (-44.18%) 💔
spectrum_event 3984.06 3003 -981.06 (-24.62%) 💔
workers_trace 9090.91 3436.43 -5654.48 (-62.2%) 💔
dns_firewall 5747.13 4347.83 -1399.3 (-24.35%) 💔
firewall_event 3105.59 2314.81 -790.78 (-25.46%) 💔
gateway_dns 4545.45 3846.15 -699.3 (-15.38%) 💔

Package couchbase 👍(1) 💚(6) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
bucket 15873.02 10526.32 -5346.7 (-33.68%) 💔
database_stats 32258.06 27027.03 -5231.03 (-16.22%) 💔
query_index 9803.92 7936.51 -1867.41 (-19.05%) 💔

Package couchdb 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
server 4405.29 3484.32 -920.97 (-20.91%) 💔

Package crowdstrike 👍(1) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alert 1071.81 813.01 -258.8 (-24.15%) 💔
host 1818.18 1356.85 -461.33 (-25.37%) 💔

Package cybereason 👍(4) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
poll_malop 1926.78 1557.63 -369.15 (-19.16%) 💔

Package darktrace 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
system_status_alert 4854.37 4000 -854.37 (-17.6%) 💔

Package eset_protect 👍(1) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
detection 2409.64 1782.53 -627.11 (-26.03%) 💔
event 2659.57 1845.02 -814.55 (-30.63%) 💔

Package f5 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
bigipafm 28571.43 19607.84 -8963.59 (-31.37%) 💔

Package forcepoint_web 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
logs 2518.89 1953.13 -565.76 (-22.46%) 💔

Package forgerock 👍(5) 💚(2) 💔(4)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
am_access 11235.96 8264.46 -2971.5 (-26.45%) 💔
am_activity 26315.79 20000 -6315.79 (-24%) 💔
am_authentication 17241.38 11111.11 -6130.27 (-35.56%) 💔
am_config 30303.03 24390.24 -5912.79 (-19.51%) 💔

Package gcp 👍(4) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
firewall 4032.26 3184.71 -847.55 (-21.02%) 💔

Package google_scc 👍(2) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
asset 1490.31 1114.83 -375.48 (-25.19%) 💔
source 35714.29 21739.13 -13975.16 (-39.13%) 💔

Package google_workspace 👍(8) 💚(2) 💔(4)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
user_accounts 13513.51 10309.28 -3204.23 (-23.71%) 💔
context_aware_access 4291.85 3533.57 -758.28 (-17.67%) 💔
gcp 7092.2 5917.16 -1175.04 (-16.57%) 💔
groups 7194.24 5000 -2194.24 (-30.5%) 💔

Package hadoop 👍(2) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
cluster 5882.35 4255.32 -1627.03 (-27.66%) 💔
namenode 11235.96 7462.69 -3773.27 (-33.58%) 💔

Package ibmmq 👍(0) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
qmgr 3717.47 2257.34 -1460.13 (-39.28%) 💔

Package jamf_pro 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
inventory 6289.31 5050.51 -1238.8 (-19.7%) 💔

Package jamf_protect 👍(2) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
telemetry_legacy 1984.13 1510.57 -473.56 (-23.87%) 💔
web_threat_events 8547.01 5494.51 -3052.5 (-35.71%) 💔

Package kubernetes 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
container_logs 200000 125000 -75000 (-37.5%) 💔

Package lastpass 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
detailed_shared_folder 10752.69 6410.26 -4342.43 (-40.38%) 💔

Package m365_defender 👍(3) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alert 857.63 594.88 -262.75 (-30.64%) 💔

Package mattermost 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 2985.07 2293.58 -691.49 (-23.16%) 💔

Package microsoft_dnsserver 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 15873.02 6849.32 -9023.7 (-56.85%) 💔

Package microsoft_exchange_server 👍(2) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
smtp 62500 41666.67 -20833.33 (-33.33%) 💔

Package mimecast 👍(4) 💚(5) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
archive_search_logs 10309.28 6250 -4059.28 (-39.38%) 💔

Package modsecurity 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
auditlog 553.4 395.26 -158.14 (-28.58%) 💔

Package mongodb_atlas 👍(6) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
project 4651.16 2109.7 -2541.46 (-54.64%) 💔

Package mysql 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
slowlog 25641.03 19607.84 -6033.19 (-23.53%) 💔

Package nagios_xi 👍(0) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
events 16393.44 13888.89 -2504.55 (-15.28%) 💔
service 3246.75 1937.98 -1308.77 (-40.31%) 💔

Package netskope 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alerts 1506.02 1180.64 -325.38 (-21.61%) 💔
events 2358.49 1540.83 -817.66 (-34.67%) 💔

Package nginx_ingress_controller 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
error 38461.54 32258.06 -6203.48 (-16.13%) 💔

Package pps 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 4975.12 3703.7 -1271.42 (-25.56%) 💔

Package proofpoint_on_demand 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 1872.66 1385.04 -487.62 (-26.04%) 💔

Package pulse_connect_secure 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 3759.4 1265.82 -2493.58 (-66.33%) 💔

Package rabbitmq 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 9090.91 4761.9 -4329.01 (-47.62%) 💔

Package redis 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 8620.69 5319.15 -3301.54 (-38.3%) 💔

Package salesforce 👍(2) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
setupaudittrail 6289.31 4739.34 -1549.97 (-24.64%) 💔

Package snort 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 17857.14 14285.71 -3571.43 (-20%) 💔

Package snyk 👍(2) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vulnerabilities 5434.78 3690.04 -1744.74 (-32.1%) 💔

Package stormshield 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 2272.73 1754.39 -518.34 (-22.81%) 💔

Package sublime_security 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
message_event 6944.44 5847.95 -1096.49 (-15.79%) 💔

Package system 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
syslog 22222.22 16666.67 -5555.55 (-25%) 💔

Package tanium 👍(2) 💚(1) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
discover 4149.38 3267.97 -881.41 (-21.24%) 💔
endpoint_config 12820.51 5847.95 -6972.56 (-54.39%) 💔
reporting 21739.13 14925.37 -6813.76 (-31.34%) 💔

Package thycotic_ss 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
logs 3367 2739.73 -627.27 (-18.63%) 💔

Package ti_cif3 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
feed 2985.07 1876.17 -1108.9 (-37.15%) 💔

Package ti_cybersixgill 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
threat 3086.42 1972.39 -1114.03 (-36.09%) 💔

Package ti_eclecticiq 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
threat 2577.32 2123.14 -454.18 (-17.62%) 💔

Package ti_eset 👍(3) 💚(1) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
apt 1730.1 1364.26 -365.84 (-21.15%) 💔
botnet 9345.79 5649.72 -3696.07 (-39.55%) 💔
cc 11363.64 7142.86 -4220.78 (-37.14%) 💔

Package ti_rapid7_threat_command 👍(1) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alert 4237.29 3436.43 -800.86 (-18.9%) 💔
ioc 2857.14 1934.24 -922.9 (-32.3%) 💔

Package tomcat 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 76923.08 58823.53 -18099.55 (-23.53%) 💔

Package trendmicro 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
deep_security 1089.32 914.08 -175.24 (-16.09%) 💔

Package vsphere 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 3676.47 3058.1 -618.37 (-16.82%) 💔

Package windows 👍(6) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
applocker_msi_and_script 8771.93 6250 -2521.93 (-28.75%) 💔
powershell_operational 4629.63 3184.71 -1444.92 (-31.21%) 💔

Package wiz 👍(1) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
issue 3333.33 2222.22 -1111.11 (-33.33%) 💔
vulnerability 2481.39 1904.76 -576.63 (-23.24%) 💔

Package zeek 👍(24) 💚(11) 💔(8)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
ntp 52631.58 35714.29 -16917.29 (-32.14%) 💔
signature 62500 20000 -42500 (-68%) 💔
connection 31250 17857.14 -13392.86 (-42.86%) 💔
software 66666.67 55555.56 -11111.11 (-16.67%) 💔
dce_rpc 21276.6 12987.01 -8289.59 (-38.96%) 💔
traceroute 30303.03 22727.27 -7575.76 (-25%) 💔
weird 40000 30303.03 -9696.97 (-24.24%) 💔
x509 13333.33 10989.01 -2344.32 (-17.58%) 💔

Package zerofox 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alerts 3367 2364.07 -1002.93 (-29.79%) 💔

Package zoom 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
webhook 3558.72 2242.15 -1316.57 (-37%) 💔

Package zscaler_zia 👍(5) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
sandbox_report 4166.67 3194.89 -971.78 (-23.32%) 💔
tunnel 4651.16 3745.32 -905.84 (-19.48%) 💔

Package zscaler_zpa 👍(2) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
app_connector_status 1996.01 1552.8 -443.21 (-22.2%) 💔
user_status 6993.01 5405.41 -1587.6 (-22.7%) 💔

To see the full report comment with /test benchmark fullreport

@qcorporation qcorporation marked this pull request as ready for review September 13, 2024 19:48
@qcorporation qcorporation requested a review from a team as a code owner September 13, 2024 19:48
@elasticmachine
Copy link

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

@qcorporation qcorporation removed the request for review from a team September 17, 2024 12:27
Copy link
Contributor

@taylor-swanson taylor-swanson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a filestream input, we should probably add one. Even if the product itself doesn't export directly to a log, we usually add one as the user may have an environment where the logs eventually get to us in the form of a file.

multi: false
required: true
show_user: true
default: 514
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this also be 1470 like TCP?

I'd rather not default to sub-1024 port since this will require root in order to spawn the listener.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call on not using < 1024.

Copy link
Author

@qcorporation qcorporation Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you take a look, the Aruba instructions at the CLI level for setting up log forward: https://www.arubanetworks.com/techdocs/AOS-CX/AOSCX-CLI-Bank/cli_4100i/Content/Chp_RSyslog/RSyslog_cmds/log-10.htm

The defaults are as follows

udp [<PORT-NUM>] | Range: 1 to 65535. Default: 514
tcp [<PORT-NUM>] | Range: 1 to 65535. Default: 1470
tls [<PORT-NUM>] | Range: 1 to 65535. Default: 6514

Trying to adhere to the same defaults from the Aruba documentation as a convenience for our customers to have ease of setup

packages/hpe_aruba_cx/changelog.yml Show resolved Hide resolved
packages/hpe_aruba_cx/data_stream/log/manifest.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@dwhyrock dwhyrock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mostly have some questions about how this was done, especially about all the fields and how they were created/generated.

Overall it looks like a solid start to the integration.

I'll withhold my approval since it looks like CI isn't quite passing:

-| aruba.vrf.id |  | long |
+| aruba.vrf.id |  | keyword |
 | aruba.vrf.name |  | keyword |
Error: checking package failed: checking readme files are up-to-date failed: files do not match

https://buildkite.com/elastic/integrations/builds/16001#0191fffc-07da-4036-903d-c3e19b432020/445-451

@@ -0,0 +1,3 @@
dependencies:
ecs:
reference: "[email protected]"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the decision-making process behind picking this version? This looks like the latest one that's been released from https://github.com/elastic/ecs/tags .

How does this part fit into the broader OTel/ECS efforts?

Understandable if you don't want to address this in this PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwhyrock good question - it looks like we point to the latest ECS release, which at this point is 8.11.0
In general, our integrations for the foreseeable future will always speak ECS.

In contrast, if we were to pick an older ECS version, then we would not be able to specify fields that are released in a later release. That will limit our 'language' in which we can describe the log message that is coming from the appliance.

@@ -0,0 +1,809 @@
- name: aruba
type: group
fields:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious how you populated this file. Did you copy another integration's fields.yml and remove the descriptions? Or did you go into the logs and derive which fields we would need to populate?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see this response for more details

For this file, I auto-generated using prompting into GitHub Copilot after I got the maps down in a file

multi: false
required: true
show_user: true
default: 514
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call on not using < 1024.

Note: Field types are defined within `fields.yml`
Note: Descriptions have not been filled out

#### [AAA events](https://www.arubanetworks.com/techdocs/AOS-CX/10.07/HTML/5200-8214/Content/events/AAA.htm)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also curious about the workflow here. Was this automated in some way? Or was this as tedious as I am imagining it would be?

Copy link
Author

@qcorporation qcorporation Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It started manually. I began to look at the field mappings documentation and broke out the aruba events into their categories. I thought that in the future, we would be able to separate the work into these categories and assign each person working on Aruba a category.

As I continued, I knew that this would NOT be scalable, I leverged GitHub copilot by:
created an Aruba fields file
reduced the ecs_flat.yml file to their field values (without the description, type, etc)

and then I asked the GH copilot to map the Aruba fields to their respective ECS fields if it could. That got me 50% of the way, but I had to hand-inspect each field to see if it made sense and rejig the mappings from the LLM if there was a better mapping.
In that process I also asked GH copilot to alphabetize and deduplicate the fields.
Of the fields that didn't have a direct mapping I tried to find if that field was repeated across the event types, e.g. aruba.cat1.vrfid, and aruba.cat2.vrfid, in which case I'd make a common aruba.* field, which can be reused in all the categories.

Moving forward, there can be lots of automation/tooling to help us in this area, it just hasn't been done yet. I have a lot of ideas that can help reduce this work. Have to go through the pain to want to fix it though

@qcorporation qcorporation marked this pull request as draft September 20, 2024 13:17
@elasticmachine
Copy link

💚 Build Succeeded

History

  • 💔 Build #16230 failed e9fb3eb5549f1ad5e967dfcb0d9f19b74854e22c
  • 💚 Build #16198 succeeded 9702a76e92953f6deb30d8d20e4b45225ec78b22
  • 💔 Build #16197 failed 5e3f48f9e45fc8b27c3785f8050f85c0baae374c
  • 💔 Build #16001 failed 9292bf1c0f213c8221b6bd20a758dda609460729
  • 💔 Build #15951 failed 671a12bd6da5832a8cb8cd47732f1a322d865682

cc @qcorporation

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

Copy link
Contributor

mergify bot commented Sep 20, 2024

⚠️ The sha of the head commit of this PR conflicts with #11201. Mergify cannot evaluate rules on this PR. ⚠️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New Integration Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants