[SecuritySolution] Service Entity Store

[machadoum]

Summary

Service Definition:

https://github.com/elastic/kibana/pull/202344/files#diff-42c7dd345e0500c97f85824904a70a11162827ea8f8df6982082a9047ca04ff1

Acceptance Criteria

• Upon installation of the entity store, the Service entity definition should be created by default
• The Service definition will be installed in the exact same way as the User and Host definitions
• The unique identifier for service entities will be service.name
• The fields captured for service entities should match the field mapping spreadsheet (see Implementation Notes below)

Stored Entity

{
          "@timestamp": "2024-12-02T10:43:13.856Z",
          "event": {
            "ingested": "2024-12-02T10:51:28.987428Z"
          },
          "entity": {
            "name": "test123 name",
            "id": "test123 name",
            "source": "logs-blito",
            "type": "service"
          },
          "service": {
            "node": {
              "roles": [
                "test123 node roles"
              ],
              "name": [
                "test123 node name"
              ]
            },
            "environment": [
              "test123 environment"
            ],
            "address": [
              "test123 address"
            ],
            "name": "test123 name",
            "id": [
              "test123 id"
            ],
            "state": [
              "test123 state"
            ],
            "ephemeral_id": [
              "test123 ephemeral_id"
            ],
            "type": [
              "test123 type"
            ],
            "version": [
              "test123 version"
            ]
          }
}

How to test it?

• Start Kibana

Create mappings

PUT /logs-test
{
  "mappings": {
    "properties": {      
      "service.name": {
        "type": "keyword"
      },
      "service.address": {
        "type": "keyword"
      },
      "service.environment": {
        "type": "keyword"
      },
      "service.ephemeral_id": {
        "type": "keyword"
      },
      "service.id": {
        "type": "keyword"
      },
      "service.node.name": {
        "type": "keyword"
      },
      "service.node.roles": {
        "type": "keyword"
      },
      "service.state": {
        "type": "keyword"
      },
      "service.type": {
        "type": "keyword"
      },
      "service.version": {
        "type": "keyword"
      },
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

Create document

PUT /logs-test
POST logs-test/_doc
{
  "service": {
    "name": "test123 name",
    "address": "test123 address",
    "environment": "test123 environment",
    "ephemeral_id": "test123 ephemeral_id",
    "id": "test123 id",
    "node.roles": "test123 node roles",
    "node.name": "test123 node name",    
    "state": "test123 state",
    "type": "test123 type",
    "version": "test123 version"
  },
  "@timestamp": "2024-12-02T10:43:13.856Z"
}

• Init the entity store
• Wait...
• Query the service index GET .entities.v1.latest.security_service_default/_search

Open Questions

• Can we merge this PR without first updating all other features that will use service entities?
  • If we merge it, the service engine will be installed together with other entities, but it won't provide any functionality
• Do we need an experimental flag?

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

[joedatlive]

@jcoyne-elastic Hi Jordyn. This PR captures the new Service entity fields we are targeting for the next release. Does customer 0 include or use service.name values in your events? Are these service.* fields satisfactory?

[jcoyne-elastic]

@jcoyne-elastic Hi Jordyn. This PR captures the new Service entity fields we are targeting for the next release. Does customer 0 include or use service.name values in your events? Are these service.* fields satisfactory?

We do use service.name in some autditbeat logs, @aarju do you know when service.* is specifically used and what fields we would look to add to a service entity store if relevant?

[tiansivive]

Thanks Pablo!
I have some suggestions regarding the mutation hooks.

Otherwise LGTM!

[machadoum]

@joedatlive @jaredburgettelastic On this PR, I am collecting all values for all service fields. We might want to store only the most recent value for some fields, as we do for risk score and asset criticality. If so, let me know which field should have this behaviour.

Some candidates: service.state and service.version

[tiansivive]

🚢

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12240705761

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 281681d

Failed CI Steps

• Jest Tests #2

Test Failures

• [job] [logs] Jest Tests #2 / StepDefinePackagePolicy default API response should display vars coming from package policy

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	14.6MB	14.6MB	+136.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	87.8KB	87.9KB	+29.0B

History

• 💚 Build #258218 succeeded 085ab88
• 💚 Build #257023 succeeded e48bca4
• 💔 Build #256955 failed 50dd854
• 💔 Build #256906 failed 6943a53
• 💔 Build #256456 failed 9e68c9b
• 💚 Build #255996 succeeded 33279ef

cc @machadoum

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 202344

Questions ?

Please refer to the Backport tool documentation

[machadoum]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation