[ResponseOps][OnWeek] Connector logs view #148291

doakalexi · 2023-01-03T12:38:16Z

Resolves #147795

Summary

Adds a connectors event log tab that gives the ability to see historical activity of connectors.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

To verify

Create a rule with a connector, create a few to help test the search capability
Go to http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/logs
Verify that you can see the execution logs for the connector
Verify that you can search
Verify that you can filter by error/success

…ors-event-logs

…-fix'

…kibana into on-week/connectors-event-logs

…-fix'

…kibana into on-week/connectors-event-logs

elasticmachine · 2023-01-09T18:42:07Z

Pinging @elastic/response-ops (Team:ResponseOps)

ymao1

Initial code review of server code. It's looking good but I think we could simplify the aggregations for this.

x-pack/plugins/actions/common/execution_log_types.ts

x-pack/plugins/actions/server/actions_client.ts

x-pack/plugins/actions/server/lib/create_action_event_log_record_object.ts

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts

Co-authored-by: Ying Mao <[email protected]>

pmuellr

finished the server-side review of this PR

x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts

x-pack/plugins/actions/server/actions_client.ts

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts

x-pack/plugins/actions/server/lib/task_runner_factory.ts

x-pack/plugins/actions/server/routes/get_global_execution_logs.ts

…kibana into on-week/connectors-event-logs

pmuellr

LGTM, and I think we can remove that "Filter out execution UUID buckets where actionExecution doc count is 0" bit, if we don't really need it.

ymao1

I noticed that preconfigured connector logs only show up in the default space. I think since preconfigured connectors are available in all spaces, the logs for them should show up in all spaces automatically, WDYT?

x-pack/plugins/actions/server/lib/create_action_event_log_record_object.ts

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts

x-pack/plugins/actions/server/actions_client.ts

ymao1 · 2023-01-18T13:58:37Z

I created a user with Read access to Actions and Connectors and logged in with them I can see the Connectors page but when I go to the Logs tab, I get a 403 unauthorized response. Should a read-only user be able to see these logs?

…ors-event-logs

doakalexi · 2023-01-18T15:33:52Z

I created a user with Read access to Actions and Connectors and logged in with them I can see the Connectors page but when I go to the Logs tab, I get a 403 unauthorized response. Should a read-only user be able to see these logs?

Resolved in this commit 5104f14

doakalexi · 2023-01-18T20:48:20Z

I noticed that preconfigured connector logs only show up in the default space. I think since preconfigured connectors are available in all spaces, the logs for them should show up in all spaces automatically, WDYT?

Resolved in this commit cb9821b

ymao1

Everything looks great and works as expected!

x-pack/plugins/triggers_actions_ui/public/application/constants/index.ts

pmuellr

Looking good. Noted one expression that might be able to be simplified, but also needs another ?. optional property operator.

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts

pmuellr

LGTM! !!!

JiaweiWu

I tested this locally, seems to work as expected. I just have a few comments

JiaweiWu · 2023-01-19T18:53:48Z

...tion/sections/actions_connectors_list/components/actions_connectors_event_log_list_table.tsx

+    [search, setSearchText]
+  );
+
+  const onShowAllSpacesChange = useCallback(() => {


I think it would be nice if we didn't duplicate this logic as much, maybe a hook to encapsulate some of this space logic? (i.e. useMultipleSpaces returning onShowAllSpacesChange and canAccessMultipleSpaces)

Resolved in this commit b9debef

JiaweiWu · 2023-01-19T19:17:46Z

...tion/sections/actions_connectors_list/components/actions_connectors_event_log_list_table.tsx

+    }
+  }, [setShowFromAllSpaces, showFromAllSpaces, visibleColumns]);
+
+  const columns: EuiDataGridColumn[] = useMemo(


Did we want to make the connector name the first column like how we have it for global rule event logs?

Also did we want to have the connectors clickable? I think it might be helpful in the cross-space scenario to navigate the user to the corresponding connector page

Oh that is a good idea! I am wondering if this maybe could be a follow on issue?

Did we want to make the connector name the first column like how we have it for global rule event logs?

I don't think so, UX asked for this column order.

JiaweiWu · 2023-01-19T19:36:18Z

...tion/sections/actions_connectors_list/components/actions_connectors_event_log_list_table.tsx

@@ -0,0 +1,616 @@
+/*


Should we have a unit test for this? This seems like the main component of this feature

Resolved in this commit 38e4176

JiaweiWu · 2023-01-19T19:41:49Z

...n/sections/actions_connectors_list/components/actions_connectors_event_log_list_kpi.test.tsx

+
+import React from 'react';
+import { act } from 'react-dom/test-utils';
+import { mountWithIntl, nextTick } from '@kbn/test-jest-helpers';


I know this was written following existing patterns but for mounting react components, we are moving away from enzyme, in favour of testing-library/react, you don't have to change this test (unless you really want to haha) but just a FYI.

Oh okay, thanks for letting me know!

…ors-event-logs

JiaweiWu

LGTM! I just have 1 more comment. Feel free to merge after addressing it

JiaweiWu · 2023-01-23T20:43:21Z

...actions_ui/public/application/sections/rule_details/components/rule_event_log_list_table.tsx

-    () => (showFromAllSpaces && spacesData ? accessibleSpaceIds : undefined),
-    [showFromAllSpaces, spacesData, accessibleSpaceIds]
-  );
+  const { onShowAllSpacesChange, canAccessMultipleSpaces, namespaces } = useMultipleSpaces({


let's move the useSpacesData call above into the hook since it's not being used anywhere else, same with actions_connectors event log table. We won't have to pass in spacesData into the hook.

Resolved in this commit b7c1703

kibana-ci · 2023-01-23T22:33:01Z

💚 Build Succeeded

Buildkite Build
Commit: b7c1703

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`actions`	12	13	+1
`triggersActionsUi`	487	495	+8
total			+9

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`actions`	215	251	+36
`eventLog`	115	116	+1
total			+37

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`triggersActionsUi`	707.7KB	722.1KB	+14.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`actions`	15.4KB	15.6KB	+217.0B
`triggersActionsUi`	114.9KB	116.2KB	+1.3KB
total			+1.5KB

Unknown metric groups

API count

id	before	after	diff
`actions`	220	256	+36
`eventLog`	115	116	+1
total			+37

async chunk count

id	before	after	diff
`triggersActionsUi`	54	57	+3

ESLint disabled in files

id	before	after	diff
`triggersActionsUi`	3	4	+1

ESLint disabled line counts

id	before	after	diff
`triggersActionsUi`	116	123	+7

Total ESLint disabled count

id	before	after	diff
`triggersActionsUi`	119	127	+8

History

💛 Build #101933 was flaky b9debef
💔 Build #101904 failed 38e4176
💚 Build #101499 succeeded 975b39f
💔 Build #101484 failed 148e496
💔 Build #101443 failed e3ed996
💔 Build #101401 failed ad8d2c4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

doakalexi and others added 18 commits December 6, 2022 08:49

Initial commit

15009db

Merge branch 'main' of github.com:elastic/kibana into on-week/connect…

3e0b06c

…ors-event-logs

Fixing query

e908595

Server side cleanup

be35b24

Reusing components

5af7883

More refactoring

d42514d

Merge branch 'main' of github.com:elastic/kibana into on-week/connect…

c804e93

…ors-event-logs

[CI] Auto-commit changed files from 'node scripts/ts_project_linter -…

6b54903

…-fix'

Fixing failed tests

311984a

Merge branch 'on-week/connectors-event-logs' of github.com:doakalexi/…

ee1f371

…kibana into on-week/connectors-event-logs

Adding new tests

2b64501

Remove circular dependency

568bba1

[CI] Auto-commit changed files from 'node scripts/ts_project_linter -…

53c5573

…-fix'

Fixing other circular dependency

605e4d1

Merge branch 'on-week/connectors-event-logs' of github.com:doakalexi/…

41a6fd3

…kibana into on-week/connectors-event-logs

Removing alerting

9f6f15f

Fixing types and test failures

94cf58f

Fixing mappings

6b82166

doakalexi changed the title ~~On week/connectors event logs~~ [ResponseOps][OnWeek] Connector logs view Jan 9, 2023

doakalexi added Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) release_note:feature Makes this part of the condensed release notes labels Jan 9, 2023

Fixing types again

5403126

doakalexi marked this pull request as ready for review January 9, 2023 18:42

doakalexi requested review from a team as code owners January 9, 2023 18:42

Merge branch 'main' into on-week/connectors-event-logs

01996a5

ymao1 requested changes Jan 10, 2023

View reviewed changes

doakalexi and others added 2 commits January 10, 2023 07:16

Update x-pack/plugins/actions/server/actions_client.ts

05828dd

Co-authored-by: Ying Mao <[email protected]>

Addressing pr feedback

578d433

pmuellr requested changes Jan 17, 2023

View reviewed changes

doakalexi added 2 commits January 17, 2023 13:32

Changing from get to post

566619b

Merge branch 'on-week/connectors-event-logs' of github.com:doakalexi/…

4526a09

…kibana into on-week/connectors-event-logs

pmuellr approved these changes Jan 17, 2023

View reviewed changes

ymao1 reviewed Jan 18, 2023

View reviewed changes

x-pack/plugins/actions/server/lib/create_action_event_log_record_object.ts Show resolved Hide resolved

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts Show resolved Hide resolved

x-pack/plugins/actions/server/actions_client.ts Show resolved Hide resolved

doakalexi added 3 commits January 18, 2023 09:30

Merge branch 'main' of github.com:elastic/kibana into on-week/connect…

5be4c9a

…ors-event-logs

Merge branch 'main' of github.com:elastic/kibana into on-week/connect…

7982161

…ors-event-logs

Addressing feedback

5104f14

Adding space agnostic filter

cb9821b

Adding preconfigured to all the logs

ad8d2c4

ymao1 approved these changes Jan 19, 2023

View reviewed changes

x-pack/plugins/triggers_actions_ui/public/application/constants/index.ts Show resolved Hide resolved

doakalexi added 2 commits January 19, 2023 10:53

Adding connector id field

e3ed996

Adding functional test

148e496

pmuellr requested changes Jan 19, 2023

View reviewed changes

x-pack/plugins/actions/server/lib/get_execution_log_aggregation.ts Outdated Show resolved Hide resolved

Fixing test failures

975b39f

pmuellr approved these changes Jan 19, 2023

View reviewed changes

JiaweiWu reviewed Jan 19, 2023

View reviewed changes

doakalexi added 3 commits January 23, 2023 08:36

Merge branch 'main' of github.com:elastic/kibana into on-week/connect…

2a46c93

…ors-event-logs

Adding new tests

38e4176

Added hook

b9debef

JiaweiWu approved these changes Jan 23, 2023

View reviewed changes

Added to the spaces hook

b7c1703

doakalexi enabled auto-merge (squash) January 23, 2023 20:50

doakalexi merged commit 77742b8 into elastic:main Jan 23, 2023

kibanamachine added v8.7.0 backport:skip This commit does not require backporting labels Jan 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ResponseOps][OnWeek] Connector logs view #148291

[ResponseOps][OnWeek] Connector logs view #148291

doakalexi commented Jan 3, 2023 •

edited

Loading

elasticmachine commented Jan 9, 2023

ymao1 left a comment

pmuellr left a comment

pmuellr left a comment

ymao1 left a comment

ymao1 commented Jan 18, 2023

doakalexi commented Jan 18, 2023

doakalexi commented Jan 18, 2023

ymao1 left a comment

pmuellr left a comment

pmuellr left a comment

JiaweiWu left a comment

JiaweiWu Jan 19, 2023

doakalexi Jan 23, 2023

JiaweiWu Jan 19, 2023

JiaweiWu Jan 19, 2023

doakalexi Jan 23, 2023 •

edited

Loading

doakalexi Jan 23, 2023

JiaweiWu Jan 19, 2023

doakalexi Jan 23, 2023

JiaweiWu Jan 19, 2023

doakalexi Jan 23, 2023

JiaweiWu left a comment

JiaweiWu Jan 23, 2023

doakalexi Jan 23, 2023

kibana-ci commented Jan 23, 2023

API count

async chunk count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

[ResponseOps][OnWeek] Connector logs view #148291

[ResponseOps][OnWeek] Connector logs view #148291

Conversation

doakalexi commented Jan 3, 2023 • edited Loading

Summary

Checklist

To verify

elasticmachine commented Jan 9, 2023

ymao1 left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

ymao1 left a comment

Choose a reason for hiding this comment

ymao1 commented Jan 18, 2023

doakalexi commented Jan 18, 2023

doakalexi commented Jan 18, 2023

ymao1 left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

JiaweiWu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

doakalexi Jan 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JiaweiWu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Jan 23, 2023

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

async chunk count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

doakalexi commented Jan 3, 2023 •

edited

Loading

doakalexi Jan 23, 2023 •

edited

Loading