-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🌊 Streams routing UI #201427
🌊 Streams routing UI #201427
Conversation
…into init-streams-plugin
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
definitions, | ||
total: typeof total === 'number' ? total : total.value, | ||
definitions: definitions.filter((_, index) => hasAccess[index]), | ||
total: definitions.length, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should total
reflect the overall total or the total the user hasAccess
to?
If the former (as implemented here) is there any risk avenue in "leaking" that there are potentially more entries?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point - there isn't really a point of sending total at the moment as we don't have pagination anyway. I just removed it
], | ||
}, | ||
}, | ||
runtime_mappings: Object.fromEntries( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you help me understand the runtime_mappings
logic, I think I might be missing something. I understand that this is here to assist with potentially unmapped fields. From the docs:
Runtime fields take precedence over fields defined with the same name in the index mappings.
So if a field is already defined in the mappings (via the stream's fields definition), should that have influence here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As discussed in our sync - Added a blurb about this as comment to explain what's going on.
<img width="659" alt="Screenshot 2024-11-25 at 17 50 18" src="https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854"> This PR adds an empty placeholder for the different parts of the management tab which will be filled later on by follow-up PRs like elastic#201427 Changes: * Make the whole tree of wrapper elements flex so eventual page content can easily grow to the full height of the viewport. This is important for the preview view for routing and parsing. Doing this required some changes to existing listing and detail pages which just took the part of the page they actually needed. The changes come down to setting `grow: false` on the header wrapper and `grow: true` on the content wrapper * Introduce another routing layer: `{key}/{tab}/{subtab}` to support the two-leveled tabs of management. As our routing helpers don't support optional path variables, I slightly extended them to pass through the `optional` flag so the stream detail component can try to parse the path for both key/tab and key/tab/subtab and go with whatever works. * Add the second tabbing layer for the management view component and add placeholders for the three subtabs we are going to need. * Pass through a callback to refresh the stream definition that's passed down - this will come in handy in the various management views because the user can make changes and we want to update the stream definition once the change is submitted --------- Co-authored-by: Dario Gieselaar <[email protected]> (cherry picked from commit 40411b4)
<img width="659" alt="Screenshot 2024-11-25 at 17 50 18" src="https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854"> This PR adds an empty placeholder for the different parts of the management tab which will be filled later on by follow-up PRs like elastic#201427 Changes: * Make the whole tree of wrapper elements flex so eventual page content can easily grow to the full height of the viewport. This is important for the preview view for routing and parsing. Doing this required some changes to existing listing and detail pages which just took the part of the page they actually needed. The changes come down to setting `grow: false` on the header wrapper and `grow: true` on the content wrapper * Introduce another routing layer: `{key}/{tab}/{subtab}` to support the two-leveled tabs of management. As our routing helpers don't support optional path variables, I slightly extended them to pass through the `optional` flag so the stream detail component can try to parse the path for both key/tab and key/tab/subtab and go with whatever works. * Add the second tabbing layer for the management view component and add placeholders for the three subtabs we are going to need. * Pass through a callback to refresh the stream definition that's passed down - this will come in handy in the various management views because the user can make changes and we want to update the stream definition once the change is submitted --------- Co-authored-by: Dario Gieselaar <[email protected]> (cherry picked from commit 40411b4)
# Backport This will backport the following commits from `main` to `8.x`: - [🌊 Streams: Management base page (#201649)](#201649) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Joe Reuter","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-27T18:27:26Z","message":"🌊 Streams: Management base page (#201649)\n\n<img width=\"659\" alt=\"Screenshot 2024-11-25 at 17 50 18\"\r\nsrc=\"https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854\">\r\n\r\nThis PR adds an empty placeholder for the different parts of the\r\nmanagement tab which will be filled later on by follow-up PRs like\r\nhttps://github.com//pull/201427\r\n\r\nChanges:\r\n* Make the whole tree of wrapper elements flex so eventual page content\r\ncan easily grow to the full height of the viewport. This is important\r\nfor the preview view for routing and parsing. Doing this required some\r\nchanges to existing listing and detail pages which just took the part of\r\nthe page they actually needed. The changes come down to setting `grow:\r\nfalse` on the header wrapper and `grow: true` on the content wrapper\r\n* Introduce another routing layer: `{key}/{tab}/{subtab}` to support the\r\ntwo-leveled tabs of management. As our routing helpers don't support\r\noptional path variables, I slightly extended them to pass through the\r\n`optional` flag so the stream detail component can try to parse the path\r\nfor both key/tab and key/tab/subtab and go with whatever works.\r\n* Add the second tabbing layer for the management view component and add\r\nplaceholders for the three subtabs we are going to need.\r\n* Pass through a callback to refresh the stream definition that's passed\r\ndown - this will come in handy in the various management views because\r\nthe user can make changes and we want to update the stream definition\r\nonce the change is submitted\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar <[email protected]>","sha":"40411b4eb59e4578e34081ec0690dd6b34812eac","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","v8.18.0","Feature:Streams"],"title":"🌊 Streams: Management base page","number":201649,"url":"https://github.com/elastic/kibana/pull/201649","mergeCommit":{"message":"🌊 Streams: Management base page (#201649)\n\n<img width=\"659\" alt=\"Screenshot 2024-11-25 at 17 50 18\"\r\nsrc=\"https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854\">\r\n\r\nThis PR adds an empty placeholder for the different parts of the\r\nmanagement tab which will be filled later on by follow-up PRs like\r\nhttps://github.com//pull/201427\r\n\r\nChanges:\r\n* Make the whole tree of wrapper elements flex so eventual page content\r\ncan easily grow to the full height of the viewport. This is important\r\nfor the preview view for routing and parsing. Doing this required some\r\nchanges to existing listing and detail pages which just took the part of\r\nthe page they actually needed. The changes come down to setting `grow:\r\nfalse` on the header wrapper and `grow: true` on the content wrapper\r\n* Introduce another routing layer: `{key}/{tab}/{subtab}` to support the\r\ntwo-leveled tabs of management. As our routing helpers don't support\r\noptional path variables, I slightly extended them to pass through the\r\n`optional` flag so the stream detail component can try to parse the path\r\nfor both key/tab and key/tab/subtab and go with whatever works.\r\n* Add the second tabbing layer for the management view component and add\r\nplaceholders for the three subtabs we are going to need.\r\n* Pass through a callback to refresh the stream definition that's passed\r\ndown - this will come in handy in the various management views because\r\nthe user can make changes and we want to update the stream definition\r\nonce the change is submitted\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar <[email protected]>","sha":"40411b4eb59e4578e34081ec0690dd6b34812eac"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201649","number":201649,"mergeCommit":{"message":"🌊 Streams: Management base page (#201649)\n\n<img width=\"659\" alt=\"Screenshot 2024-11-25 at 17 50 18\"\r\nsrc=\"https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854\">\r\n\r\nThis PR adds an empty placeholder for the different parts of the\r\nmanagement tab which will be filled later on by follow-up PRs like\r\nhttps://github.com//pull/201427\r\n\r\nChanges:\r\n* Make the whole tree of wrapper elements flex so eventual page content\r\ncan easily grow to the full height of the viewport. This is important\r\nfor the preview view for routing and parsing. Doing this required some\r\nchanges to existing listing and detail pages which just took the part of\r\nthe page they actually needed. The changes come down to setting `grow:\r\nfalse` on the header wrapper and `grow: true` on the content wrapper\r\n* Introduce another routing layer: `{key}/{tab}/{subtab}` to support the\r\ntwo-leveled tabs of management. As our routing helpers don't support\r\noptional path variables, I slightly extended them to pass through the\r\n`optional` flag so the stream detail component can try to parse the path\r\nfor both key/tab and key/tab/subtab and go with whatever works.\r\n* Add the second tabbing layer for the management view component and add\r\nplaceholders for the three subtabs we are going to need.\r\n* Pass through a callback to refresh the stream definition that's passed\r\ndown - this will come in handy in the various management views because\r\nthe user can make changes and we want to update the stream definition\r\nonce the change is submitted\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar <[email protected]>","sha":"40411b4eb59e4578e34081ec0690dd6b34812eac"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Joe Reuter <[email protected]>
…ibana into flash1293/streams-routing-ui
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes 👌
@simianhacker could I get a look from your side? |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
History
|
Starting backport for target branches: 8.x |
First iteration of the streams partitioning/routing UI: <img width="1455" alt="Screenshot 2024-11-27 at 21 31 13" src="https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b"> ## Changes ### Unified search bar An empty `FlexItem` would be rendered even if there is no query input which will stick around as a 320px wide empty element. This change avoids rendering this empty wrapper. ### Streams API * Add logic to extract fields (and their types) from a condition * Add logic to turn the streams condition dialect into query dsl * Add dot expander to the root logs pipeline to normalize incoming docs for consistent access in painless and querydsl (this is a stopgap solution) * Add some additional validation to incoming definitions * Add a sample API which takes a stream and a condition, turns the condition into querydsl and searches for a bunch of docs and returns their sources (also sets runtime mappings so non-mapped fields can be used in the condition) ### Streams app * Adjust page height based on whether the new nav is enabled * Add a condition editor to show and change conditions (can also be reused in other UIs) * Add UI to show child routing conditions, change existing routings, partition new streams and delete streams as well --------- Co-authored-by: Chris Cowan <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 4495e74)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
First iteration of the streams partitioning/routing UI: <img width="1455" alt="Screenshot 2024-11-27 at 21 31 13" src="https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b"> ## Changes ### Unified search bar An empty `FlexItem` would be rendered even if there is no query input which will stick around as a 320px wide empty element. This change avoids rendering this empty wrapper. ### Streams API * Add logic to extract fields (and their types) from a condition * Add logic to turn the streams condition dialect into query dsl * Add dot expander to the root logs pipeline to normalize incoming docs for consistent access in painless and querydsl (this is a stopgap solution) * Add some additional validation to incoming definitions * Add a sample API which takes a stream and a condition, turns the condition into querydsl and searches for a bunch of docs and returns their sources (also sets runtime mappings so non-mapped fields can be used in the condition) ### Streams app * Adjust page height based on whether the new nav is enabled * Add a condition editor to show and change conditions (can also be reused in other UIs) * Add UI to show child routing conditions, change existing routings, partition new streams and delete streams as well --------- Co-authored-by: Chris Cowan <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: kibanamachine <[email protected]>
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
1 similar comment
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
# Backport This will backport the following commits from `main` to `8.x`: - [🌊 Streams routing UI (#201427)](#201427) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Joe Reuter","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-04T08:40:18Z","message":"🌊 Streams routing UI (#201427)\n\nFirst iteration of the streams partitioning/routing UI:\r\n\r\n<img width=\"1455\" alt=\"Screenshot 2024-11-27 at 21 31 13\"\r\nsrc=\"https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b\">\r\n\r\n## Changes\r\n\r\n### Unified search bar\r\n\r\nAn empty `FlexItem` would be rendered even if there is no query input\r\nwhich will stick around as a 320px wide empty element. This change\r\navoids rendering this empty wrapper.\r\n\r\n### Streams API\r\n\r\n* Add logic to extract fields (and their types) from a condition\r\n* Add logic to turn the streams condition dialect into query dsl\r\n* Add dot expander to the root logs pipeline to normalize incoming docs\r\nfor consistent access in painless and querydsl (this is a stopgap\r\nsolution)\r\n* Add some additional validation to incoming definitions\r\n* Add a sample API which takes a stream and a condition, turns the\r\ncondition into querydsl and searches for a bunch of docs and returns\r\ntheir sources (also sets runtime mappings so non-mapped fields can be\r\nused in the condition)\r\n\r\n### Streams app\r\n\r\n* Adjust page height based on whether the new nav is enabled\r\n* Add a condition editor to show and change conditions (can also be\r\nreused in other UIs)\r\n* Add UI to show child routing conditions, change existing routings,\r\npartition new streams and delete streams as well\r\n\r\n---------\r\n\r\nCo-authored-by: Chris Cowan <[email protected]>\r\nCo-authored-by: Dario Gieselaar <[email protected]>\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"4495e745ebd807ee6103b87202e1d50dc91a67d9","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","v8.18.0","Feature:Streams"],"title":"🌊 Streams routing UI","number":201427,"url":"https://github.com/elastic/kibana/pull/201427","mergeCommit":{"message":"🌊 Streams routing UI (#201427)\n\nFirst iteration of the streams partitioning/routing UI:\r\n\r\n<img width=\"1455\" alt=\"Screenshot 2024-11-27 at 21 31 13\"\r\nsrc=\"https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b\">\r\n\r\n## Changes\r\n\r\n### Unified search bar\r\n\r\nAn empty `FlexItem` would be rendered even if there is no query input\r\nwhich will stick around as a 320px wide empty element. This change\r\navoids rendering this empty wrapper.\r\n\r\n### Streams API\r\n\r\n* Add logic to extract fields (and their types) from a condition\r\n* Add logic to turn the streams condition dialect into query dsl\r\n* Add dot expander to the root logs pipeline to normalize incoming docs\r\nfor consistent access in painless and querydsl (this is a stopgap\r\nsolution)\r\n* Add some additional validation to incoming definitions\r\n* Add a sample API which takes a stream and a condition, turns the\r\ncondition into querydsl and searches for a bunch of docs and returns\r\ntheir sources (also sets runtime mappings so non-mapped fields can be\r\nused in the condition)\r\n\r\n### Streams app\r\n\r\n* Adjust page height based on whether the new nav is enabled\r\n* Add a condition editor to show and change conditions (can also be\r\nreused in other UIs)\r\n* Add UI to show child routing conditions, change existing routings,\r\npartition new streams and delete streams as well\r\n\r\n---------\r\n\r\nCo-authored-by: Chris Cowan <[email protected]>\r\nCo-authored-by: Dario Gieselaar <[email protected]>\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"4495e745ebd807ee6103b87202e1d50dc91a67d9"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201427","number":201427,"mergeCommit":{"message":"🌊 Streams routing UI (#201427)\n\nFirst iteration of the streams partitioning/routing UI:\r\n\r\n<img width=\"1455\" alt=\"Screenshot 2024-11-27 at 21 31 13\"\r\nsrc=\"https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b\">\r\n\r\n## Changes\r\n\r\n### Unified search bar\r\n\r\nAn empty `FlexItem` would be rendered even if there is no query input\r\nwhich will stick around as a 320px wide empty element. This change\r\navoids rendering this empty wrapper.\r\n\r\n### Streams API\r\n\r\n* Add logic to extract fields (and their types) from a condition\r\n* Add logic to turn the streams condition dialect into query dsl\r\n* Add dot expander to the root logs pipeline to normalize incoming docs\r\nfor consistent access in painless and querydsl (this is a stopgap\r\nsolution)\r\n* Add some additional validation to incoming definitions\r\n* Add a sample API which takes a stream and a condition, turns the\r\ncondition into querydsl and searches for a bunch of docs and returns\r\ntheir sources (also sets runtime mappings so non-mapped fields can be\r\nused in the condition)\r\n\r\n### Streams app\r\n\r\n* Adjust page height based on whether the new nav is enabled\r\n* Add a condition editor to show and change conditions (can also be\r\nreused in other UIs)\r\n* Add UI to show child routing conditions, change existing routings,\r\npartition new streams and delete streams as well\r\n\r\n---------\r\n\r\nCo-authored-by: Chris Cowan <[email protected]>\r\nCo-authored-by: Dario Gieselaar <[email protected]>\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"4495e745ebd807ee6103b87202e1d50dc91a67d9"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Joe Reuter <[email protected]>
First iteration of the streams partitioning/routing UI: <img width="1455" alt="Screenshot 2024-11-27 at 21 31 13" src="https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b"> ## Changes ### Unified search bar An empty `FlexItem` would be rendered even if there is no query input which will stick around as a 320px wide empty element. This change avoids rendering this empty wrapper. ### Streams API * Add logic to extract fields (and their types) from a condition * Add logic to turn the streams condition dialect into query dsl * Add dot expander to the root logs pipeline to normalize incoming docs for consistent access in painless and querydsl (this is a stopgap solution) * Add some additional validation to incoming definitions * Add a sample API which takes a stream and a condition, turns the condition into querydsl and searches for a bunch of docs and returns their sources (also sets runtime mappings so non-mapped fields can be used in the condition) ### Streams app * Adjust page height based on whether the new nav is enabled * Add a condition editor to show and change conditions (can also be reused in other UIs) * Add UI to show child routing conditions, change existing routings, partition new streams and delete streams as well --------- Co-authored-by: Chris Cowan <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: kibanamachine <[email protected]>
<img width="659" alt="Screenshot 2024-11-25 at 17 50 18" src="https://github.com/user-attachments/assets/1f623207-a5cb-4a76-9d0e-c4ff811ab854"> This PR adds an empty placeholder for the different parts of the management tab which will be filled later on by follow-up PRs like elastic#201427 Changes: * Make the whole tree of wrapper elements flex so eventual page content can easily grow to the full height of the viewport. This is important for the preview view for routing and parsing. Doing this required some changes to existing listing and detail pages which just took the part of the page they actually needed. The changes come down to setting `grow: false` on the header wrapper and `grow: true` on the content wrapper * Introduce another routing layer: `{key}/{tab}/{subtab}` to support the two-leveled tabs of management. As our routing helpers don't support optional path variables, I slightly extended them to pass through the `optional` flag so the stream detail component can try to parse the path for both key/tab and key/tab/subtab and go with whatever works. * Add the second tabbing layer for the management view component and add placeholders for the three subtabs we are going to need. * Pass through a callback to refresh the stream definition that's passed down - this will come in handy in the various management views because the user can make changes and we want to update the stream definition once the change is submitted --------- Co-authored-by: Dario Gieselaar <[email protected]>
First iteration of the streams partitioning/routing UI: <img width="1455" alt="Screenshot 2024-11-27 at 21 31 13" src="https://github.com/user-attachments/assets/9768b0a0-6143-41a2-9cc0-ed25bbf0d38b"> ## Changes ### Unified search bar An empty `FlexItem` would be rendered even if there is no query input which will stick around as a 320px wide empty element. This change avoids rendering this empty wrapper. ### Streams API * Add logic to extract fields (and their types) from a condition * Add logic to turn the streams condition dialect into query dsl * Add dot expander to the root logs pipeline to normalize incoming docs for consistent access in painless and querydsl (this is a stopgap solution) * Add some additional validation to incoming definitions * Add a sample API which takes a stream and a condition, turns the condition into querydsl and searches for a bunch of docs and returns their sources (also sets runtime mappings so non-mapped fields can be used in the condition) ### Streams app * Adjust page height based on whether the new nav is enabled * Add a condition editor to show and change conditions (can also be reused in other UIs) * Add UI to show child routing conditions, change existing routings, partition new streams and delete streams as well --------- Co-authored-by: Chris Cowan <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: kibanamachine <[email protected]>
First iteration of the streams partitioning/routing UI:
Changes
Unified search bar
An empty
FlexItem
would be rendered even if there is no query input which will stick around as a 320px wide empty element. This change avoids rendering this empty wrapper.Streams API
Streams app