Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grouping feature #133

Merged
merged 28 commits into from
Nov 22, 2023
Merged

Grouping feature #133

merged 28 commits into from
Nov 22, 2023

Conversation

Lasica
Copy link
Contributor

@Lasica Lasica commented Nov 17, 2023

Description

Added grouping nodes execution to single vertexai nodes. Updated docs.

PR Checklist

Copy link
Contributor

@marrrcin marrrcin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments, really nice job 🥩!

CHANGELOG.md Outdated Show resolved Hide resolved
docs/source/02_installation/02_configuration.md Outdated Show resolved Hide resolved
docs/source/02_installation/02_configuration.md Outdated Show resolved Hide resolved
docs/source/02_installation/02_configuration.md Outdated Show resolved Hide resolved
docs/source/02_installation/02_configuration.md Outdated Show resolved Hide resolved
kedro_vertexai/config.py Outdated Show resolved Hide resolved
kedro_vertexai/grouping.py Show resolved Hide resolved
kedro_vertexai/grouping.py Outdated Show resolved Hide resolved
node(some_operation, "C", "D", name="node3", tags=["baz"]),
])
```
The result will be 2 VertexAI nodes for this pipeline, first with name `nodegroup` that will run `node1` and `node2` Kedro nodes inside and provide output `C` and second VertexAI node: `node3`. Additional MLflow node can be present if `kedro-mlflow` is used. Right now it is not possible to group it. If you feel you need that functionality search for/create an issue on [github page of the plugin](https://github.com/getindata/kedro-vertexai/issues).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it would be awesome to show the following thing in the documentation below the description:

  1. Screenshot of Kedro-viz of regular spaceflights pipeline
  2. Screenshot of code from e2e tests with tags (groups)
  3. Screenshot from Vertex AI
    All side-by-side with arrows from one to another. Visualization always speaks better than a wall of text and people will know at glance what to expect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I'll add the pictures.

@Lasica Lasica merged commit ecba16f into develop Nov 22, 2023
13 checks passed
@Lasica Lasica deleted the grouping-feature branch November 22, 2023 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants