Feature branch/remove experiment tracking (#2237)

* [Frontend] Remove experiment tracking (#2212) * Remove exp tracking from feature hits and global toolbar; remove ExperimentsIcon Signed-off-by: tynandebold <[email protected]> * Remove /experiment-tracking directory; fix tests Signed-off-by: tynandebold <[email protected]> * Remove exp. tracking utils, reducers, styles Signed-off-by: tynandebold <[email protected]> * Update moto requirement from ~=5.0.9 to ~=5.0.21 in /package (#2218) Updates the requirements on [moto](https://github.com/getmoto/moto) to permit the latest version. - [Release notes](https://github.com/getmoto/moto/releases) - [Changelog](https://github.com/getmoto/moto/blob/master/CHANGELOG.md) - [Commits](getmoto/moto@5.0.9...5.0.21) --- updated-dependencies: - dependency-name: moto dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update httpx requirement from ~=0.27.0 to ~=0.28.0 in /package (#2216) Updates the requirements on [httpx](https://github.com/encode/httpx) to permit the latest version. - [Release notes](https://github.com/encode/httpx/releases) - [Changelog](https://github.com/encode/httpx/blob/master/CHANGELOG.md) - [Commits](encode/httpx@0.27.0...0.28.0) --- updated-dependencies: - dependency-name: httpx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update bandit requirement from ~=1.7 to ~=1.8 in /package (#2215) Updates the requirements on [bandit](https://github.com/PyCQA/bandit) to permit the latest version. - [Release notes](https://github.com/PyCQA/bandit/releases) - [Commits](PyCQA/bandit@1.7.0...1.8.0) --- updated-dependencies: - dependency-name: bandit dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.34 to ~=1.35 in /package (#2214) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Commits](boto/boto3@1.34.0...1.35.71) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jitendra Gundaniya <[email protected]> * Revert change from PR #2179 Signed-off-by: tynandebold <[email protected]> * Remove the sanitizedPathname() function Signed-off-by: tynandebold <[email protected]> * Remove Cypress tests for exp tracking Signed-off-by: tynandebold <[email protected]> * Merge localStorageState state instead of the older allLocalStorageState Signed-off-by: tynandebold <[email protected]> --------- Signed-off-by: tynandebold <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jitendra Gundaniya <[email protected]> * Feature/remove session store (#2219) * remove session store Signed-off-by: Huong Nguyen <[email protected]> * remove relateds tests Signed-off-by: Huong Nguyen <[email protected]> * remove doc about session store Signed-off-by: Huong Nguyen <[email protected]> * lint formatting Signed-off-by: Huong Nguyen <[email protected]> * remove session store from tests Signed-off-by: Huong Nguyen <[email protected]> * update docs reference Signed-off-by: Huong Nguyen <[email protected]> * update test_server Signed-off-by: Huong Nguyen <[email protected]> * remove sessions store from load_data Signed-off-by: Huong Nguyen <[email protected]> * remove make_db_session_factory Signed-off-by: Huong Nguyen <[email protected]> * remove session store from _load_data_helper Signed-off-by: Huong Nguyen <[email protected]> * remove database.py entirely Signed-off-by: Huong Nguyen <[email protected]> * remove test graphql folder Signed-off-by: Huong Nguyen <[email protected]> * remove cy test related to ET tab Signed-off-by: Huong Nguyen <[email protected]> * Revert changes for .md as it will be done separately Signed-off-by: Huong Nguyen <[email protected]> * revert contributing.md Signed-off-by: Huong Nguyen <[email protected]> * remove e2e test failed from ET Signed-off-by: Huong Nguyen <[email protected]> --------- Signed-off-by: Huong Nguyen <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> * Feature/remove-grahpql (#2222) * revert back the getVersion in graphql Signed-off-by: Huong Nguyen <[email protected]> * update queries and schema Signed-off-by: Huong Nguyen <[email protected]> * fix lint error Signed-off-by: Huong Nguyen <[email protected]> * revert graphql_router since it's still using in getVersion Signed-off-by: Huong Nguyen <[email protected]> * remove experiment-tracking.py, and runs and tracking datatset from repos Signed-off-by: Huong Nguyen <[email protected]> * fix lint Signed-off-by: Huong Nguyen <[email protected]> * update format lint Signed-off-by: Huong Nguyen <[email protected]> * remove set_database_session Signed-off-by: Huong Nguyen <[email protected]> --------- Signed-off-by: Huong Nguyen <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> * Feature/remove et doc (#2232) * revert back the getVersion in graphql Signed-off-by: Huong Nguyen <[email protected]> * update queries and schema Signed-off-by: Huong Nguyen <[email protected]> * fix lint error Signed-off-by: Huong Nguyen <[email protected]> * revert graphql_router since it's still using in getVersion Signed-off-by: Huong Nguyen <[email protected]> * remove experiment-tracking.py, and runs and tracking datatset from repos Signed-off-by: Huong Nguyen <[email protected]> * fix lint Signed-off-by: Huong Nguyen <[email protected]> * update format lint Signed-off-by: Huong Nguyen <[email protected]> * remove set_database_session Signed-off-by: Huong Nguyen <[email protected]> * remove ET mentioned in the doc and readme Signed-off-by: Huong Nguyen <[email protected]> * remove ET and session store from readme Signed-off-by: Huong Nguyen <[email protected]> * update new architecture diagram to remove apollo graphQL Signed-off-by: Huong Nguyen <[email protected]> --------- Signed-off-by: Huong Nguyen <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> * Get version using restAPI (#2242) * set up rest for getversion Signed-off-by: Huong Nguyen <[email protected]> * include get_latest_version, is_running_outdated_version Signed-off-by: Huong Nguyen <[email protected]> * fixing the error Signed-off-by: Huong Nguyen <[email protected]> * remove graphql Signed-off-by: Huong Nguyen <[email protected]> * include test for getting version Signed-off-by: Huong Nguyen <[email protected]> * remove check schema in lint Signed-off-by: Huong Nguyen <[email protected]> * fix lint format error Signed-off-by: Huong Nguyen <[email protected]> * remove test for graphql Signed-off-by: Huong Nguyen <[email protected]> * fix syntax error Signed-off-by: Huong Nguyen <[email protected]> * remove apollo Signed-off-by: Huong Nguyen <[email protected]> * update libasound2 to libasound2t64 with the lates ubutun Signed-off-by: Huong Nguyen <[email protected]> * update libgconf-2-4 Signed-off-by: Huong Nguyen <[email protected]> * update versions to version Signed-off-by: Huong Nguyen <[email protected]> --------- Signed-off-by: Huong Nguyen <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> * remove deprecation banner for ET Signed-off-by: Huong Nguyen <[email protected]> * update release note Signed-off-by: Huong Nguyen <[email protected]> * Experiment Tracking CleanUp (#2278) I did a final clean-up of Experiment Tracking on Kedro-viz mostly minor stuff here and there removed dataset_factory resolver implementation as it only affected Exp. Tracking also Experiment Tracking last run data was also originally showed in the flowchart metadata panel, so I removed that besides that some plotly references and other stuff. also quite a few of these files are removing experiment tracking data files from demo-project. * update github-pr-check to github-pr-review to avoid the max file review on PR Signed-off-by: Huong Nguyen <[email protected]> * ignore value on mergekeeper Signed-off-by: rashidakanchwala <[email protected]> * Fix typo in ignored CircleCI checks list --------- Signed-off-by: tynandebold <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Huong Nguyen <[email protected]> Signed-off-by: rashidakanchwala <[email protected]> Co-authored-by: Tynan DeBold <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jitendra Gundaniya <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> Co-authored-by: rashidakanchwala <[email protected]> Co-authored-by: rashidakanchwala <[email protected]>
kedro-org · Feb 25, 2025 · 0abdc4e · 0abdc4e
1 parent 1f4770f
commit 0abdc4e
Show file tree

Hide file tree

Showing 327 changed files with 52,035 additions and 3,128,052 deletions.
diff --git a/.github/img/frontend-architecture.png b/.github/img/frontend-architecture.png
diff --git a/.github/workflows/docs-language-linter.yml b/.github/workflows/docs-language-linter.yml
@@ -14,4 +14,4 @@ jobs:
       - uses: actions/checkout@v4
       - uses: errata-ai/vale-action@reviewdog
         with:
-          reporter: github-pr-check
+          reporter: github-pr-review
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -38,8 +38,5 @@ jobs:
           - name: Run security scan
             run: make security-scan
 
-          - name: Verify GraphQL schema is up to date
-            run: make schema-check
-
           - name: Run Python formatters and linters
             run: make format-check lint-check          
diff --git a/.github/workflows/merge-gatekeeper.yml b/.github/workflows/merge-gatekeeper.yml
@@ -27,4 +27,4 @@ jobs:
           token: ${{ secrets.GITHUB_TOKEN }}
           timeout: ${{ env.TIMEOUT }}
           interval: ${{ env.INTERVAL }}
-          ignored: 'ci/circleci: win_unit_tests-3.9,ci/circleci: win_unit_tests-3.10,ci/circleci: win_unit_tests-3.8,ci/circleci: unit_tests-3.10,ci/circleci: unit_tests-3.8,ci/circleci: unit_tests-3.9,ci/circleci: win_e2e_tests-3.10,ci/circleci: win_e2e_tests-3.9,ci/circleci: win_e2e_tests-3.8,ci/circleci: e2e_tests-3.8,ci/circleci: e2e_tests-3.9,ci/circleci: e2e_tests-3.10,ci/circleci: lint-3.8,ci/circleci: lint-3.9,ci/circleci: lint-3.10,ci/circleci: javascript_lint_and_tests,ci/circleci: check-updated-files,ci/circleci: all_circleci_checks_succeeded,CircleCI Pipeline'
+          ignored: 'ci/circleci: win_unit_tests-3.9,ci/circleci: win_unit_tests-3.10,ci/circleci: win_unit_tests-3.8,ci/circleci: unit_tests-3.10,ci/circleci: unit_tests-3.8,ci/circleci: unit_tests-3.9,ci/circleci: win_e2e_tests-3.10,ci/circleci: win_e2e_tests-3.9,ci/circleci: win_e2e_tests-3.8,ci/circleci: e2e_tests-3.8,ci/circleci: e2e_tests-3.9,ci/circleci: e2e_tests-3.10,ci/circleci: lint-3.8,ci/circleci: lint-3.9,ci/circleci: lint-3.10,ci/circleci: javascript_lint_and_tests,ci/circleci: check-updated-files,ci/circleci: all_circleci_checks_succeeded,CircleCI Pipeline, runner / vale'
diff --git a/.gitignore b/.gitignore
@@ -65,3 +65,7 @@ coverage.xml
 
 # Kedro
 *.log
+
+# Python
+.venv/
+mise.toml
diff --git a/.stylelintignore b/.stylelintignore
@@ -1,5 +1,4 @@
 src/**/*.css
 src/components/search-list/search-list.scss
 src/components/sidebar/sidebar.scss
-src/components/experiment-tracking/time-series/time-series.scss
 src/components/node-list/styles/_row-toggle.scss
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
@@ -65,18 +65,14 @@ The `localStorage` state is updated automatically on every Redux store update, v
 
 ![Kedro-Viz data flow diagram](/.github/img/frontend-architecture.png)
 
-Kedro-Viz currently utilizes two different methods of data ingestion: the Redux setup for the pipeline and flowchart-view related components, and GraphQL via Apollo Client for the experiment tracking components.
+Kedro-Viz currently utilizes one method of data ingestion: the Redux setup for the pipeline and flowchart-view related components.
 
 On initialisation for the Redux setup, Kedro-Viz [manually normalises pipeline data](/src/store/normalize-data.js), in order to [make immutable state updates as performant as possible](https://redux.js.org/recipes/structuring-reducers/normalizing-state-shape).
 
 Next, it [initialises the Redux data store](https://github.com/kedro-org/kedro-viz/blob/main/src/store/initial-state.js), by merging this normalised state with other data sources such as saved user preferences from `localStorage`, URL flags, and default values.
 
 During preparation, the initial state is separated into two parts: pipeline and non-pipeline state. This is because the non-pipeline state should persist for the session duration, even if the pipeline state is reset/overwritten - i.e. if the user selects a new top-level pipeline.
 
-Kedro run data used for experiment tracking are stored in a SQLite database that is generated automatically once [experiment tracking is enabled in your Kedro project](https://kedro.readthedocs.io/en/stable/08_logging/02_experiment_tracking.html). By default, the session store database sits under the `/data` directory of your Kedro project as `session_store.db`. On loading Kedro-Viz from the Kedro project, the Kedro-Viz backend will consume the run data stored in the database and serve the data via the GraphQL endpoint via GraphQL query requests from the Apollo client on the front end.
-
-The server also allows updates to the database for certain fields of the run data (name, notes, etc.) via mutations.
-
 ## React components
 
 React components are all to be found in `/src/components/`. The top-level React component for the standalone app is `Container`, which includes some extra code (e.g. global styles and data loading) that aren't included in the component library. The entry-point component for the library (as set by the `main` property in package.json) is `App`.
@@ -87,7 +83,7 @@ The `App` component contains the [Redux store Provider](https://react-redux.js.o
 
 ## State management
 
-The following sections outline the Redux state management used to manage the data and app state for the flowchart view. The data for the runs view (experiment tracking features) are managed by React hooks and the Apollo GraphQL client.
+The following sections outline the Redux state management used to manage the data and app state for the flowchart view.
 
 ![Kedro-Viz app architecture](.github/img/app-architecture.png)
 
@@ -107,18 +103,6 @@ Selectors can be found in `/src/selectors/`. We use [Reselect](https://github.co
 
 We have used Kedro-Viz to visualize the selector dependency graph - [visit the demo to see it in action](https://demo.kedro.org/?data=selectors).
 
-## Apollo
-
-The `src/apollo` directory contains all the related setup for ingesting data from the GraphQL endpoint for the experiment tracking features. This includes the schema that defines all query and mutation types, the config that sets up the Apollo Client to be used within React components, and other files containing helper functions, such as mocks to generate random data for the mock server.
-
-The GraphQL schema is defined on the backend by Strawberry and automatically converted to GraphQL SDL (schema definition language) with `make schema-fix`. A CI check ensures that the resulting `schema.graphql` and below visualization are always in sync with the backend definition.
-
-![Kedro-Viz GraphQL schema](.github/img/schema.graphql.png)
-
-You can see documentation for the schema and run mock queries using GraphiQL, the GraphQL integrated development environment. This is possible without launching the full backend server: run `make strawberry-server` and then go to [http://127.0.0.1:8000/graphql](http://127.0.0.1:8000/graphql).
-
-⚠️ When a query supplies an ordered argument, the backend response must maintain the same ordering. For example, a the response to a query that calls for `runIds = [2, 3, 1]` should respond with runs in that same order.
-
 ## Utils
 
 The `/src/utils/` directory contains miscellaneous reusable utility functions.
@@ -153,4 +137,4 @@ The app uses [redux-watch](https://github.com/ExodusMovement/redux-watch) with a
 
 ![Kedro-Viz backend architecture](/.github/img/backend-architecture.png)
 
-The backend of Kedro-Viz serves as the data provider and API layer that interacts with Kedro projects and manages data access for visualisations in the frontend.  It offers both REST and GraphQL APIs to support data retrieval for the frontend, allowing access to pipeline structures, node-specific details, and experiment tracking data. Key components include the `DataAccessManager`, which interfaces with data `Repositories` to fetch and structure data. The CLI enables users launch with Kedro-Viz from the command line, while deploy and build options enables seamless sharing of pipeline visualisations on any static website hosting platform.
+The backend of Kedro-Viz serves as the data provider and API layer that interacts with Kedro projects and manages data access for visualisations in the frontend.  It offers REST API to support data retrieval for the frontend, allowing access to pipeline structures and node-specific details. Key components include the `DataAccessManager`, which interfaces with data `Repositories` to fetch and structure data. The CLI enables users launch with Kedro-Viz from the command line, while deploy and build options enables seamless sharing of pipeline visualisations on any static website hosting platform.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -210,23 +210,6 @@ make run PROJECT_PATH=<path-to-your-test-project>/new-kedro-project
 
 > **Note**: Once the backend development server is launched at port 4142, the local app will always pull data from that server. To prevent this, you can comment out the proxy setting in `package.json` and restart the dev server at port 4141.
 
-#### Launch the development server with the `SQLiteSessionStore`
-
-Kedro-Viz provides a `SQLiteSessionStore` that users can use in their project to enable experiment tracking functionality. If you want to use this session store with the development server, make sure you don't use a relative path when specifying the store's location in `settings.py`. For example, `demo-project` specifies the local `data` directory within a project as the session store's location as follows:
-
-```python
-from kedro_viz.integrations.kedro.sqlite_store import SQLiteStore
-SESSION_STORE_ARGS = {"path": str(Path(__file__).parents[2] / "data")}
-```
-
-Owing to this coupling between the project settings and Kedro-Viz, if you wish to execute any Kedro commands on `demo-project` (including `kedro run`), you will need to install the Kedro-Viz Python package. To install your local development version of the package, run:
-
-```bash
-pip3 install -e package
-```
-
-Since Kedro 0.18, a session can only contain one run. In Kedro-Viz, once a session has been retrieved from the store we always use the terminology "run" rather than "session", e.g. `run_id` rather than `session_id`.
-
 ## Testing guidelines
 
 - Scope out major journeys from acceptance criteria from the ticket for manual end-to-end testing

diff --git a/README.npm.md b/README.npm.md
@@ -27,7 +27,7 @@ Live Demo: <a href="https://demo.kedro.org/" target="_blank">https://demo.kedro.
 
 ## Introduction
 
-Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project.
+Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro).
 
 ## Features
 
@@ -39,7 +39,6 @@ Kedro-Viz is an interactive development tool for building data science pipelines
 - 📊 Rich metadata side panel to display parameters, plots, and more.
 - 📊 Supports all types of [Plotly charts](https://plotly.com/javascript/)
 - ♻️ Autoreload on code change
-- 🧪 Supports tracking and comparing runs in a Kedro project
 - 🎩 Many more to come
 
 ## Installation

diff --git a/RELEASE.md b/RELEASE.md
@@ -8,6 +8,7 @@ Please follow the established format:
 # Upcoming Release
 
 ## Major features and improvements
+- Remove experiment tracking. (#2237)
 
 ## Bug fixes and other changes
-Original file line number
+Diff line change
@@ Expand Up / @@ -65,3 +65,7 @@ coverage.xml @@
     # Kedro
     *.log
+    # Python
+    .venv/
+    mise.toml