Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove pandas and plotly dependency #999

Closed
antonymilne opened this issue Aug 2, 2022 · 5 comments · Fixed by #1268
Closed

Remove pandas and plotly dependency #999

antonymilne opened this issue Aug 2, 2022 · 5 comments · Fixed by #1268
Assignees
Labels
Python Pull requests that update Python code

Comments

@antonymilne
Copy link
Contributor

antonymilne commented Aug 2, 2022

Note. Probably it's only worth doing this once the scheme for making plots to compare metrics has become clear, just in case that ends up requiring plotly and/or pandas on the backend.

Currently kedro-viz has these requirements which ideally shouldn't be necessary:

plotly>=4.0
pandas>=0.24

These are only used in flowchart.DataNodeMetadata.create_metrics_plot for generating the metrics plot in the metadata panel. The reason they shouldn't be required is that they are used for very simple data wrangling, which we should be able to handle without much obfuscation by using purely Python dictionary ➡️ reshape as necessary ➡️ pure json (should not even need any particular plotly syntax) ➡️ make plot on frontend.

Warning. Make sure that kedro-viz still works on projects without any plotly/metrics datasets without these dependencies (both flowchart and experiment tracking screens). The imports should not break.

@antonymilne antonymilne added the Python Pull requests that update Python code label Aug 2, 2022
@tynandebold tynandebold moved this to Inbox in Kedro-Viz Aug 15, 2022
@tynandebold tynandebold moved this from Inbox to Backlog in Kedro-Viz Aug 22, 2022
@tynandebold
Copy link
Member

More context: if we implement #907 then Pandas may become a dependency we need to keep around.

@antonymilne
Copy link
Contributor Author

@tynandebold Just to emphasise the may on the above point - this depends very much on where the preview implementation ends up (kedro-datasets side vs. kedro-viz side). If we do end up needing pandas for this on kedro-viz then it should probably be an optional dependency like pip install kedro-viz[pandas] anyway. So whatever happens I think we should go ahead and remove the pandas dependency for now anyway.

@tynandebold
Copy link
Member

Makes sense. Should we remove Pandas as well as Plotly now then? Or wait until we implement comparing metrics, as you said here?

Note. Probably it's only worth doing this once the scheme for making plots to compare metrics has become clear, just in case that ends up requiring plotly and/or pandas on the backend.

@antonymilne
Copy link
Contributor Author

On second thoughts I don't think we need to wait for that actually, because we can always add the plotly/pandas dependency back in if we do require it. And I think we should work hard to try to avoid requiring it (should be possible).

What this does depend on though is what we do with #1000:

  • if we remove this plot then we can remove the dependencies right away
  • if we want to keep it just in the short term then probably just leave as is but maybe fix the bugs in it
  • if we want to keep it longer term then we should fix the bugs and rewrite so it doesn't use these dependencies and then remove them

Overall I'm inclined to go for the 1st option.

@rashidakanchwala
Copy link
Contributor

rashidakanchwala commented Jan 30, 2023

Yes, I think we can remove the dependencies if we show the plot in d3 as discussed here #1000

We might need to add it back when we work on dataset preview (as we will mostly be doing the reading in Kedro-viz) but if we are going to save all the rows in a DB then we might not even need it. So it makes sense to remove it for now.

@tynandebold tynandebold moved this from Backlog to Todo in Kedro-Viz Feb 21, 2023
@rashidakanchwala rashidakanchwala moved this from Todo to In Progress in Kedro-Viz Feb 22, 2023
@rashidakanchwala rashidakanchwala self-assigned this Feb 22, 2023
@rashidakanchwala rashidakanchwala moved this from In Progress to In Review in Kedro-Viz Feb 28, 2023
@github-project-automation github-project-automation bot moved this from In Review to Done in Kedro-Viz Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Python Pull requests that update Python code
Projects
Status: Done
3 participants