Proposed scivision workflow for UI users with minimal coding exp.

[edwardchalstrey1]

After #246 is merged, we will have a scivision UI that new users can look at to find models or datasets from the catalog that can be used together.

If you are a new user of scivision getting started with scivision, the primary way of working could be as follows:

1. Use the UI to find the model and dataset that you want to use
2. At best, you can then open Python and simply run something like:
   • pip install [model package name] (or similar)
   • data = load_dataset(<dataset url>)
   • model = load_pretrained_model(<model url>)
   • model.predict(data)

(perhaps the UI should even spit out these lines of code for you to copy?)

However, some models when selected in the UI should show a warning like "This model has additional dependencies that must be installed". The user would be directed to the model repo README which includes the dependency commands (e.g. activate a certain Conda env) to run before the model can be used (or maybe we scrape the README somehow and spit that out too?).

All models included in the catalog will definitely include a README with clear instructions, because scivision maintainers will not accept any model repos for inclusion in the catalog that do not have this.

I believe this solves the conundrum presented in #159

[scotthosking]

Could we offer a link to spin-up a bespoke binder instance from the UI including the first few lines to load the user's chosen data and model? I would only suggest doing this where dependencies are not a major issue

[edwardchalstrey1]

We could investigate this - the nice thing here could be that we could configure it to have scivision and other packages like pandas etc ready to go, so no install steps.

Perhaps if it's easier and less slow another option would be a button to download a .py or .ipynb with pre-populated lines of code - though the user would at minimum still need a Python 3 installation and to run pip install scivision (and jupyter) in that case.

[edwardchalstrey1]

based on Rob's experience in #315 - I think we should be very hesitant to make people's first experience of scivision be Binder - too likely to cause problems.

[ots22]

(Copying reply on Slack)

This is closely related to the ongoing discussions about how to handle the python package management situation (and the lack of a reliable solution for external packages/system dependencies). I really like this suggestion, because it is a concrete plan that takes a clear stance on the issue (in this case, that handling external packages is beyond the scope of Scivision).

There are other possibilities too, but the above seems the simplest thing we could do, so perhaps we could start with it?
It would mean automated installation of packages (allow_install=True -- if we keep that) would sometimes refuse, ideally linking to some instructions when it does (which is a step up from seemingly working only to fail at runtime).

It does put some additional burden on a user (particularly a new user who might not be familiar with the landscape), so we might want to revisit some other ideas if we don’t like that, but it is something self-consistent and workable!

@edwardchalstrey1, how much does the above depend on there being a frontend? Could we do the same with the programmatic/python interface as it currently is?

[edwardchalstrey1]

I think the thing that is nice about starting from the Python interface is that you can search for a model, then load it straight away. Perhaps when you search for a model this way and get a result of a model that cannot be easily installed with pip (via load_pretrained_model) we should also print a warning that says something like "Warning: this model requires some additional dependencies that may require you to quit Python and install first... consult the README..." or similar

this way I think we get the best of both worlds - super easy pip installable models and slightly more complex dependency (but well documented) models can both be included in the catalog

[edwardchalstrey1]

Extra ideas from co-working 12.10.22

The UI could auto-generate a "model card" like in huggingface, but for a model-dataset selection, rather than just for a model. It would include both the model (or data plugin) install instructions (from their READMEs, see above) and also the scivision lines of code needed for loading both the data and model and running a prediction. This "model card" would therefore be split into "Installation" which you run in the terminal and "How to use" where it shows the Python lines.

There could also be a download .py or .ipynb for the Python parts. And perhaps even open that in Binder.