Tracking issue for adding interactive documentation

[agriyakhetarpal]

Description

This is an issue to collect tasks for adding interactive documentation to PyWavelets via JupyterLite kernels/notebooks.

Proposal

• Build against Pyodide out-of-tree (Build and test PyWavelets Pyodide wheels in CI #701)
• Migrate to pydata-sphinx-theme (DOC: Migrate to pydata-sphinx-theme #704)
• Investigate where to upload wasm32 wheels (needs an external index that accepts emscripten_X_Y_wasm32 platform tags)
• Add CI jobs to run nightly builds of Pyodide wheels (or on every push to master, rather?), while configuring versioning to include git hashes or dates (Upload dev builds for all supported platforms to anaconda.org #712, Upload dev wheels to Anaconda.org + revamp wheels publishing workflow #714)
• Convert all current usage examples to literate-programming-style notebooks, adding instructions and resolving issues as necessary
  • API reference section (Initial steps towards interactive documentation via JupyterLite #728)
  • Usage examples section (DOC: Add interactive notebooks to pages in the "Usage Examples" section #741)
• Add jupyterlite-sphinx and buttons based on screen size¹ to provide interactive kernels for these examples, and
• Check if we can download wheels from the external index on Read the Docs or if an alternative step is needed
  • Wheels cannot be pre-loaded with jupyterlite-pyodide-kernel currently, so the global warning will be used for this purpose
  • Look to see a resolution to CORS headers issue from Anaconda (micropip.install("<...>") does not uphold custom PyPI indices pyodide/micropip#101)
• Enable swapping of wheels so that the latest built ones are picked up by jupyterlite-pyodide-kernel or any other Pyodide interpreter that powers the documentation, always.

Additional comments

The intermediate step would mean that the .whl files would have to be downloaded manually using wget or curl first (a helper script can be created for this) and pointed to the Pyodide-pip interpreter for installation.

Footnotes

1. https://github.com/scipy/scipy/pull/20019/ ↩

[rgommers]

Add CI jobs to run nightly builds of Pyodide wheels (or on every push to master, rather?),

Every push to master would be better; this package moves very slowly so more than that is a bit wasteful.

while configuring versioning to include git hashes or dates

That may be simplified by this just-merged Meson feature: mesonbuild/meson@524a661. It'll only be available once 1.4.0 comes out. You could already try locally though whether you can make that work. Note that it would need a .py script rather than the version.sh in the docs, otherwise things won't work on Windows.

[agriyakhetarpal]

Thanks, a 1.4.0 non-RC release for Meson should be out in a week or so when we should be able to do this. A .py script would work nicely too.

I am more concerned about how we would make Read the Docs work with what we are intending to do – I suppose only the process of the implementation can tell what things end up being feasible.

[rgommers]

It's purely client-side, so probably fine on RTD. Switching to GitHub Pages would also be pretty straightforward if needed.

[rgommers]

Investigate where to upload wasm32 wheels (needs an external index that accepts emscripten_X_Y_wasm32 platform tags)

I opened scientific-python/upload-nightly-action#75 to request access to upload to https://anaconda.org/scientific-python-nightly-wheels/repo.

[rgommers]

Add CI jobs to run nightly builds of Pyodide wheels (or on every push to master, rather?)

I'll note that that anaconda.org bucket cleans up nightlies older than 30 days. So how about on every push to master and every 25 days, to be safe?

[agriyakhetarpal]

I'll note that that anaconda.org bucket cleans up nightlies older than 30 days. So how about on every push to master and every 25 days, to be safe?

Yes, that should be fair enough – we would want at least one wheel being uploaded before other ones get cleaned up

[agriyakhetarpal]

Next step is actually getting started with interactive documentation. We already have the usage examples – how should we go ahead? Should I convert them to notebooks as I mentioned above?

[rgommers]

Guess we're getting closer, 4 out of 7 done. I'm starting to prepare the 1.6.0 release. I think we can probably get the "Try Examples" in before that is final (in 2 weeks or so).

[rgommers]

We already have the usage examples – how should we go ahead? Should I convert them to notebooks as I mentioned above?

It is not necessary to convert them to notebooks. The jupyterlite-sphinx extension should be able to work with any regular examples in docstrings. I suggest adding that extension first, seeing what works with it and what doesn't using the PyWavelets 1.5.0 shipped in Pyodide, and then fixing things up as needed.

[agriyakhetarpal]

Guess we're getting closer, 4 out of 7 done. I'm starting to prepare the 1.6.0 release. I think we can probably get the "Try Examples" in before that is final (in 2 weeks or so).

Just adding a comment here to update things: I assume this has been checked out and #731 should now bring v1.6.0 with some new examples. The Emscripten and Pyodide versions are not decoupled at the time of writing, so we won't have v1.6.0 WASM wheels until there is a new version of Pyodide, but it should be okay since there have been no previous API changes as discussed earlier. I have updated the checklist to showcase the most recent scenario. The next task to check off would be to keep investigating the CORS headers issue. We recently merged jupyterlite/jupyterlite-sphinx#161, and a release for jupyterlite-sphinx should be coming soon – after that, we can remove the :button_text: input from all examples (which shall fix the issue we had faced, mentioned in the PR linked above).

[agriyakhetarpal]

From #737 (comment), the "Usage Examples" section is now being worked on; it can be targeted for a v1.6.1 milestone/release.

[agriyakhetarpal]

Note to self: jupyterlite/jupyterlite#461 might be useful; I hope it is added to the Pyodide kernel. It might be possible to write a custom JupyterLite kernel for this such as https://github.com/jobovy/jupyterlite-repl-prerun.