VSCode Jupyter Notebooks (w venv)

VSCode Jupyter Notebooks (w/ venv)

If VSCode dies (AKA you can't connect to the remote ssh window) nuke it by first killing all processes:

lsof +D ~/.vscode-server | awk '{print $2}' | tail -n +2 | xargs -r kill -9

Then delete the directory from path:

cd ~
rm -rf ~/.vscode-server

Inspired by https://vikingdocs.york.ac.uk/applications/jupyter.html

First load venv:

source .venv/bin/activate

Install jupyter labs:

pip install jupyterlab

Create interactive partition:

srun --nodes=1 --ntasks=1 --cpus-per-task=5 --time=08:00:00 --pty /bin/bash

Run jupyter notebooks with no browser:

jupyter lab --no-browser

[!NOTE] Example output

[I 2024-08-21 14:16:53.564 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-08-21 14:16:53.568 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-08-21 14:16:53.572 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-08-21 14:16:53.573 ServerApp] Writing Jupyter server cookie secret to /users/bmp535/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-08-21 14:16:53.879 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-08-21 14:16:53.986 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-08-21 14:16:53.988 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-08-21 14:16:53.989 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-08-21 14:16:53.992 LabApp] JupyterLab extension loaded from /mnt/scratch/users/bmp535/.venv/lib/python3.11/site-packages/jupyterlab
[I 2024-08-21 14:16:53.992 LabApp] JupyterLab application directory is /mnt/scratch/users/bmp535/.venv/share/jupyter/lab
[I 2024-08-21 14:16:53.992 LabApp] Extension Manager is 'pypi'.
[I 2024-08-21 14:16:54.000 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-08-21 14:16:54.001 ServerApp] Serving notebooks from local directory: /mnt/scratch/users/bmp535/epoch_runner
[I 2024-08-21 14:16:54.001 ServerApp] Jupyter Server 2.14.2 is running at:
[I 2024-08-21 14:16:54.001 ServerApp] <http://localhost:8888/lab?token=2e1d6ee53c730e0906863c805f00a53fd43728cbe13f186e>
[I 2024-08-21 14:16:54.001 ServerApp]     <http://127.0.0.1:8888/lab?token=2e1d6ee53c730e0906863c805f00a53fd43728cbe13f186e>
[I 2024-08-21 14:16:54.001 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

From a new ssh window on a login node run the following, make sure to change the node to point to the one created for the interactive partition:

 ssh -N -L 8889:localhost:8888 node001

Open the link from the notebook making sure to change the 8888 to 8889:

http://127.0.0.1:8889/lab?token=2e1d6ee53c730e0906863c805f00a53fd43728cbe13f186e

If you're having troubles locating the port run the following command to see all running ports:

ps -fu $USER | grep "ssh -N -L" | grep -v grep

Finally kill all the ports using the command:

kill $(ps -fu $USER | grep "ssh -N -L" | grep -v grep | awk '{print $2}')