env updates for fall2024; change course number and mem reqs

[profjsb]

• In common.yaml:

  • change the course number for Fall 2024 (AY128)
  • downgrade the memory requirement (will need to bump it up again at the end of the semester)

• update the conda environment, pushing to python 3.11 and dropping support for keras

• add in a few packages needed for the AY128/256 course

[profjsb]

@shaneknapp this should be good to go, to try out in staging. Please let me know if you have any questions.

[shaneknapp]

@profjsb this should be ready for testing on astro-staging in a few minutes!

[profjsb]

@shaneknapp looks like the staging server does not spawn correctly:

2024-08-20T00:21:49Z [Warning] Back-off restarting failed container notebook in pod jupyter-joshbloom_astro-staging(258d98cc-c165-473b-9f93-05d42dafed77)
Spawn failed: Server at http://10.28.1.6:8888/user/joshbloom/ didn't respond in 30 seconds

are we missing some dependencies? I cannot see the logs to help debug.

[shaneknapp]

hmm not sure what's going on... i wasn't able to launch my server either but will debug tomorrow morning.

…

On Mon, Aug 19, 2024 at 5:25 PM Joshua Bloom ***@***.***> wrote: @shaneknapp <https://github.com/shaneknapp> looks like the staging server does not spawn correctly: 2024-08-20T00:21:49Z [Warning] Back-off restarting failed container notebook in pod jupyter-joshbloom_astro-staging(258d98cc-c165-473b-9f93-05d42dafed77) Spawn failed: Server at http://10.28.1.6:8888/user/joshbloom/ didn't respond in 30 seconds are we missing some dependencies? I cannot see the logs to help debug. — Reply to this email directly, view it on GitHub <#5971 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMIHLHKUSI2MD7CKIOFZDDZSKEHNAVCNFSM6AAAAABMYR7KKOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJXG42DEMBYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[shaneknapp]

ok, so i ran repo2docker locally on my workstation, and at the very end of the image build it errors out w/the following message:

Step 55/55 : CMD ["jupyter", "notebook", "--ip", "0.0.0.0"]
 ---> Running in c11f09242ed3
Removing intermediate container c11f09242ed3
 ---> e68de72bbf5e
{"aux": {"ID": "sha256:e68de72bbf5ec8208626295fded333fce4a7f48583382ba73b52bae86141ce36"}}Successfully built e68de72bbf5e
Successfully tagged r2d-2e1724171162:latest
/srv/conda/envs/notebook/lib/python3.11/subprocess.py:1016: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdout = io.open(c2pread, 'rb', bufsize)
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/jupyter_server/services/sessions/sessionmanager.py", line 14, in <module>
    import sqlite3
  File "/srv/conda/envs/notebook/lib/python3.11/sqlite3/__init__.py", line 57, in <module>
    from sqlite3.dbapi2 import *
  File "/srv/conda/envs/notebook/lib/python3.11/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: /srv/conda/envs/notebook/lib/python3.11/lib-dynload/_sqlite3.cpython-311-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/conda/envs/notebook/bin/jupyter-notebook", line 5, in <module>
    from notebook.app import main
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/notebook/app.py", line 17, in <module>
    from jupyter_server.serverapp import flags
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/jupyter_server/serverapp.py", line 106, in <module>
    from jupyter_server.gateway.managers import (
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/jupyter_server/gateway/managers.py", line 32, in <module>
    from ..services.sessions.sessionmanager import SessionManager
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/jupyter_server/services/sessions/sessionmanager.py", line 17, in <module>
    from pysqlite2 import dbapi2 as sqlite3  # type:ignore[no-redef]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'pysqlite2'
CONTAINER FINISHED RUNNING.

i added pysqlite2 to environment.yaml and am re-running the build now.

[shaneknapp]

ok, it's still building locally but failing to launch. i'll work on this a bit more today and see if i can't figure out what's going on.

also, i would strongly recommend installing jupyter-repo2docker and using that to build/launch notebooks on your device before creating a PR and using CI/CD to test, as the latter method will not catch issues like this.

[ryanlovett]

@shaneknapp That seems like a conda packaging bug, if jupyter_server needed pysqlite2 but it didn't get pulled in.

[shaneknapp]

@shaneknapp That seems like a conda packaging bug, if jupyter_server needed pysqlite2 but it didn't get pulled in.

yeah, i agree. but i did fix it by explicitly installing sqlite...

#5979