Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DH-303] Set up dev environment for discovery interns to play with dashboards #5783

Merged
merged 4 commits into from
Jun 8, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions deployments/dev/config/common.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ jupyterhub:
# this role will be assigned to...
groups:
- course::1524699::group::all-admins

## Course NUM, Spring 2024, #xyz
#course-staff-N:
# description: Enable course staff to view and access servers.
Expand All @@ -66,8 +67,8 @@ jupyterhub:
- display_name: "Dockerfile image"
description: "This is the original dev image."
default: true
- display_name: "repo2docker image"
description: "A newer repo2docker-based image with similar components as the primary."
- display_name: "Dashboard image"
description: "A newer repo2docker-based image with a primary focus of building dashboards."
kubespawner_override:
image: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-secondary:1a64e0a
- display_name: "1524699: DataHub Infrastructure"
Expand Down Expand Up @@ -113,7 +114,16 @@ jupyterhub:
# admin: true
# mem_limit: 2096M
# mem_guarantee: 2048M
#

## Datahub Discovery Program, Summer 2024, #DH-303
course::1524699::group::datahub-discovery-su24-interns:
shaneknapp marked this conversation as resolved.
Show resolved Hide resolved
mem_limit: 12288M #12 GB RAM should be enough for parsing data and generating dashboards
mem_guarantee: 12288M
extraVolumeMounts:
- name: home
mountPath: /home/jovyan/discovery-su24-dataset
subPath: _shared/discovery-su24-dataset
readOnly: true
#
# # Example: a fully specified CanvasOAuthenticator group name.
# # This could be useful for temporary resource bumps where the
Expand Down
82 changes: 65 additions & 17 deletions deployments/dev/images/secondary/environment.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,68 @@
dependencies:
- python=3.11.*
- pip=23.2.*
- jupyter-server-proxy==4.1.2
- jupyter-rsession-proxy==2.2.0
- jupyterlab-myst==2.0.2
- syncthing==1.25.0
- pyppeteer==1.0.2

# for nbconvert
- pandoc==3.1.3
name: dashboard-image

# for jupyter-tree-download
- zip==3.0
channels:
- conda-forge
- pytorch

# bug w/notebook and traitlets: https://github.com/jupyter/notebook/issues/7048
- traitlets=5.9.*
dependencies:
- python==3.11.*
- git==2.39.1
- jupyter-resource-usage==1.0.0
- jupyterlab==4.0.11
- jupyterlab-favorites==3.0.0
- jupyterlab_server==2.23.0
- jupyterlab_widgets==3.0.8
- jupyter_server==2.7.0
- nbgitpuller==1.2.1
- notebook==7.0.7
- folium==0.14.0
- h5netcdf==1.0.2
- ipywidgets==8.0.7
- jupysql==0.8.0
- jupyter-archive==3.4.0
- matplotlib==3.7.1
- mdit-py-plugins==0.4.0
- numpy==1.24.2
- pandas==2.0.2
- plotly==5.13.1
- requests==2.28.2
- scikit-image==0.19.3
- scikit-learn==1.2.2
- scipy==1.10.1
- seaborn==0.12.2
- statsmodels==0.14.0
- tensorflow-cpu==2.12.1
- sqlalchemy==2.0.16
- mlxtend==0.23.0
# Spring 2024 data 100
- pytorch==2.1.2
- cpuonly==2.0
- transformers==4.37.1
# Spring 2024 table demos
- lxml==5.1.0
# Spring 2024 Econ 148 Packages
- geopandas==0.14.2
- geopy==2.4.1
- lifelines==0.27.8
- pycountry==22.3.5
# Install voila for generating dashboards
- voila==0.5.7
# Install myst for generating dashboards
- mystmd==1.2.5
- jupyterlab-myst==2.4.2
- pip
- pip:
- -r infra-requirements.txt
- jupyter-shiny-proxy==1.1
# - -r infra-requirements.txt
- ipywidgets==8.0.7
# disable until fixed (probably this: https://github.com/jupyterlab/jupyter-collaboration/issues/162)
balajialg marked this conversation as resolved.
Show resolved Hide resolved
# - jupyter_collaboration==1.0.1
- jupyterhub==4.1.5
- nbconvert[webpdf]
# - pyppeteer==2.0.0
- pytest-notebook==0.8.1
- gh-scoped-creds==4.1
- git+https://github.com/shaneknapp/python-popularity-contest.git@add-error-handling
- ydata-profiling==4.6.4
- otter-grader==5.4.0
- duckdb==0.10.1
- duckdb_engine==0.11.2
Loading