Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DH-301] prep logodev for new image repo #5778

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,15 @@ workflows:
ignore:
- staging
- prod
- hubploy/build-image:
deployment: logodev
name: logodev hub image build
# Filters can only be per-job? wtf
filters:
branches:
ignore:
- staging
- prod
- hubploy/build-image:
deployment: publichealth
name: publichealth image build
Expand Down
7 changes: 5 additions & 2 deletions deployments/logodev/hubploy.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
images:
images:
- name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/primary-user-image
path: ../datahub/images/default
# temporary update
- name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image
path: image/
repo2docker:
base_image: docker.io/library/buildpack-deps:jammy
registry:
provider: gcloud
gcloud:
Expand Down
93 changes: 93 additions & 0 deletions deployments/logodev/image/apt.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Some linux packages for basic terminal work, particularly
# oriented at users new to Unix/cmd line environments.

# installing less as more just isn't enough
less

# Basic unix tools
man
man-db
manpages-posix
manpages-dev
manpages-posix-dev

# Download tools
curl
wget

# Core text editors on a *nix box: vim
vim

# A couple of CLI editors that are easier than vim
# micro # currently not working on 18.04
nano
jed
jed-extra

# powerful terminal-based file manager, better than the one in JLab
mc

# for easily managing multiple repositories with one command (perl-doc
# is needed for its help pages to work)
mr
perl-doc

# Regular build tools for compiling common stuff
build-essential
gfortran

# Dependencies for nbconvert
texlive-xetex
texlive-fonts-recommended
texlive-plain-generic
# https://github.com/berkeley-dsep-infra/datahub/issues/3719
texlive-lang-chinese
lmodern

# Other useful document-related tools
pandoc
latexdiff

# Some useful git utilities use basic Ruby
ruby

# Other niceties for command-line work and life
ack # powerful grep-like tool
pydf # colorized disk usage
tmux
screen
htop
nnn # cmd line file manager
zsh
rsync
tig # console UI for git
multitail

# For later, these are not available in 18.04
#browsh # text-based web browser, occasionally handy
#dasel # json/yml/csv/etc data wrangling at the terminal
#fzf # fuzzy file finder

## This section adds tools for desktop environment usage
dbus-x11
xorg
xubuntu-icon-theme
xfce4
xfce4-goodies
xclip
xsel
firefox
chromium-browser

# GUI text editors
vim-gtk3
gedit

# Git clients and tools
git-gui
gitg
qgit
meld

# For jupyter-tree-download. Ref: https://github.com/berkeley-dsep-infra/datahub/issues/3979
zip
63 changes: 63 additions & 0 deletions deployments/logodev/image/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: data100-FA23

channels:
- conda-forge
- pytorch

dependencies:
- python==3.11.*
- git==2.39.1
- jupyter-resource-usage==1.0.0
- jupyterlab==4.0.11
- jupyterlab-favorites==3.0.0
- jupyterlab_server==2.23.0
- jupyterlab_widgets==3.0.8
- jupyter_server==2.7.0
- nbgitpuller==1.2.1
- notebook==7.0.7
- folium==0.14.0
- h5netcdf==1.0.2
- ipywidgets==8.0.7
- jupysql==0.8.0
- jupyter-archive==3.4.0
- matplotlib==3.7.1
- mdit-py-plugins==0.4.0
- numpy==1.24.2
- pandas==2.2.2
- plotly==5.13.1
- requests==2.28.2
- scikit-image==0.19.3
- scikit-learn==1.2.2
- scipy==1.10.1
- seaborn==0.12.2
- statsmodels==0.14.0
- tensorflow-cpu==2.12.1
- sqlalchemy==2.0.16
- mlxtend==0.23.0
# Spring 2024 data 100
- pytorch==2.1.2
- cpuonly==2.0
- transformers==4.37.1
# Spring 2024 table demos
- lxml==5.1.0
# Spring 2024 Econ 148 Packages
- geopandas==0.14.2
- geopy==2.4.1
- lifelines==0.27.8
- pycountry==22.3.5
- pip
- pip:
# - -r infra-requirements.txt
- ipywidgets==8.0.7
# disable until fixed (probably this: https://github.com/jupyterlab/jupyter-collaboration/issues/162)
# - jupyter_collaboration==1.0.1
- jupyterhub==4.1.5
- nbconvert[webpdf]
# - pyppeteer==2.0.0
- pytest-notebook==0.8.1
- gh-scoped-creds==4.1
- git+https://github.com/shaneknapp/python-popularity-contest.git@add-error-handling
- ydata-profiling==4.6.4
- otter-grader==5.4.0
- duckdb==0.10.1
- duckdb_engine==0.11.2
29 changes: 29 additions & 0 deletions deployments/logodev/image/infra-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# WARNING: Original source at scripts/infra-packages/requirements.txt
# PLEASE DO NOT EDIT ELSEWHERE
# After editing scripts/infra-packages/requirements.txt, please run
# scripts/infra-packages/sync.bash.

# This file pins versions of notebook related python packages we want
# across all hubs. This makes sure we don't need to upgrade them
# everwhere one by one.

# FIXME: Freeze this to get exact versions of all dependencies
notebook==7.0.7
jupyterlab==4.0.11
nbgitpuller==1.2.1
jupyter-resource-usage==1.0.1
# Matches version in images/hub/Dockerfile
jupyterhub==4.1.5
appmode==0.8.0
ipywidgets==8.0.7
jupyter-tree-download==1.0.1
git-credential-helpers==0.2
# Measure popularity of different packages in our hubs
# https://discourse.jupyter.org/t/request-for-implementation-instrument-libraries-actively-used-by-users-on-a-jupyterhub/7994?u=yuvipanda
git+https://github.com/shaneknapp/python-popularity-contest.git@add-error-handling
# RISE is useful for presentations - see https://github.com/berkeley-dsep-infra/datahub/issues/2527
RISE==5.7.1
# syncthing for dropbox-like functionality
jupyter-syncthing-proxy==1.0.3
# jupyter archival tool for easy user downloads
jupyter-archive==3.4.0
6 changes: 6 additions & 0 deletions deployments/logodev/image/postBuild
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash

# installing chromium browser to enable webpdf conversion using nbconvert
export PLAYWRIGHT_BROWSERS_PATH=${CONDA_DIR}
npm install -g playwright
playwright install chromium
Loading