-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building in docker #40
Changes from 17 commits
80f5e99
2f6d85d
2b94345
941445e
eb05461
23333d4
91d2cc9
047abb8
2f00224
77d21cf
c6c9396
af020fc
e7f5992
65deef8
68815df
fc03a26
8763a65
7f1eb1c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# tagged version, not latest! | ||
FROM rocker/geospatial:3.5.2 | ||
|
||
# install node and npm (see https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions) | ||
RUN sudo apt-get install -y curl &&\ | ||
sudo apt-get install -y gnupg &&\ | ||
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - &&\ | ||
sudo apt-get update &&\ | ||
sudo apt-get install -y nodejs &&\ | ||
sudo apt-get install -y build-essential | ||
|
||
RUN sudo npm install -g\ | ||
webpack\ | ||
webpack-cli\ | ||
d3-geo-projection | ||
|
||
#bring in DOI root cert. Remove this statement for non-USGS persons | ||
RUN /usr/bin/wget -O /usr/lib/ssl/certs/DOIRootCA.crt http://sslhelp.doi.net/docs/DOIRootCA2.cer && \ | ||
ln -sf /usr/lib/ssl/certs/DOIRootCA.crt /usr/lib/ssl/certs/`openssl x509 -hash -noout -in /usr/lib/ssl/certs/DOIRootCA.crt`.0 && \ | ||
echo "\\n\\nca-certificate = /usr/lib/ssl/certs/DOIRootCA.crt" >> /etc/wgetrc; | ||
|
||
RUN Rscript -e 'installed.packages()' | ||
#Note that version rocker images are already set up to use the MRAN mirror corresponding to the | ||
#date of the R version, so package dates are already set (unless forcing another repo) | ||
RUN Rscript -e 'devtools::install_github("richfitz/remake")' && \ | ||
Rscript -e 'install.packages("grithub", repos = c(getOption("repos"), "https://owi.usgs.gov/R"))' && \ | ||
Rscript -e 'devtools::install_github("USGS-VIZLAB/[email protected]")' | ||
#note that most common packages will already be installed as part of the geospatial image | ||
RUN install2.r --error \ | ||
sbtools \ | ||
geojsonio \ | ||
dataRetrieval\ | ||
bit64 | ||
|
||
RUN mkdir /home/rstudio/gage-conditions | ||
RUN chown rstudio /home/rstudio/gage-conditions | ||
WORKDIR /home/rstudio/gage-conditions |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
pipeline { | ||
agent none | ||
stages { | ||
stage('Checkout repo') { | ||
agent any | ||
steps { | ||
sh 'wget -O DOIRootCA2.cer http://sslhelp.doi.net/docs/DOIRootCA2.cer' | ||
git "https://github.com/wdwatkins/gage-conditions" | ||
} | ||
} | ||
stage('build viz') { | ||
agent { | ||
dockerfile { | ||
args '-v ${WORKSPACE}:/home/rstudio/gage-conditions' | ||
} | ||
} | ||
steps { | ||
sh 'mkdir -p target' | ||
sh ''' | ||
Rscript -e 'vizlab::vizmake("dv_sites_sp")' | ||
Rscript -e "remake::make(remake_file = 'map_preprocess.yml')" | ||
Rscript -e "vizlab::vizmake()" | ||
''' | ||
} | ||
} | ||
stage('push to S3') { | ||
agent any | ||
steps { | ||
sh 'aws s3 sync ./target/ s3://dev-owi.usgs.gov/vizlab/water-use-15/ --exclude "*.svg" --exclude "*.json"; \ | ||
aws s3 sync ./target/ s3://dev-owi.usgs.gov/vizlab/water-use-15/ --exclude "*" --include "*.svg" --content-type "image/svg+xml"; \ | ||
aws s3 sync ./target/ s3://dev-owi.usgs.gov/vizlab/water-use-15/ --exclude "*" --include "*.json" --content-type "application/json"' | ||
} | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
version: "3" | ||
services: | ||
docker-dev-mode: | ||
image: gage-conditions | ||
build: | ||
context: . | ||
dockerfile: Dockerfile | ||
ports: | ||
- "8787:8787" | ||
volumes: | ||
- gage-conditions-data:/home/rstudio/gage-conditions | ||
environment: | ||
- ROOT=TRUE | ||
- PASSWORD=mypass | ||
|
||
volumes: | ||
gage-conditions-data: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,7 +12,7 @@ fetch.dv_sites <- function(viz){ | |
year <- deps[["year"]][["year"]] | ||
|
||
startDate <- paste0(year, "-01-01") | ||
endDate <- paste0(year, "-12-31") | ||
endDate <- paste0(year, "-01-02") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. revert this change? |
||
hucs <- zeroPad(1:21, 2) | ||
|
||
sites <- c() | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,7 +18,7 @@ info: | |
version: 0.9-7 | ||
dataRetrieval: | ||
repo: CRAN | ||
version: 2.7.4 | ||
version: 2.7.3 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. does the older version work better? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It was just to avoid having to set package installs to GRAN, since CRAN only has 2.7.3 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. oh, OK |
||
dplyr: | ||
repo: CRAN | ||
version: 0.7.4 | ||
|
@@ -28,6 +28,12 @@ info: | |
sp: | ||
repo: CRAN | ||
version: 1.2.7 | ||
sbtools: | ||
repo: CRAN | ||
version: 1.1.6 | ||
geojsonio: | ||
repo: CRAN | ||
version: 0.6.0 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It'd be nice to think about removing all of these declared package dependencies as we switch to Docker (because now the Dockerfile is where the decisions actually get made about which package version gets used). Is now too soon to do so, and if not, is it as simple as deleting all the lines in this subsection, or does vizlab give errors if you do that? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We just would need to remove the There is probably a fair bit of code associated with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, thanks for thinking it through. I'll create a GitHub issue in vizlab for this, and we can resolve it (including cleaning up the extra code) after we've had a chance to kick the tires on the Docker option for a while. What happens if you make a packages YAML block but leave it empty? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added this issue: USGS-VIZLAB/vizlab#395 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That just returns a null from |
||
contributors: | ||
- | ||
name: Lindsay Carr | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Safe to revert this change now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, although really I'd say these dates should all be moved into the viz.yaml so they can be set in a single place
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair. Up to you how you do it, but note that we're not currently sure that anybody will be developing this interactive viz any more this year, so code quality/maintainability matters somewhat less than usual.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, probably wouldn't bother doing it now