Skip to content

Commit

Permalink
updated spark version to 3.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
sllynn committed Nov 19, 2024
1 parent 6e5bd65 commit 6b049f8
Show file tree
Hide file tree
Showing 8 changed files with 12 additions and 12 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build_main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
python: [ 3.10.12 ]
numpy: [ 1.22.4 ]
gdal: [ 3.4.1 ]
spark: [ 3.4.1 ]
spark: [ 3.5.0 ]
R: [ 4.2.2 ]
steps:
- name: checkout code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
python: [ 3.10.12 ]
numpy: [ 1.22.4 ]
gdal: [ 3.4.1 ]
spark: [ 3.4.1 ]
spark: [ 3.5.0 ]
R: [ 4.2.2 ]
steps:
- name: checkout code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_r.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
python: [ 3.10.12 ]
numpy: [ 1.22.4 ]
gdal: [ 3.4.1 ]
spark: [ 3.4.1 ]
spark: [ 3.5.0 ]
R: [ 4.2.2 ]
steps:
- name: checkout code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_scala.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
python: [ 3.10.12 ]
numpy: [ 1.22.4 ]
gdal: [ 3.4.1 ]
spark: [ 3.4.1 ]
spark: [ 3.5.0 ]
R: [ 4.2.2 ]
steps:
- name: checkout code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
python: [ 3.10.12 ]
numpy: [ 1.22.4 ]
gdal: [ 3.4.1 ]
spark: [ 3.4.1 ]
spark: [ 3.5.0 ]
R: [ 4.2.2 ]
steps:
- name: checkout code
Expand Down
4 changes: 2 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@
<properties>
<scala.version>2.12.10</scala.version>
<scala.compat.version>2.12</scala.compat.version>
<spark.version>3.4.1</spark.version>
<spark.version>3.5.0</spark.version>
<mosaic.version>0.4.4</mosaic.version>
</properties>
<build>
Expand Down Expand Up @@ -291,7 +291,7 @@
<properties>
<scala.version>2.12.10</scala.version>
<scala.compat.version>2.12</scala.compat.version>
<spark.version>3.4.1</spark.version>
<spark.version>3.5.0</spark.version>
<mosaic.version>0.4.4</mosaic.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
Expand Down
4 changes: 2 additions & 2 deletions scripts/docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
## Steps

1. Cmd `GDAL_VERSION=3.4.1 LIBPROJ_VERSION=7.1.0 SPARK_VERSION=3.4.1 CORES=4 ./build`
builds the docker image for DBR 13.3 LTS. Name will be 'mosaic-dev:ubuntu22-gdal3.4.1-spark3.4.1'.
1. Cmd `GDAL_VERSION=3.4.1 LIBPROJ_VERSION=7.1.0 SPARK_VERSION=3.5.0 CORES=4 ./build`
builds the docker image for DBR 13.3 LTS. Name will be 'mosaic-dev:ubuntu22-gdal3.4.1-spark3.5.0'.
2. Cmd `sh scripts/docker/mosaic-docker.sh` to run. That script launches a container and further (optionally) configures.

## Additional Notes
Expand Down
6 changes: 3 additions & 3 deletions scripts/docker/mosaic-docker.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/bin/bash

# [1] Build the image under 'docker-build':
# `GDAL_VERSION=3.4.1 LIBPROJ_VERSION=7.1.0 SPARK_VERSION=3.4.1 CORES=4 ./build`
# - produces image 'ubuntu22-gdal3.4.1-spark3.4.1' [default is JDK 8]
# `GDAL_VERSION=3.4.1 LIBPROJ_VERSION=7.1.0 SPARK_VERSION=3.5.0 CORES=4 ./build`
# - produces image 'ubuntu22-gdal3.4.1-spark3.5.0' [default is JDK 8]
# [2] run this in root of (mosaic repo), e.g. `sh scripts/docker/mosaic-docker.sh`
# - for IDE driven or Jupyter notebook testing
# [3] if you want to run tests within the container shell
Expand All @@ -18,6 +18,6 @@
# NOTE: Ignore 'ERRO[0000] error waiting for container: context canceled'
docker run -q --privileged --platform linux/amd64 --name mosaic-dev -p 5005:5005 -p 8888:8888 \
-v $PWD:/root/mosaic -e JAVA_TOOL_OPTIONS="-agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n" \
-itd --rm mosaic-dev:ubuntu22-gdal3.4.1-spark3.4.1 /bin/bash
-itd --rm mosaic-dev:ubuntu22-gdal3.4.1-spark3.5.0 /bin/bash
docker exec -it mosaic-dev /bin/bash -c "sh /root/mosaic/scripts/docker/docker_init.sh"
docker exec -it mosaic-dev /bin/bash -c "unset JAVA_TOOL_OPTIONS && cd /root/mosaic && /bin/bash"

0 comments on commit 6b049f8

Please sign in to comment.