Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Julia support to gprc4bmi #144

Merged
merged 31 commits into from
Oct 30, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b913f8d
Add Julia support to gprc4bmi
sverhoeven Sep 29, 2023
57bb251
Switch to 2 arg constructor + Add tests for julia
sverhoeven Oct 2, 2023
3607d56
Add output arg + get package from model name + more tests
sverhoeven Oct 2, 2023
2b612dc
Convert Julia type to Python type
sverhoeven Oct 2, 2023
5d64b15
Force install of heat.jl
sverhoeven Oct 2, 2023
45c9879
Make sure BasicModelInterface.jl is installed
sverhoeven Oct 2, 2023
6bb4e47
Upgrade sonar action
sverhoeven Oct 5, 2023
4f1de4e
Dont need BasicModelInterface.jl to be installed explicitly?
sverhoeven Oct 5, 2023
340067e
Revert "Dont need BasicModelInterface.jl to be installed explicitly?"
sverhoeven Oct 16, 2023
ceff03c
Try to test against a jl file
sverhoeven Oct 16, 2023
ac4fdac
Try to test against a jl file (python side)
sverhoeven Oct 16, 2023
d5d38cd
Fake model.jl does not config file
sverhoeven Oct 16, 2023
badc5f7
More tests + py index to julia index + 1
sverhoeven Oct 17, 2023
a8d4e59
Fix get_value_at_indices
sverhoeven Oct 17, 2023
daeca16
Make implementatation name settable from cli
sverhoeven Oct 17, 2023
575b0f9
Trying to call Julia from C
sverhoeven Oct 17, 2023
fe42fc7
Calling sqrt in julia from c works, now try to call https://github.co…
sverhoeven Oct 17, 2023
92cb27c
Trying to convert repl calls into c
sverhoeven Oct 17, 2023
05e1b47
Got it to work, until it segfaults somewhere
sverhoeven Oct 18, 2023
ffbe702
TODOs
sverhoeven Oct 18, 2023
0a2037a
Convert numpy array to Julia vector
BSchilperoort Oct 24, 2023
9f413af
Remove install method use jl.Pkg.add directly
sverhoeven Oct 25, 2023
125560a
Keep BmiJulia and heat-images/julia-c, remove other mentions of Julia
sverhoeven Oct 25, 2023
7d1c3e5
Newline
sverhoeven Oct 25, 2023
a54c731
Add note that test/heat-images/c-julia is broken
sverhoeven Oct 25, 2023
c6bddde
Add timeout to test
sverhoeven Oct 25, 2023
ead67cd
Log thread id
sverhoeven Oct 26, 2023
4333f13
Merge branch 'julia' of github.com:eWaterCycle/grpc4bmi into julia
sverhoeven Oct 26, 2023
7c29974
Increase delay in BmiClientSubProcess tests
sverhoeven Oct 26, 2023
3fbb905
Dont wait forever after subproces is terminated
sverhoeven Oct 26, 2023
2fb9ffb
Kill subprocesses instead of letting them terminate
sverhoeven Oct 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
run: |
python -m pip install --upgrade pip wheel
pip install -r dev-requirements.txt
pip install -e .[R]
pip install -e .[R,julia]
- name: Setup Apptainer
uses: eWaterCycle/setup-apptainer@v2
with:
Expand All @@ -45,13 +45,17 @@ jobs:
run: |
Rscript -e "install.packages('remotes')"
Rscript -e "install.packages('R6')"
- name: Install Julia
uses: julia-actions/setup-julia@v1
with:
version: '^1.9'
- name: Test with pytest
run: |
pytest -vv --cov=grpc4bmi --cov-report xml
- name: Correct coverage paths
run: sed -i "s+$PWD/++g" coverage.xml
- name: SonarCloud analysis
uses: sonarsource/sonarcloud-github-action@v1.3
uses: sonarsource/sonarcloud-github-action@v2.0.2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
Expand Down
29 changes: 25 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ on the client (Python) side. If your server model is implemented in Python, do t
pip install grpc4bmi[R]
```

If the model is implemented in Julia, run instead

```bash
pip install grpc4bmi[julia]
```

in the server environment. For bleeding edge version from GitHub use

```bash
Expand Down Expand Up @@ -90,6 +96,25 @@ For example with [WALRUS](https://github.com/eWaterCycle/grpc4bmi-examples/tree/
run-bmi-server --lang R --path ~/git/eWaterCycle/grpc4bmi-examples/walrus/walrus-bmi.r --name WalrusBmi --port 55555
```

### Models written in Julia

The grpc4bmi Python package can also run BMI models written in Julia if the model has an implementation of the [BasicModelInterface.jl](https://github.com/Deltares/BasicModelInterface.jl).

Run the Julia model as a server with

```bash
run-bmi-server --lang julia --name <MODEL-NAME> --port <PORT>
```

For example with [Wflow.jl](https://github.com/Deltares/Wflow.jl/) use

```bash
# Install Wflow.jl package in the Julia environment managed by the juliacall Python package.
python3 -c 'from grpc4bmi.bmi_julia_model import install;install("Wflow")'
# Run the server
run-bmi-server --lang julia --name Wflow.Model --port 55555
```

### The client side

The client side has only a Python implementation. The default BMI client assumes a running server process on a given port.
Expand Down Expand Up @@ -154,7 +179,3 @@ pip install -e .[docs]

and install the C++ runtime and `protoc` command as described in <https://github.com/google/protobuf/blob/master/src/README.md>.
After this, simply executing the `proto_gen.sh` script should do the job.

## Future work

More language bindings are underway.
24 changes: 24 additions & 0 deletions docs/container/building.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,30 @@ The WALRUS model has a `Dockerfile`_ file which can be used as an example.

.. _Dockerfile: https://github.com/eWaterCycle/grpc4bmi-examples/blob/master/walrus/Dockerfile

Julia
-----

The docker file for the model container simply contains the installation instructions of grpc4bmi and the BMI-enabled model itself, and as entrypoint the ``run-bmi-server`` command. For the :ref:`python example <python-example>` the Docker file will read

.. code-block:: Dockerfile

FROM ubuntu:jammy
MAINTAINER your name <your email address>

# Install grpc4bmi
RUN pip install grpc4bmi

# Install your BMI model:
python3 -c 'from grpc4bmi.bmi_julia_model import install;install("<JULIA-PACKAGE-NAME>")'

# Run bmi server
ENTRYPOINT ["run-bmi-server", "--lang", "julia", "--name", "<MODEL-NAME>"]

# Expose the magic grpc4bmi port
EXPOSE 55555

The port 55555 is the internal port in the Docker container that the model communicates over. It is the default port for ``run_bmi_server`` and also the default port that all clients listen to.

C/C++/Fortran
-------------

Expand Down
47 changes: 47 additions & 0 deletions docs/server/Julia.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
Julia
=====

Grpc4bmi allows you to wrap a Hydrological model written in the `Julia language`_ into a GRPC server.

.. _Julia language: https://julialang.org/

Creating
--------

The model should implement `BasicModelInterface.jl`_.

.. _BasicModelInterface.jl: https://github.com/Deltares/BasicModelInterface.jl

See `Wflow.jl`_ for an example.

.. _Wflow.jl: https://deltares.github.io/Wflow.jl/dev/

Running
-------

Once the model has an BMI interface it can be run as a GRPC server by installing the `grpc4bmi[julia]` Python package with

.. code-block:: bash

pip install grpc4bmi[julia]

The model Julia package must be installed in the Julia environment managed by juliacall,
for Wflow use

.. code-block:: bash

python3 -c 'from grpc4bmi.bmi_julia_model import install;install("Wflow")'

The server can be started with

.. code-block:: sh

run-bmi-server --lang julia --name <MODEL-NAME> --port <PORT>

For example with [Wflow.jl](https://github.com/Deltares/Wflow.jl/) use

.. code-block:: sh

run-bmi-server --lang julia --name Wflow.Model --port 55555

The Python grpc4bmi :ref:`usage` can then be used to connect to the server.
1 change: 1 addition & 0 deletions docs/server/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ Creating a BMI server

python
R
Julia
Cpp
Loading
Loading