Skip to content

Remove Keras HDF5 #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 28 additions & 30 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,52 +14,50 @@ on:
- "fpm.toml"
- "**.f90"

env:
FPM_FFLAGS: -I/usr/include/hdf5/serial
FPM_LDFLAGS: -L/usr/lib/x86_64-linux-gnu/hdf5/serial

jobs:

build_and_test_debug_profile:
name: Build and test in debug mode
gnu-cmake-debug:
name: gnu-cmake-debug
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Compile
run: cmake -DCMAKE_BUILD_TYPE=Debug -DSERIAL=1 . && make
- name: Test
run: make test

gnu-cmake-release:
name: gnu-cmake-release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Compile
run: cmake -DCMAKE_BUILD_TYPE=Release -DSERIAL=1 . && make
- name: Test
run: make test

- uses: fortran-lang/setup-fpm@v4
gnu-fpm-debug:
name: gnu-fpm-debug
runs-on: ubuntu-latest
steps:
- uses: fortran-lang/setup-fpm@v5
with:
fpm-version: "v0.6.0"

- name: Install HDF5
run: |
sudo apt update
sudo apt install --no-install-recommends libhdf5-dev
- uses: actions/checkout@v2

fpm-version: "v0.10.1"
- uses: actions/checkout@v4
- name: Compile
run: fpm build --profile debug

- name: Test
run: fpm test --profile debug

build_and_test_release_profile:
name: Build and test in release mode
gnu-fpm-release:
name: gnu-fpm-release
runs-on: ubuntu-latest

steps:

- uses: fortran-lang/setup-fpm@v4
- uses: fortran-lang/setup-fpm@v5
with:
fpm-version: "v0.6.0"

- name: Install HDF5
run: |
sudo apt update
sudo apt install --no-install-recommends libhdf5-dev
- uses: actions/checkout@v2

fpm-version: "v0.10.1"
- uses: actions/checkout@v4
- name: Compile
run: fpm build --profile release

- name: Test
run: fpm test --profile release
21 changes: 2 additions & 19 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,13 @@ if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Default build Release")
endif()

project(neural-fortran
LANGUAGES C Fortran
)
project(neural-fortran LANGUAGES Fortran)

enable_testing()

include(FetchContent)

include(cmake/options.cmake)
include(cmake/compilers.cmake)

include(cmake/functional.cmake)
include(cmake/h5fortran.cmake)
include(cmake/json.cmake)

# library to archive (libneural-fortran.a)
add_library(neural-fortran
src/nf.f90
Expand All @@ -40,8 +32,6 @@ add_library(neural-fortran
src/nf/nf_input1d_layer_submodule.f90
src/nf/nf_input3d_layer.f90
src/nf/nf_input3d_layer_submodule.f90
src/nf/nf_keras.f90
src/nf/nf_keras_submodule.f90
src/nf/nf_layer_constructors.f90
src/nf/nf_layer_constructors_submodule.f90
src/nf/nf_layer.f90
Expand All @@ -61,16 +51,9 @@ add_library(neural-fortran
src/nf/nf_reshape_layer_submodule.f90
src/nf/io/nf_io_binary.f90
src/nf/io/nf_io_binary_submodule.f90
src/nf/io/nf_io_hdf5.f90
src/nf/io/nf_io_hdf5_submodule.f90
)

target_link_libraries(neural-fortran PRIVATE
functional::functional
h5fortran::h5fortran
HDF5::HDF5
jsonfortran::jsonfortran
)
target_link_libraries(neural-fortran PRIVATE)

install(TARGETS neural-fortran)

Expand Down
50 changes: 10 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ Read the paper [here](https://arxiv.org/abs/1902.06714).
RMSProp, Adagrad, Adam, AdamW
* More than a dozen activation functions and their derivatives
* Loss functions and metrics: Quadratic, Mean Squared Error, Pearson Correlation etc.
* Loading dense and convolutional models from Keras HDF5 (.h5) files
* Data-based parallelism
* Loading dense and convolutional models from Keras HDF5 (.h5) files
(see the [nf-keras-hdf5](https://github.com/neural-fortran/nf-keras-hdf5) add-on)

### Available layers

Expand Down Expand Up @@ -51,14 +52,8 @@ cd neural-fortran
Required dependencies are:

* A Fortran compiler
* [HDF5](https://www.hdfgroup.org/downloads/hdf5/)
(must be provided by the OS package manager or your own build from source)
* [functional-fortran](https://github.com/wavebitscientific/functional-fortran),
[h5fortran](https://github.com/geospace-code/h5fortran),
[json-fortran](https://github.com/jacobwilliams/json-fortran)
(all handled by neural-fortran's build systems, no need for a manual install)
* [fpm](https://github.com/fortran-lang/fpm) or
[CMake](https://cmake.org) for building the code
[CMake](https://cmake.org) to build the code

Optional dependencies are:

Expand All @@ -79,23 +74,7 @@ Compilers tested include:
With gfortran, the following will create an optimized build of neural-fortran:

```
fpm build \
--profile release \
--flag "-I$HDF5INC -L$HDF5LIB"
```

HDF5 is now a required dependency, so you have to provide it to fpm.
The above command assumes that the `HDF5INC` and `HDF5LIB` environment
variables are set to the include and library paths, respectively, of your
HDF5 install.

If you use Conda, the following instructions work:

```
conda create -n nf hdf5
conda activate nf
fpm build --profile release --flag "-I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -Wl,-rpath -Wl,$CONDA_PREFIX/lib"
fpm test --profile release --flag "-I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -Wl,-rpath -Wl,$CONDA_PREFIX/lib"
fpm build --profile release
```

#### Building in parallel mode
Expand All @@ -106,25 +85,20 @@ Once installed, use the compiler wrappers `caf` and `cafrun` to build and execut
in parallel, respectively:

```
fpm build \
--compiler caf \
--profile release \
--flag "-I$HDF5INC -L$HDF5LIB"
fpm build --compiler caf --profile release
```

#### Testing with fpm

```
fpm test \
--profile release \
--flag "-I$HDF5INC -L$HDF5LIB"
fpm test --profile release
```

For the time being, you need to specify the same compiler flags to `fpm test`
as you did in `fpm build` so that fpm knows it should use the same build
profile.

See [Fortran Package Manager](https://github.com/fortran-lang/fpm) for more info on fpm.
See the [Fortran Package Manager](https://github.com/fortran-lang/fpm) for more info on fpm.

### Building with CMake

Expand Down Expand Up @@ -156,8 +130,7 @@ cafrun -n 4 bin/mnist # run MNIST example on 4 cores
#### Building with a different compiler

If you want to build with a different compiler, such as Intel Fortran,
set the `HDF5_ROOT` environment variable to the root path of your
Intel HDF5 build, and specify `FC` when issuing `cmake`:
specify `FC` when issuing `cmake`:

```
FC=ifort cmake ..
Expand Down Expand Up @@ -213,6 +186,7 @@ You can configure neural-fortran by setting the appropriate options before
including the subproject.

The following should be added in the CMake file of your directory:

```cmake
if(NOT TARGET "neural-fortran::neural-fortran")
find_package("neural-fortran" REQUIRED)
Expand All @@ -230,11 +204,7 @@ examples, in increasing level of complexity:
3. [dense_mnist](example/dense_mnist.f90): Hand-written digit recognition
(MNIST dataset) using a dense (fully-connected) network
4. [cnn_mnist](example/cnn_mnist.f90): Training a CNN on the MNIST dataset
5. [dense_from_keras](example/dense_from_keras.f90): Creating a pre-trained
dense model from a Keras HDF5 file and running the inference.
6. [cnn_from_keras](example/cnn_from_keras.f90): Creating a pre-trained
convolutional model from a Keras HDF5 file and running the inference.
7. [get_set_network_params](example/get_set_network_params.f90): Getting and
5. [get_set_network_params](example/get_set_network_params.f90): Getting and
setting hyperparameters of a network.

The examples also show you the extent of the public API that's meant to be
Expand Down
18 changes: 0 additions & 18 deletions cmake/functional.cmake

This file was deleted.

15 changes: 0 additions & 15 deletions cmake/h5fortran.cmake

This file was deleted.

35 changes: 0 additions & 35 deletions cmake/json.cmake

This file was deleted.

4 changes: 0 additions & 4 deletions example/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
foreach(execid
cnn_mnist
cnn_from_keras
dense_mnist
dense_from_keras
get_set_network_params
network_parameters
simple
Expand All @@ -12,8 +10,6 @@ foreach(execid
add_executable(${execid} ${execid}.f90)
target_link_libraries(${execid} PRIVATE
neural-fortran
h5fortran::h5fortran
jsonfortran::jsonfortran
${LIBS}
)
endforeach()
58 changes: 0 additions & 58 deletions example/cnn_from_keras.f90

This file was deleted.

Loading
Loading