Skip to content

Commit

Permalink
Builder changes for 13.0.1 (#226)
Browse files Browse the repository at this point in the history
  • Loading branch information
Aman-Surkar authored May 9, 2024
1 parent 1a6bee7 commit ea66c70
Show file tree
Hide file tree
Showing 6 changed files with 23 additions and 23 deletions.
2 changes: 1 addition & 1 deletion conda/recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set name = "open-ce-builder" %}
{% set version = "13.0.0" %}
{% set version = "13.0.1" %}

package:
name: {{ name }}
Expand Down
4 changes: 2 additions & 2 deletions doc/README.cuda_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ the builds are taking place. This can be accomplished in two ways (see below).
directory where CUDA has been installed.**

The standard CUDA installation location is in `/usr/local/cuda` which is typically
a symbolic link to either `/usr/local/cuda-11.2` or `/usr/local/cuda-11.8`. Most of
a symbolic link to either `/usr/local/cuda-11.8` or `/usr/local/cuda-12.2`. Most of
the code bases that are built in Open-CE are flexible enough to tolerate CUDA
installations in non-standard locations, but not all of them. The reference table below
includes the recipes that will not work when CUDA is installed elsewhere.
Expand Down Expand Up @@ -103,7 +103,7 @@ The tag can also be used per line shown in this example for the xgboost [meta.ya
The `--cuda_versions` flag can be passed to `open-ce` to specify which version of CUDA to build conda packages for.
```shell
open-ce build env --build_types cuda --cuda_versions 11.2 envs/opence-env.yaml
open-ce build env --build_types cuda --cuda_versions 12.2 envs/opence-env.yaml
```
Only one CUDA version can be specified at a time. To build for both on the same system, the build must be run twice in one of the following ways:
Expand Down
18 changes: 9 additions & 9 deletions doc/README.open_ce_build.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ optional arguments:
current working directory. (default: )
--python_versions PYTHON_VERSIONS
Comma delimited list of python versions to build for ,
such as "3.9" or "3.10". (default: 3.11 when --build_types=cuda 12.2 or cpu) (default: 3.9 when --build_types=cuda 11.2)
such as "3.10" or "3.11". (default: 3.11 when --build_types=cuda 12.2 or cpu) (default: 3.10 when --build_types=cuda 11.8)
--build_types BUILD_TYPES
Comma delimited list of build types, such as "cpu" or
"cuda". (default: cpu,cuda)
Expand Down Expand Up @@ -220,30 +220,30 @@ optional arguments:
`open-ce build env` also generates conda environment files based on the configuration
selected for a build. For e.g. if `open-ce build env` is run for `tensorflow-env.yaml` and
for python_versions `3.8`, build_type `cuda` and mpi_type being `openmpi`, then a
conda environment file with name `open-ce-conda-env-py3.8-cuda-openmpi.yaml` gets
for python_versions `3.11`, build_type `cuda` and mpi_type being `openmpi`, then a
conda environment file with name `open-ce-conda-env-py3.11-cuda-openmpi.yaml` gets
generated. This environment file can be used to create a conda environment with
the packages listed in `tensorflow-env.yaml` installed in it.
```shell
open-ce build env --python_versions=3.8 --build_type=cuda --mpi_type=openmpi
open-ce build env --python_versions=3.11 --build_type=cuda --mpi_type=openmpi
tensorflow-env.yaml
```
The above command will output `open-ce-conda-env-py3.8-cuda-openmpi.yaml` in the specified
The above command will output `open-ce-conda-env-py3.11-cuda-openmpi.yaml` in the specified
output folder (or by default `./condabuild` directory).
The following command can be used to create a conda environment using the generated conda
environment file -
```shell
conda env create -f open-ce-conda-env-py3.8-cuda-openmpi.yaml
conda env create -f open-ce-conda-env-py3.11-cuda-openmpi.yaml
```
There could be one or more conda environment files generated for each variant based on inputs
given to `open-ce build env`. For example, if `open-ce build env` is run without any `build_type` and python_versions
`3.8` and mpi_type as `openmpi`, then two files will be generated namely -
`open-ce-conda-env-py3.8-cuda-openmpi.yaml`, `open-ce-conda-env-py3.8-cpu-openmpi.yaml`.
`3.11` and mpi_type as `openmpi`, then two files will be generated namely -
`open-ce-conda-env-py3.11-cuda-openmpi.yaml`, `open-ce-conda-env-py3.11-cpu-openmpi.yaml`.
`open-ce build env` can generate these target conda environment files for a given Open-CE environment file
and provided build configuration even without performing an actual build.
Expand Down Expand Up @@ -398,7 +398,7 @@ For example,
```shell
open-ce build image --local_conda_channel=./condabuild
--conda_env_file=open-ce-conda-env-py3.8-cuda-openmpi.yaml
--conda_env_file=open-ce-conda-env-py3.11-cuda-openmpi.yaml
--container_tool podman --container_build_args="--build-arg ENV1=test1 --cpuset-cpus 0,1"
```
Expand Down
8 changes: 4 additions & 4 deletions doc/README.open_ce_test.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@ tests:

## Running Tests After a Build

Tests can be run immediately after a build has completed by passing the `--run_tests` argument to `open-ce build env`. Tests will be run for every package that was built (but not until every package has been built). Tests will be run for every combination of build variants that were specified by the build command. For example, if `python_versions` is set to `3.8,3.9`, tests will be run for the python 3.8 packages and python 3.9 packages. For more information on the `open-ce build env` command, see [doc/README.open_ce_build.md](README.open_ce_build.md). When run as part of a build, the `test_results.xml` file will be located in the output folder.
Tests can be run immediately after a build has completed by passing the `--run_tests` argument to `open-ce build env`. Tests will be run for every package that was built (but not until every package has been built). Tests will be run for every combination of build variants that were specified by the build command. For example, if `python_versions` is set to `3.10,3.11`, tests will be run for the python 3.10 packages and python 3.11 packages. For more information on the `open-ce build env` command, see [doc/README.open_ce_build.md](README.open_ce_build.md). When run as part of a build, the `test_results.xml` file will be located in the output folder.

## `open-ce test env` sub command

The `open-ce test env` command can be used to run tests for every package listed within an Open-CE Environment [file](README.yaml.md). Tests will be run for every combination of build variants that are specified. For example, if `python_versions` is set to `3.8,3.9,3.10`, tests will be run for the python 3.8 packages, python 3.9 packages and python 3.10 packages.
The `open-ce test env` command can be used to run tests for every package listed within an Open-CE Environment [file](README.yaml.md). Tests will be run for every combination of build variants that are specified. For example, if `python_versions` is set to `3.10,3.11`, tests will be run for the python 3.10 packages and python 3.11 packages.

### Command usage for `open-ce test env`

Expand Down Expand Up @@ -116,15 +116,15 @@ optional arguments:
current working directory. (default: )
--python_versions PYTHON_VERSIONS
Comma delimited list of python versions to build for ,
such as "3.8" or "3.9" or "3.10". (default: 3.10 when --build_types=cuda 11.8 or cpu) (default: 3.9 when --build_types=cuda 11.2) 
such as "3.10" or "3.11". (default: 3.10 when --build_types=cuda 11.8 or cpu) (default: 3.11 when --build_types=cuda 12.2) 
--build_types BUILD_TYPES
Comma delimited list of build types, such as "cpu" or
"cuda". (default: cpu,cuda)
--mpi_types MPI_TYPES
Comma delimited list of mpi types, such as "openmpi"
or "system". (default: openmpi)
--cuda_versions CUDA_VERSIONS
CUDA version to build for , such as "11.2" or "11.8".
CUDA version to build for , such as "11.8" or "12.2".
(default: 11.8)
--container_build, --docker_build
Perform a build within a container. NOTE: When
Expand Down
2 changes: 1 addition & 1 deletion open_ce/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@
# limitations under the License.
# *****************************************************************
"""
__version__ = "13.0.0"
__version__ = "13.0.1"
12 changes: 6 additions & 6 deletions open_ce/inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ class Argument(Enum):
type=str,
default=constants.DEFAULT_PYTHON_VERS,
help='Comma delimited list of python versions to build for '
', such as "3.9" or "3.10" or "3.11".'))
', such as "3.10" or "3.11".'))

BUILD_TYPES = (lambda parser: parser.add_argument(
'--build_types',
Expand Down Expand Up @@ -388,15 +388,15 @@ def _create_env_config_paths(args):

def _check_cuda_versions(args):
'''
This will check if build_types is cuda and cuda_versions is 11.2
then set default python_versions to 3.9
This will check if build_types is cuda and cuda_versions is 11.8
then set default python_versions to 3.10
'''
if "build_types" in vars(args).keys() and args.build_types:
if "cuda_versions" in vars(args).keys() and args.cuda_versions:
if args.cuda_versions == "11.2":
if args.cuda_versions == "11.8":
if args.python_versions == "3.10":
opence_globals.DEFAULT_PYTHON_VERS = "3.9"
args.python_versions = "3.9"
opence_globals.DEFAULT_PYTHON_VERS = "3.10"
args.python_versions = "3.10"

def _check_ppc_arch(args):
'''
Expand Down

0 comments on commit ea66c70

Please sign in to comment.