Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CI, CD #88

Merged
merged 42 commits into from
Oct 1, 2024
Merged

Update CI, CD #88

merged 42 commits into from
Oct 1, 2024

Conversation

mfep
Copy link
Contributor

@mfep mfep commented Oct 17, 2023

  • Updated format enforcing script.
  • Minor changes to source files to avoid warnings on all compilers tested in CI.
    • -Werror//WX is enabled in CI, however a list of warnings had to be enabled, since their resolution was deemed out-of-scope.
  • Added clinfo as a dependency, that is built with the project.
  • Refactored the CMakeLists.txt to be able to produce binary Debian packages using cpack.
    • CMake minimum version is bumped to 3.16.
  • Updated the CI (pre-release) script.
    • Updated Linux CI to use a new container image
      • Most jobs in the matrix are based on Ubuntu 22.04
      • Compilers gcc-11, gcc-13, clang-14, clang-16 and gcc-9 on Ubuntu 20.04
      • Binary targets: x86-64 and x86 on Ubuntu 20.04
      • CMake versions: 3.26 and 3.16 on Ubuntu 20.04
      • CMake generators: Ninja Multi-Config, Unix Makefiles
      • Added test execution
      • Added the generation of the Debian package and testing the consumption of the package, as well as the installed tree
    • Updated Windows CI, which runs on Windows Server 2022
      • MSVC toolsets: v141, v142, v143, clang-cl (this latter with msbuild only)
      • Binary targets: x86-64, x86
      • CMake: system installed, currently 3.27
      • CMake generators: Ninja Multi-Config, Visual Studio 17 2022
      • Added the result checking of the external command invocations in the PowerShell snippets
      • Added test execution
      • Added build and test steps checking the consumption of the installed tree
    • Updated MacOS CI, which runs on MacOS 13
      • Compiler: Apple-Clang 14
      • Binary target: x86-64
      • CMake: system installed, currently 3.27
      • CMake generators: Ninja Multi-Config, XCode
      • Added test execution
      • Added build and test steps checking the consumption of the installed tree
    • Added Android CI, which is build only (virtualization is not available with the default GitHub runners)
      • Binary targets: x86-64, arm64
      • Android API levels: 19, 33
  • Updated the release workflow
    • Added DebSourcePkg.cmake which is intended to run in CMake script mode. This script generates the debian/control, debian/changelog and debian/rules files which are required to build a Debian source package.
    • The release workflow generates the Debian source package and pushes it to a prescribed Launchpad PPA. The details of the release (e.g. maintainer, signing key) has to be set up as repository variables.
    • clinfo is not part of the aforementioned Debian source package, since the Debian package build on Launchpad cannot access the Internet. A potential solution would be forking clinfo and referencing it as a submodule, but this was decided to be out-of-scope for now.
    • Detailed instruction regarding the release procedure is added to RELEASE.md.

A ToDo item is to revert the referenced submodule commits to the KhronosGroup repos, once the respective PRs have been merged.

This PR is considered to be complete, albeit review remarks and/or changes to related PRs might warrant minor updates.

@CLAassistant
Copy link

CLAassistant commented Oct 17, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@bashbaug bashbaug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, I did not review all of the CI/CD and CMake changes, but here are some comments about the sample updates themselves.

.gitmodules Outdated Show resolved Hide resolved
docs/RELEASE.md Outdated Show resolved Hide resolved
lib/src/Extensions/src/openclext.cpp Show resolved Hide resolved
lib/src/Utils/File.c Outdated Show resolved Hide resolved
Comment on lines +133 to +135
install(TARGETS ${OPENCL_SAMPLE_TARGET} CONFIGURATIONS ${CONFIG} DESTINATION ${CMAKE_INSTALL_BINDIR}/${CONFIG})
install(FILES ${OPENCL_SAMPLE_KERNELS} CONFIGURATIONS ${CONFIG} DESTINATION ${CMAKE_INSTALL_BINDIR}/${CONFIG})
install(FILES ${OPENCL_SAMPLE_SHADERS} CONFIGURATIONS ${CONFIG} DESTINATION ${CMAKE_INSTALL_BINDIR}/${CONFIG})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that instead of the samples being installed to <install_prefix>/install/bin they will be installed to <install_prefix>/install/bin/<config>, so something like /path/to/sdk/install/bin/Release. Was this intended? I'm not opposed to this change, but it is different than what we had previously.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It happened when adding the tests jobs to CI that in some environments the execution of the tests failed when configuring and building for Debug and Release (so first we configured for Debug and Release, then we built for Debug and Release and then we executed), and separating the install of the executables was solving it. Currently I'm not able to reproduce the old buggy behaviour, so probably some of the other changes of this PR triggered the fix for it. We can undo this change then IMHO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the older behavior still works I think I'd slightly lean towards keeping the same installation directories, but this is something the working group should decide.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let's see it then in the next WG meeting

Comment on lines 1207 to 1203
// cl_khr_subgroup_shuffle requires OpenCL 2.0
strcat(kernel_op, " -cl-std=CL2.0 ");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above: there is no dependency between cl_khr_subgroup_shuffle and OpenCL C 2.0.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(IIRC) I think this was a consequence of one runtime which didn't behave well if it wasn't set. I'll verify tomorrow the reason for this.

samples/core/blur/main.cpp Outdated Show resolved Hide resolved
samples/core/blur/main.cpp Outdated Show resolved Hide resolved
samples/core/blur/main.cpp Outdated Show resolved Hide resolved
samples/core/blur/main.cpp Outdated Show resolved Hide resolved
@Beanavil
Copy link
Contributor

Latest commits:

  • update Docker images so that the ones from the khronosgroup Docker Hub are used

  • fix MSVC compiler toolset version due to breaking change in the MSVC versioning (see cppblog)

    We have incremented the minor version number of the MSVC toolset from 19.39 (VS 2022 v17.9) to 19.40 (VS 2022 v17.10)

Copy link
Contributor

@bashbaug bashbaug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, many issues are fixed, but a few new ones have cropped up (or I didn't notice them the first time).

I don't think anything is broken with these changes per se, but the checks for OpenCL 2.0 and cl_khr_subgroups do mean that these samples cannot run on as many devices as they should. So, if we want to merge these changes as-is and fix things up afterwards to make progress, I'd be OK with that, though ideally they'd be fixed before merging.

samples/core/blur/blur.cpp Outdated Show resolved Hide resolved
samples/core/blur/main.c Outdated Show resolved Hide resolved
samples/core/blur/main.cpp Outdated Show resolved Hide resolved
@Beanavil Beanavil force-pushed the release-cd branch 3 times, most recently from c954192 to 807a04a Compare July 6, 2024 01:25
.gitmodules Outdated Show resolved Hide resolved
.gitmodules Outdated Show resolved Hide resolved
Copy link

@EwanC EwanC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert in this SDK repo, so not clicked the big green approve button. But I have looked through this PR to try help unblock getting reviews on it and looks fine to me apart from the few minor comments I've made.

cmake/Dependencies/clinfo/ext.h Outdated Show resolved Hide resolved
test/cmake/platformenum.cpp Show resolved Hide resolved
Comment on lines 516 to 524
// 5) Query OpenCL version to compile for.
// If no -cl-std option is specified then the highest 1.x version
// supported by each device is used to compile the program. Therefore,
// it's only necessary to add the -cl-std option for 2.0 and 3.0 OpenCL
// versions.
const std::string dev_version = device.getInfo<CL_DEVICE_VERSION>();
cl::string compiler_options;
constexpr int max_major_version = 3;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect other samples will need to do something similar, so this would be a great SDK utility function. This code from the OpenCL CTS may be a useful starting point:

https://github.com/KhronosGroup/OpenCL-CTS/blob/cd74e0264391e0424a27f657852fd1e9f10bf42e/test_common/harness/kernelHelpers.cpp#L1645

Strictly speaking, this really ought to be checking for the OpenCL C versions supported by the device, not for the OpenCL version supported by the device, but given the current couplings between the OpenCL version and the OpenCL C versions I think this is OK.

@Beanavil
Copy link
Contributor

@bashbaug so should we merge?

@bashbaug
Copy link
Contributor

bashbaug commented Oct 1, 2024

Merging as discussed in the October 1st teleconference.

We'll check that the install directories are working as we intend after merging. Specifically, with a non-multi-config generator (e.g. Makefiles), does the install go into per-config directories, or into a single lib or bin directory?

@bashbaug bashbaug merged commit dbcae2c into KhronosGroup:main Oct 1, 2024
122 checks passed
Beanavil added a commit to Beanavil/OpenCL-SDK that referenced this pull request Oct 9, 2024
This reverts the changes made on KhronosGroup#88 that modified the install location
of the samples
bashbaug pushed a commit that referenced this pull request Oct 11, 2024
* Revert install tree modifications

This reverts the changes made on #88 that modified the install location
of the samples

* Make vendors path not depend on hard-coded POCL install path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants