Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Conan as a dependency manager #33

Closed
wants to merge 6 commits into from
Closed

Conversation

jellespijker
Copy link
Member

@jellespijker jellespijker commented Aug 8, 2021

I'm currently working on integrating Conan as a dependency manager in my workflow. While Charon doesn't have a lot of dependencies, it is one of Cura. This script allows Conan to create a package that stores the Python files and adds that path to the PYTHONPATH. The added conanfile.py can be seen as completely separate from your current setup. Meaning that it won't require any changes in how you guys set up your build environment. Although I think that Conan could also be beneficial for you.

This PR is part of the following PRs:

The purpose of these changes is to set up a dependency manager for Cura and here repositories. Cura uses both third-party and Ultimaker maintained dependencies, written in both Python and C++ (or mixtures of both). Not all of these dependencies can be downloaded with the help of a dependency manager such as pip. This makes setting up Cura from source, a pain in the $%^#$$%^. See the graph below for the current dependencies. Adding to the complexity is the way how we're currently consuming third-party dependencies; Some have to be present on the system/provided by the user, some are shipped within the repo, while others are downloaded by CMake.

dep graph

All of the above-mentioned PRs and this one, add a conanfile.py to the root of this project. This is a recipe written in Python which instructs Conan (https://docs.conan.io/en/latest/) how to build and package the repository in such a way that it can be reused by other dependencies. If a required dependency has no binary for your OS and compiler it will build that dependency from scratch and store it in the cache. Making installing Cura from source as simple as:

conan install Cura/4.10.0@ultimaker/testing --build=missing

For a more detailed description see the README.md in this repository https://github.com/jellespijker/conan-um

For testing purposes, I have set up a small home server that can be used by Ultimaker employees. Other developers can test this by cloning the above-mentioned repositories and performing a conan export . ultimaker/testing in each root. That only leaves the SIP package, if you execute a conan export . riverbankingcomputing/testing in this folder https://github.com/jellespijker/conan-um/tree/main/recipes/sip it creates a Conan package for SIP 4.19.25

You can use your own profiles for this, but I have personally tested and developed them with my own jinja template profiles on Linux Manjaro with a GCC compiler, Mac OS Big Sur with a Clang compiler and Windows 11 with a Visual Studio 2019 compiler.
These profiles can be installed with the conan config install https://github.com/jellespijker/conan-config. Make sure you add -pr:b cura_release.jinja -pr:h cura_release.jinja to your install instructions

Conan allows for multiple ways of working. Either the exiting package can be used from the cache, or if you want to work on multiple repositories you can put that repo in editable mode such that the xxx-config.cmake in the project that is depending on the other, will point to the paths of your repo, see https://docs.conan.io/en/latest/developing_packages/editable_packages.html

Because the best practice method to use Conan, which is also the preferred way in Conan 2.0, is to use ( https://docs.conan.io/en/latest/reference/conanfile/tools/cmake.html ) With the tools CMakeDeps, CMakeToolchain, and CMake. The CMakeDeps class will generate xxx-config.cmake files per dependency, while the CMakeToolchain will generate a toolchain to be passed to CMake -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake These two prepare the way for CMake to actually build the project such that it won't need to change the CMakeLists.txt. Allowing Conan to be optional and not mandatory. Unfortunately, our repositories have organically grown CMakeLists.txt mixing old and modern syntax and a build system that uses CMake 3.13.

NOTE:
I decided to harmonize our CMakeLists.txt (in CuraEngine, libArcus, libSavitar and libnest2d) This might also be interresting for this repo. This was mainly done because often the CMake scripts were inconsistent, for instance how we set the fPIC or MD/MDd and MT/MTd flags. Often I had to changes these methods in one repo, while it already worked in another repo. Getting stuff to work on three different OSs in three different languages is a bit time-consuming, to say the least. These uniform methods of preparing CMake instructions can be found in cmake/StandardProjectSettings.cmake. These are tried and tested methods that we already use in our Spatial plugin and are used by Json Turner in this https://github.com/lefticus/cpp_starter_project and are based on the best practices described here: https://github.com/lefticus/cppbestpractices

See this file: https://github.com/Ultimaker/CuraEngine/blob/conan/cmake/StandardProjectSettings.cmake

The changes in CMake basically boil down to:

  • StandardProjectsSettings.cmake
  • Set all compiler warnings known to mankind.
  • Prefer BUILD_SHARED_LIBS over BUILD_STATIC but no change in the interface so no change is needed on
    the build servers.
  • Set build_type to Release if not specified
  • Export compile commands to a json for easier debugging
  • Use position-independent code if applicable (-fPIC)
  • Switch between MT/MTd of MD/MDd for Visual Studio using text generators
  • Set C++17 standard
  • Set stdlib=libc++ for clang-apple
  • Add option ENABLE_BUILD_WITH_TIME_TRACE for generating time tracing json on Clang compilers
  • Created a uniform method of using threading which prefers pthread if the platform supports it and has multiple
    threading libraries installed.
  • ENABLE_CPPCHECK "Enable static analysis with cppcheck" default OFF
  • ENABLE_CLANG_TIDY "Enable static analysis with clang-tidy" default OFF
  • ENABLE_INCLUDE_WHAT_YOU_USE "Enable static analysis with include-what-you-use" default OFF
  • ENABLE_COVERAGE "Enable coverage reporting for gcc/clang" default OFF
  • ENABLE_SANITIZER_ADDRESS "Enable address sanitizer" default OFF
  • ENABLE_SANITIZER_LEAK "Enable leak sanitizer" default OFF
  • ENABLE_SANITIZER_UNDEFINED_BEHAVIOR "Enable undefined behavior sanitizer" default OFF
  • ENABLE_SANITIZER_THREAD "Enable thread sanitizer" default OFF
  • ENABLE_SANITIZER_MEMORY "Enable memory sanitizer" default OFF
  • Disallow in-source builds by default but added the option ALLOW_IN_SOURCE_BUILD, some build systems such as
    Conan copies the source and builds from there.
  • Enable IPO/LTO https://en.wikipedia.org/wiki/Interprocedural_optimization
  • Use a modern way of finding Python if you want to link to a specific version pass -DPython_version=3.9 now it will link to
    3.8 by default.

The above-mentioned CMake changes should keep the build interface the same for existing environments (only add extra options) Now you guys might get a bit discouraged when you see all of those warnings scroll up when building CuraEngine. We can opt to disable it. But I personally am strongly in favor of keeping these visible as a motivation to clean up our codebase each time we're working in a certain section.

Still, WIP at the moment, since I'm finalizing some last changes across all repos

I'm currently working on integrating Conan as a dependency manager in my workflow.
While Charon doesn't have a lot of dependencies, it is one of Cura.
This script allows Conan to create a package which stores the Python files and
adds that path to the PYTHONPATH.

> **Pro tip!**
> Install Conan
>
> Install Cura configuration files:
> `conan config install https://github.com/jellespijker/conan-config.git`
>
> If you want to let Conan handle the dependencies just use:
> `-DCMAKE_TOOLCHAIN_FILE=cmake-build-<build_type>/conan_toolchain.cmake`
> Install Conan plugin in Clion go to Conan settings `ctrl+alt+s`add Conan arguments
> `--build=missing pr:b cura_release.jinja`
> Match the build profiles with the Conan profiles in the Conan plugin sidebar
>
> If you want to use this as a dependency add the requirement `Charon/4.10.0@ultimaker/testing`
> to your `conanfile.py` or `conanfile.txt`
> If you want to use this source as the dependency (work on multiple projects at the same time)
> make this an editable with `conan editable . Charon/4.10.0@ultimaker/testing` the auto
> generated config files should now point to these source files and the compiled binaries.
source_folder was None at that time
Build and config nor needed. This is basically a pure Python package
for the consumer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant