Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python bindings #105

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
94d231d
Install pybind
ll-nick Oct 8, 2024
4b83fa7
Add basic pybind structure and bind dummy behavior
ll-nick Oct 8, 2024
2408f98
Bind abstract behavior, DummyBehavior and add python unit test equiva…
ll-nick Oct 10, 2024
947ada2
Bind YAML printing via std::string
ll-nick Oct 10, 2024
84a4bc9
Re-implement DummyBehavior in python and run equivalent unit test
ll-nick Oct 10, 2024
99fe5a3
Fix linter warnings, tidy up
ll-nick Oct 10, 2024
60b8f73
Create a testing submodule
ll-nick Oct 10, 2024
ea87760
Bind the placebo verifier
ll-nick Oct 10, 2024
4630942
Bind the abstract Arbitrator
ll-nick Oct 10, 2024
8a6748d
Bind the priority arbitrator
ll-nick Oct 10, 2024
099b02a
Fix unit test for priority arbitrator with different subcommand
ll-nick Oct 11, 2024
4bae4de
Bind the cost arbitrator incl. cost estimator
ll-nick Oct 11, 2024
6bced45
Fix unit test for cost arbitrator with different sub command
ll-nick Oct 11, 2024
e3f7d68
Add a python implemented cost estimator and unit test
ll-nick Oct 11, 2024
8020cdb
Build bindings only if test are enabled, copy test to build directory
ll-nick Oct 11, 2024
fbd62a5
Add centralized binding function and reorder files
ll-nick Oct 11, 2024
969517d
Tidy up python print tests
ll-nick Oct 14, 2024
38413ac
Add python equivalent of nested_arbitrators.cpp test case
ll-nick Oct 14, 2024
04e3a77
Remove to_stream bindings
ll-nick Oct 14, 2024
269d1a5
Tidy up for consistency
ll-nick Oct 14, 2024
70b03c6
Make the py test ouput a little prettier
ll-nick Oct 14, 2024
9ef9976
Bind exceptions
ll-nick Oct 14, 2024
7886578
Bind conjunctive coordinator
ll-nick Oct 14, 2024
fc31afa
Add test for conjunctive coordinator with different sub command
ll-nick Oct 14, 2024
33e54aa
Move DummyVerifier definition into dummy_types.hpp
ll-nick Oct 14, 2024
d112187
Merge command differs from sub command tests into one file
ll-nick Oct 14, 2024
2f96827
Bind DummyVerifier
ll-nick Oct 14, 2024
557daf9
Add python equivalent of verification.cpp test case
ll-nick Oct 15, 2024
f26d902
Restructure bindings for expected print outputs
ll-nick Oct 16, 2024
2c6e54c
Add verified conjunctive coordinator test case to verification.cpp
ll-nick Oct 16, 2024
062be19
Add bindings and unit test for JointCoordinator
ll-nick Oct 16, 2024
f60f261
Adjust DummyVerifier to allow rejecting everything
ll-nick Oct 17, 2024
faa1a30
Add missing verification tests of JointCoordinator to verifier.py
ll-nick Oct 17, 2024
1cfb3c4
Add missing joint coordinator test for different subcommand
ll-nick Oct 17, 2024
b32781e
Bind random arbitrator and add corresponding py unit test
ll-nick Oct 17, 2024
6307606
Add missing random arbitrator test for different sub command
ll-nick Oct 17, 2024
ae2b21b
Remove some unnecessary trampoline classes
ll-nick Oct 17, 2024
ecd18e9
Fix test file path since tests now have their own CMakeLists.txt
ll-nick Dec 18, 2024
9396e69
Add python tests as ctests
ll-nick Dec 18, 2024
1d6e654
PlaceboResult.isOk is no longer static
ll-nick Dec 18, 2024
b14ed2d
Remove duplicated import
ll-nick Dec 18, 2024
c2a20c1
Rename python files containing tests to _test.py to easily distinguis…
ll-nick Dec 18, 2024
3ef743b
Fit pybind documentation into updated Readme structure
ll-nick Dec 18, 2024
41caffd
Bind util_caching types
ll-nick Dec 18, 2024
39a60cc
Remove toYaml bindings
ll-nick Dec 18, 2024
e28824f
Pass unittest flag to python executable to make sure error code is se…
ll-nick Dec 19, 2024
ee43976
Fix wrong boolean in python test
ll-nick Dec 19, 2024
bf8e41e
Add python bindings in contribution section
ll-nick Dec 19, 2024
584563d
Final cleanup before merging
ll-nick Dec 19, 2024
66bd8a1
Pass the entire binding name instead of just a suffix for the Behavio…
ll-nick Jan 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -152,4 +152,4 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/
include(cpack_config)
include(CPack)

message(STATUS "Components to pack: ${CPACK_COMPONENTS_ALL}")
message(STATUS "Components to pack: ${CPACK_COMPONENTS_ALL}")
6 changes: 5 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,11 @@ RUN apt-get update && \
cmake \
libgoogle-glog-dev \
libgtest-dev \
libyaml-cpp-dev && \
libyaml-cpp-dev \
pybind11-dev \
python3-dev \
python3-pybind11 \
python3-yaml && \
apt-get clean

# Install Crow dependencies
Expand Down
38 changes: 34 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
Your behavior is unreliable or unsafe? Arbitrators will gracefully fall back to the next-best option.
- 📦 **Header-Only**
Simple integration – just include this header-only C++ library!
- 🐍 **Python Bindings**
For easy prototyping, testing, and integration of machine learning algorithms, all the functionality is available via Python bindings.
- 📜 **Permissive License**
Published under MIT license to ensure maximum flexibility for your projects.

Expand Down Expand Up @@ -94,6 +96,7 @@ First make sure all dependencies are installed:
- [yaml-cpp](https://github.com/jbeder/yaml-cpp)
- [Googletest](https://github.com/google/googletest) (optional, if you want to build unit tests)
- [Crow](https://crowcpp.org) (optional, needed for GUI only)
- [pybind11](https://pybind11.readthedocs.io/en/stable/) (optional, if you want to build Python bindings and Python unit tests)

See also the [`Dockerfile`](./Dockerfile) for how to install these packages under Debian or Ubuntu.
</details>
Expand Down Expand Up @@ -169,6 +172,31 @@ In order to skip compiling the GUI, use `cmake -DBUILD_GUI=false ..` instead.

</details>

## Python Bindings

The library can be used in Python via pybind11 bindings.
Since `arbitration_graphs` is a templated C++ library,
you need to explicitly instantiate the template for the types you want to use in Python.
For this, we provide convenience functions to bind the library for the desired types.
Simply call them in a pybind11 module definition, e.g.:

```cpp
PYBIND11_MODULE(arbitration_graphs, m) {
python_api::bindArbitrationGraphs<MyCommand>(m);
}
```
and use them in Python.
For example, you could implement a custom behavior that inherits from the abstract `Behavior` class.

```python
from arbitration_graphs import Behavior

class MyBehavior(Behavior):
def __init__(self, "my_behavior"):
super().__init__(name)
```
We re-implemented all of the C++ unit tests in Python, so take a closer look at those for more advanced usage examples.


## Development

Expand All @@ -193,6 +221,9 @@ There, you can edit the source code, compile and run the tests etc.
<details>
<summary>Compiling unit tests</summary>

To compile and run the C++ unit tests, make sure Googletest is available.
For the Python tests, make sure pybind11 is installed.

In order to compile with tests define `BUILD_TESTS=true`
```bash
mkdir -p arbitration_graphs/build
Expand All @@ -201,15 +232,13 @@ cmake -DBUILD_TESTS=true ..
cmake --build . -j9
```

Run all unit tests with CTest:
Run all unit tests (C++ and Python) with CTest:

```bash
cmake --build . --target test
```

</details>


<details>
<summary>Serving the WebApp GUI</summary>

Expand Down Expand Up @@ -287,6 +316,7 @@ This library and repo has been crafted with ❤️ by
Christoph and Piotr coded the core in a pair-programming session.
Piotr also contributed the GUI and GitHub Page.
Nick implemented the awesome PacMan demo and tutorial, with drafting support by Christoph, reviews and finetuning by Piotr.
The Python bindings have been contributed by Nick and reviewed py Piotr.

The repository is maintained by Piotr Spieker&nbsp;
<a href="https://github.com/orzechow" aria-label="View GitHub profile">
Expand Down Expand Up @@ -381,4 +411,4 @@ _Martin Lauer, Roland Hafner, Sascha Lange, and Martin Riedmiller, “Cognitive
}
```

</details>
</details>
114 changes: 114 additions & 0 deletions include/arbitration_graphs/internal/arbitrator_py.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#pragma once

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <yaml-cpp/yaml.h>

#include "../arbitrator.hpp"
#include "../behavior.hpp"

namespace arbitration_graphs::python_api {

namespace py = pybind11;

template <typename CommandT, typename SubCommandT, typename VerifierT, typename VerificationResultT>
class PyArbitratorOption : public Arbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT>::Option {
public:
using BaseT = typename Arbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT>::Option;
using FlagsT = typename BaseT::FlagsT;

explicit PyArbitratorOption(const typename Behavior<SubCommandT>::Ptr& behavior, const FlagsT& flags)
: BaseT(behavior, flags) {
}

std::string toYamlAsString(const Time& time) const {
YAML::Emitter out;
out << BaseT::toYaml(time);
return out.c_str();
}
};

template <typename CommandT, typename SubCommandT, typename VerifierT, typename VerificationResultT>
class PyArbitrator : public Arbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT> {
public:
using BaseT = Arbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT>;

explicit PyArbitrator(const std::string& name = "Arbitrator", const VerifierT& verifier = VerifierT())
: BaseT(name, verifier) {
}

// NOLINTBEGIN(readability-function-size)
CommandT getCommand(const Time& time) override {
PYBIND11_OVERRIDE_PURE_NAME(CommandT, BaseT, "get_command", getCommand, time);
}

void addOption(const typename BaseT::Option::Ptr& behavior, const typename BaseT::Option::FlagsT& flags) {
PYBIND11_OVERRIDE_NAME(void, BaseT, "add_option", addOption, behavior, flags);
}

bool checkInvocationCondition(const Time& time) const override {
PYBIND11_OVERRIDE_NAME(bool, BaseT, "check_invocation_condition", checkInvocationCondition, time);
}

bool checkCommitmentCondition(const Time& time) const override {
PYBIND11_OVERRIDE_NAME(bool, BaseT, "check_commitment_condition", checkCommitmentCondition, time);
}

void gainControl(const Time& time) override {
PYBIND11_OVERRIDE_NAME(void, BaseT, "gain_control", gainControl, time);
}

void loseControl(const Time& time) override {
PYBIND11_OVERRIDE_NAME(void, BaseT, "lose_control", loseControl, time);
}
// NOLINTEND(readability-function-size)
};

template <typename CommandT,
typename SubCommandT = CommandT,
typename VerifierT = verification::PlaceboVerifier<SubCommandT>,
typename VerificationResultT = typename decltype(std::function{VerifierT::analyze})::result_type>
void bindArbitrator(py::module& module) {
using ArbitratorT = Arbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT>;
using PyArbitratorT = PyArbitrator<CommandT, SubCommandT, VerifierT, VerificationResultT>;

using OptionT = typename ArbitratorT::Option;
using PyOptionT = PyArbitratorOption<CommandT, SubCommandT, VerifierT, VerificationResultT>;

using FlagsT = typename OptionT::FlagsT;

py::class_<ArbitratorT, PyArbitratorT, Behavior<CommandT>, std::shared_ptr<ArbitratorT>> arbitrator(module,
"Arbitrator");
arbitrator.def("add_option", &ArbitratorT::addOption, py::arg("behavior"), py::arg("flags"))
.def("options", &ArbitratorT::options)
.def("get_command", &ArbitratorT::getCommand, py::arg("time"))
.def("check_invocation_condition", &ArbitratorT::checkInvocationCondition, py::arg("time"))
.def("check_commitment_condition", &ArbitratorT::checkCommitmentCondition, py::arg("time"))
.def("gain_control", &ArbitratorT::gainControl, py::arg("time"))
.def("lose_control", &ArbitratorT::loseControl, py::arg("time"))
.def("__repr__", [](const ArbitratorT& self) { return "<Arbitrator '" + self.name_ + "'>"; });

py::class_<OptionT, PyOptionT, std::shared_ptr<OptionT>> option(arbitrator, "Option");
option
.def(py::init<const typename Behavior<SubCommandT>::Ptr&, const FlagsT&>(),
py::arg("behavior"),
py::arg("flags"))
.def("has_flags", &OptionT::hasFlag, py::arg("flags_to_check"))
.def(
"to_yaml_as_str",
[](const OptionT& self, const Time& time) {
return static_cast<const PyOptionT&>(self).toYamlAsString(time);
},
py::arg("time"))
.def_readwrite("behavior", &OptionT::behavior_)
.def_readwrite("flags", &OptionT::flags_)
.def_readwrite("verification_result", &OptionT::verificationResult_);

py::enum_<typename OptionT::Flags>(option, "Flags")
.value("NO_FLAGS", OptionT::NO_FLAGS)
.value("INTERRUPTABLE", OptionT::INTERRUPTABLE)
.value("FALLBACK", OptionT::FALLBACK)
.export_values();
}

} // namespace arbitration_graphs::python_api
78 changes: 78 additions & 0 deletions include/arbitration_graphs/internal/behavior_py.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#pragma once

#include <pybind11/pybind11.h>
#include <yaml-cpp/yaml.h>

#include "../behavior.hpp"

namespace arbitration_graphs::python_api {

namespace py = pybind11;

template <typename CommandT>
class PyBehavior : public Behavior<CommandT> {
public:
using BaseT = Behavior<CommandT>;

explicit PyBehavior(const std::string& name) : BaseT(name) {
}

// NOLINTBEGIN(readability-function-size)
CommandT getCommand(const Time& time) override {
PYBIND11_OVERRIDE_PURE_NAME(CommandT, BaseT, "get_command", getCommand, time);
}

bool checkInvocationCondition(const Time& time) const override {
PYBIND11_OVERRIDE_NAME(bool, BaseT, "check_invocation_condition", checkInvocationCondition, time);
}

bool checkCommitmentCondition(const Time& time) const override {
PYBIND11_OVERRIDE_NAME(bool, BaseT, "check_commitment_condition", checkCommitmentCondition, time);
}

void gainControl(const Time& time) override {
PYBIND11_OVERRIDE_NAME(void, BaseT, "gain_control", gainControl, time);
}

void loseControl(const Time& time) override {
PYBIND11_OVERRIDE_NAME(void, BaseT, "lose_control", loseControl, time);
}

std::string to_str(const Time& time,
const std::string& prefix = "",
const std::string& suffix = "") const override {
PYBIND11_OVERRIDE(std::string, BaseT, to_str, time, prefix, suffix);
}
// NOLINTEND(readability-function-size)

std::string toYamlAsString(const Time& time) const {
YAML::Emitter out;
out << BaseT::toYaml(time);
return out.c_str();
}
};

template <typename CommandT>
void bindBehavior(py::module& module, const std::string& bindingName = "Behavior") {
using BehaviorT = Behavior<CommandT>;
using PyBehaviorT = PyBehavior<CommandT>;

py::class_<BehaviorT, PyBehaviorT, std::shared_ptr<BehaviorT>>(module, bindingName.c_str())
.def(py::init<const std::string&>(), py::arg("name") = "Behavior")
.def("get_command", &BehaviorT::getCommand, py::arg("time"))
.def("check_invocation_condition", &BehaviorT::checkInvocationCondition, py::arg("time"))
.def("check_commitment_condition", &BehaviorT::checkCommitmentCondition, py::arg("time"))
.def("gain_control", &BehaviorT::gainControl, py::arg("time"))
.def("lose_control", &BehaviorT::loseControl, py::arg("time"))
.def("to_str", &BehaviorT::to_str, py::arg("time"), py::arg("prefix") = "", py::arg("suffix") = "")
.def(
"to_yaml_as_str",
[](const BehaviorT& self, const Time& time) {
return static_cast<const PyBehaviorT&>(self).toYamlAsString(time);
},
py::arg("time"))
.def_readonly("name", &BehaviorT::name_)
.def("__repr__", [](const BehaviorT& self) { return "<Behavior '" + self.name_ + "'>"; });
}

} // namespace arbitration_graphs::python_api
Loading
Loading