-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GSoC 2024] Binding PyElastica with Elastica++ #391
[GSoC 2024] Binding PyElastica with Elastica++ #391
Conversation
- Moves the metadata from setup.py to pyproject.toml
- brigand and cxxopts resolved via meson wraps - blaze, blaze_tensor and sleef are installed via a custom shell script that's called from the meson build
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## elasticapp-integration #391 +/- ##
=======================================================
Coverage 93.98% 93.98%
=======================================================
Files 51 51
Lines 3193 3193
Branches 347 347
=======================================================
Hits 3001 3001
Misses 142 142
Partials 50 50 ☔ View full report in Codecov by Sentry. |
Errors I'm getting when trying to compile (click to expand) C++ compiler error log
UPDATE: this issue has been fixed now |
- Updated benchmarking and test scripts accordingly
…into ankith26-gsoc
- Also adds corresponding test and benchmarking script
…into ankith26-gsoc
We'll merge this |
GSoC 2024 Project PR
GSoC experience, learnings and acknowledgements
I'd like to thank all developers of PyElastica, the members at Python Software Foundation, GSoC admins at Google and everyone else who's work has gone behind making my GSoC project possible. This was indeed a great learning opportunity for me. Over the course of the project I got to learn and use various tools, in particular, this project helped me better understand using
pybind11
in C++.I had a great experience contributing to this project. Special thanks to all my mentors, who were very helpful, active and understading during this project. It goes without saying that their guidance and support has been crucial for this project.
Objective
The goal of the project is to implement bindings for C++ based
Elastica++
to the python basedPyElastica
, to achieve speedups. It covers selecting a suitable C++ build system and setting up the compilation architecture, writing binding code usingpybind11
or alternative methods, benchmarking against existing Python implementations, expanding unit tests, polishing documentation and typestubs, and updating the CI pipeline for building and testing on multiple Python versions.Summary of what was done
Rod
modules #341meson
based buildsystem set up for handling the building. I set up a script to autoinstall and manage the dependencies that are needed for compilation.difference_kernel
,batch_matmul
,batch_matvec
,batch_cross
,batch_dot
,batch_norm
).Vector
/Matrix
/Tensor
objects in theelasticapp
backend.rotate
andinv_rotate
).CosseratRod
class usable in my branch. However, this needs more work before it is completely usable.I also have a week-wise breakdown of what I did, available here:
Week-wise breakdown (click to expand)
Week 1 (May 27 - June 2)
This week, I'm mostly reading up on and experimenting with build systems.
scikit-build-core
(which usescmake
), removing the currentsetup.py
in the process. The metadata from the removedsetup.py
has been ported topyproject.toml
meson-python
(which usesmeson
) based system that also integrates with a pybind11 example, to serve as a POC. This example compiles and is usable from python code.The goal this week is to experiment and compare both systems side by side in action, and later pick and polish the more suitable option (based on mentor feedback)
Week 2 (June 3 - June 9)
This week I played around with dependency handling. I was keen to look into the "meson wraps" system, but I quickly hit into a couple of issues with this approach.
For one, to install
blaze
andblaze_tensor
, I had to pass custom arguments to the underlyingcmake
build, which was not possible via the simple meson wraps interface. I also had other troubles/errors while trying to make wraps I had written work for the other dependencies. The only dependencies that worked via wraps werebrigand
andcxxopts
.To handle
blaze
,blaze_tensor
andsleef
I wrote a custom shell script that can clone, build and install them in a deps directory in the source tree. I got the meson buildconfig to work with these dependencies.Week 3 (June 10 - June 16)
I experimented with and implemented the
batch_matmul
linalg function in different ways in C++ for testing and benchmarking purposes. Added theelasticapp._linalg
submodule for the same.Also added a simple benchmarking script in the
backend/benchmarks
folder.Week 4 (June 17 - June 23)
I wrapped 6 linalg functions that were in
Utilities/Math/BlazeDetail/BlazeLinearAlgebra.hpp
in the pythonelasticapp._linalg
module and also added tests and basic docs for the same.I spent a while trying to find out more optimal ways to do the wrapping, and I could not figure out a way to improve upon my current naive approach (which involves a lot of copying in input/output arrays).
Week 5 (June 24 - June 30) and Week 6 (July 1 - July 7)
I first integrated the
Vector
/Matrix
/Tensor
object wrapping implemented inUtilities/Math/Python
(with a couple of minor fixes done to get it compiling) in the python_PyArrays
submodule. I then updated the linalg wrappings to make use of these objects directly. This lead to the expected performance enhancement, as the problem of performance hits due to unneeded array copying was solved.Updated the benchmarks and tests accordingly to measure the performance enhancement and test correctness of the new approach, respectively.
Finally, to wrap up the first half of my project, I documented the setup implemented so far (with installation, testing and benchmarking instructions)
Week 7 (July 8 - July 14)
batch_cross
operation.inv_rotate
rotation functionality (both SIMD and scalar variants) while handling thespan_vector
argument as an optional overload. Also added the corresponding tests and benchmarking (observed a consistent improvement here).Week 8 (July 15 - July 21)
I binded the
rotate
functionality inelasticapp._rotations
along with the corresponding benchmarking and testing.I also started looking into the
CosseratRod
bindings.Week 9 (July 22 - July 28) and Week 10 (July 29 - August 4)
I continued experimenting with
CosseratRod
bindings, but this turned out to be a fairly involved task. With tens of thousands of lines of rod related code available, I'll admit I couldn't make much sense of it.But I could fix the compilation by commenting out the problematic code and then applying a few minor patches. At this point, a CosseratRod class is importable in python but not very usable (I couldn't figure out a constructor for the class)
Week 11 (August 5 - August 11)
After I got stuck on issues last week, @skim0119 implemented the
_CosseratRodBlock
bindings. I fixed a couple of failing unit tests as a result of the new changes, by fixing_PyTags
link-time issues.I also added a test script to test the construction and attributes of the
_CosseratRodBlock
on the python side.Week 12 (August 12 - August 18) and the Final week (August 19 - August 26)
In these weeks I'm focusing on cleanups, documentation and other general minor enhancements to wrap up my project.
What's next
Elastica++ is a large codebase and there are many more submodules that can be integrated into the backend. Many of the submodules were out of scope for this project. The buildsystem can be expanded to handle more dependencies like
oneTBB
that can be used to add threading/parallelism to further speed up the operations at the C++ level.