Parallelism support: PoC/Suggestion #1694
Conversation
In order to maintain backwards compatibility, we need to provide stubs for functionality needed for parallelism when no libraries provide this. Also, to keep possibly diverging parts to a minimum, the utilities for parallelism are kept in a separate module. This commit is a sketch of how we could go about it. In particular, it is not clear if `ppx_optcomp` is the best solution for this.
| @@ -0,0 +1,62 @@ | |||
| [%%if ocaml_version >= (5, 0, 0)] | |||
| (* Simple Mutex Implementation using Domain-Local Await (https://github.com/ocaml-multicore/domain-local-await) | |||
There was a problem hiding this comment.
It's described as
At the time of writing this, the Stdlib
Muteximplementation does not take into account the possibility of having an effects based scheduler and simply blocks the current domain (or (sys)thread) without giving a potential scheduler the opportunity to schedule another fiber on the domain.
Is this actually needed? Do the parallel solvers also use effects-based schedulers per-domain?
There was a problem hiding this comment.
At the time of writing it was needed for solvers using the domainslib library. I have included this as it is one of the simplest examples.
I am currently evaluating this and other workarounds that were implemented at the time, so this in particular might be replaced by the Stdlib or Batteries Mutex.
However, even then we will need to benchmark this. With 5.0.0 even mutexes that were guaranteed to always be free caused slowdowns. If this is still the case, a No-Op mutex could still be useful. (Of course, we can also go the middle-way and configure the behaviour of the Mutex at runtime)
There was a problem hiding this comment.
Yes, the used Domainslib uses an internal scheduler. This incompatibility was something that took us a while to track down in one of our solvers (Helmut's variant iirc):
See also ocaml-multicore/domainslib#127
|
This should be superseded by #1708. |
This PR should provide a base for discussion.
In order to maintain backwards compatibility, we need to provide stubs for functionality needed for parallelism when no libraries provide this. Also, to keep possibly diverging parts to a minimum, the utilities for parallelism are kept in a separate module.
This commit is a sketch of how we could go about it. In particular, it is not clear if
ppx_optcompis the best solution for this.