Subclassing Basic

[ellisonbg]

@certik hi, hope you are well! Looks like symengine is going well. I am looking at symengine for the sympy.physics.quantum stuff. Some questions:

• In sympy.physics.quantum we subclass Basic here (https://github.com/sympy/sympy/blob/master/sympy/physics/quantum/qexpr.py). Is it possible to do that type of thing on top of the python bindings to symengine? Is there an examples like that?
• Is it even worth rebuilding the quantum stuff on top of symengine at a deep level?
• How would you go about beginning to use symengine for that type of thing where we have lots of custom subclasses of Basic.
• How are assumptions handled in symengine?

[ellisonbg]

I see this PR that begins to use symengine in physics.quantum, but it isn't obvious how much benefit there will be...

https://github.com/sympy/sympy/pull/13189/files

[certik]

Hi @ellisonbg, I am doing great, I hope you are well too. Thanks for the interest. To answer your questions:

• We would need to see. The symengine's Basic class is here:

  symengine.py/symengine/lib/symengine_wrapper.pyx

  Line 760 in 3fd1e48

  cdef class Basic(object):

  , I think you can subclass it in Python. But I don't know what exact things you need for quantum.

• That depends if the speed of sympy.quantum is sufficient. If not, where is the bottleneck? If SymPy operations, then it makes sense to port on top of symengine.

• As you mentioned, Use SymEngine in Physics/Quantum sympy/sympy#13189 is a start. Then we'll go from there.

• We haven't implemented assumptions yet, I just made an issue for it: Implement assumptions handling symengine#1334

In general, I would add, that even though you were initially reserved about using C++ for symengine, it worked really well. The main library (https://github.com/symengine/symengine) is just C++, no Python, and it's pretty clean, and as you can see, lots of people have contributed to it, and students in general have no problems getting up to speed. The modern subset of C++ that we use allows us to be pretty much safe in Debug mode, just like in Python, for all practical purposes. In Release mode there doesn't seem to be any sacrifices in speed that we had to make. It is also completely multiplatform, it works with all major compilers. The main thing however is the speed (in Release mode). I want to implement more systematic benchmarks (symengine/symengine#1318), but from what we ran by hand, it looks like we are one of the fastest, if not the fastest.

So if you need speed, symengine is the way to go. For quantum, do you have some test cases which you want to make faster? One can implement that in C++, then one can concentrate on the speed, data structures, algorithms, and nail it. It's work, but if speed is what you want, I think that's the best way to go. If you only need fast symbolics, then use symengine from Python, and build the rest on top in Python. That's much easier, and delivers most of the time.

[ellisonbg]

Thanks for the response! Some of the things we need to do for quantum: * A custom `__new__(cls, *args)` method where we process args, instantiate using `Expr.__new__` and then set some slots. * A custom `doit` method (is that supported?) * A custom `_eval_rewrite` method * Other custom `_eval` methods such as `_eval_inverse`, `_eval_power`, `_eval_adjoint`, `_eval_trace`. Some of these are for base sympy logic to work, others are custom to quantum. * All the custom print methods for pretty, latex, etc. that return strings (not print to stdout) * Custom `__mul__`. Speed might be an issue eventually for certain types of algorithms in quantum computing, but I don't want to start in C++. The speed critical will be very localized and we could drop into C++ at that point if needed. I just want to make sure our custom Basic subclasses in Python will have the right information so that when we do drop into C++, it will all work (without us having to go back and implement everything in C++). The bigger place we struggled in building the quantum stuff was in the actual overall model and APIs of sympy. I am fine if the overall model is the same as sympy, but that stuff doesn't seem to be documented anywhere. The most subtle stuff was things like: * All the magic in `__new__` with @classmethods that process args. * Attempting to have custom multiplication logic - the combination of python's model for `__mul__` and sympy made this extremely subtle. * Assumptions * No clear APIs and policies for when things are evaluated (in __new__ versus in methods such as doit). * Documentation about the protocols (such as _eval_*) that objects need to implement to work with other parts of the code base. As always, I care much more about those things than speed. But we can't fully ignore speed either. Thanks!

…

On Thu, Sep 28, 2017 at 9:00 AM, Ondřej Čertík ***@***.***> wrote: Hi @ellisonbg <https://github.com/ellisonbg>, I am doing great, I hope you are well too. Thanks for the interest. To answer your questions: - We would need to see. The symengine's Basic class is here: https://github.com/symengine/symengine.py/blob/ 3fd1e48/symengine/lib/ symengine_wrapper.pyx#L760 <https://github.com/symengine/symengine.py/blob/3fd1e481cf2d3f6b37ec72f8ba89b75ced587283/symengine/lib/symengine_wrapper.pyx#L760>, I think you can subclass it in Python. But I don't know what exact things you need for quantum. - That depends if the speed of sympy.quantum is sufficient. If not, where is the bottleneck? If SymPy operations, then it makes sense to port on top of symengine. - As you mentioned, sympy/sympy#13189 <sympy/sympy#13189> is a start. Then we'll go from there. - We haven't implemented assumptions yet, I just made an issue for it: symengine/symengine#1334 <symengine/symengine#1334> In general, I would add, that even though you were initially reserved about using C++ for symengine, it worked really well. The main library ( https://github.com/symengine/symengine) is just C++, no Python, and it's pretty clean, and as you can see, lots of people have contributed to it, and students in general have no problems getting up to speed. The modern subset of C++ that we use allows us to be pretty much safe in Debug mode, just like in Python, for all practical purposes. In Release mode there doesn't seem to be any sacrifices in speed that we had to make. It is also completely multiplatform, it works with all major compilers. The main thing however is the speed (in Release mode). I want to implement more systematic benchmarks (symengine/symengine#1318 <symengine/symengine#1318>), but from what we ran by hand, it looks like we are one of the fastest, if not the fastest. So if you need speed, symengine is the way to go. For quantum, do you have some test cases which you want to make faster? One can implement that in C++, then one can concentrate on the speed, data structures, algorithms, and nail it. It's work, but if speed is what you want, I think that's the best way to go. If you only need fast symbolics, then use symengine from Python, and build the rest on top in Python. That's much easier, and delivers most of the time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#194 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABr0MCuef3TnZmmG9mOX3RwIEwLV1yAks5sm8KjgaJpZM4Pmsii> .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub [email protected] and [email protected]

[certik]

@ellisonbg I see. Your main motivation is to ensure that the SymEngine design allows the incorporation of the quantum stuff, if that was ever needed in the future, so that we don't run into a dead end.

It seems most of the stuff is with relation to the Python wrappers. Few of it requires interaction on the C++ level. The C++ level allows to accept any "unknown" function implemented for example in Python, here is a test which takes SymPy's spherical harmonics (not yet implemented in C++) and passes it to the C++ symbolic expression:

symengine.py/symengine/tests/test_functions.py

Line 123 in b03f044

@unittest.skipUnless(have_sympy, "SymPy not installed")

On the C++ level, that is tested here:

https://github.com/symengine/symengine/blob/fdf132fcb4425589b69b40d60a90234944870b28/symengine/tests/basic/test_functions.cpp#L3702

where you can see that we defined MySin, implemented in that test file, and pass it to a SymEngine C++ expression.

And we use something similar with Lambdify I believe.

So we should go over the list from your last comment, and write tests for each of these things, to ensure that our API and our design supports these idioms (both in C++ and in the Python wrappers), if they were ever needed in the future.