Testing, Debugging and Mocking Python API

[jbflow]

Starting this discussion as it's a problem I've consistently come up against, does any one here have any helpful processes for writing tests and debugging the python scripts.

The whole process of having to restart Live and copy the scripts over is frustrating, and not being able to write any kind of testing because of the dependence on the c_instance of Live is also frustrating. Also not being able to use breakpoints and step through the code makes narrowing down some problems more difficult. Has any one made any headway with trying to mock the Live c_instance so that offline testing could be achieved.

I myself have written two scripts which help me, one for copying remote scripts into the correct location, the other for continuously reading log.txt file

read live logs
copy remote script

[leolabs]

I've got some similar scripts in this project's package.json file. The one I use most is ableton11:start, which kills Live if it's currently running, copies the script over to the user library, starts Live again, and then streams the log files filtered to just messages from AbletonJS.

Unfortunately, I haven't found a way to properly debug plugins running in Live's environment, other than adding lots of log statements and hoping that helps 😅

[jbflow]

Ah very nice thank you, I think I overlooked these scripts when creating my PR, so I will amend.

With logging, this is the approach I take, but it is a bit frustrating. It would be nice if we could use breakpoints and be able to step through the code. Some way of mocking the Ableton c_instance would make writing tests and debugging much simpler.

I'm sure Ableton must have some internal tools they use for this process, I'm wondering if we could politely ask them to release some of their tools to us third party developers.

[jbflow]

@leolabs found this

[jbflow]

@MartinSahlen Yeah brilliant idea, it definitely needed doing, thank you for your work,. I have copied it into my current working directory simply for the auto complete functionality. Thanks for sharing the AbletonOSC resource, I will check it out today as that will streamline my process quite a bit.

I am working on a project that takes a snapshot of the live set as JSON and stores it in the browser - It does this using the
listeners and sends the data over MIDI - everything from track and device names to browser items etc. Yeah as you mention I'm thinking that by combining this kind of JSON structure for a "live set config" with the work you have done for the Live classes it should be possible to essentially mock the Ableton Live API in pure Python to run completely separate without Live even running. By doing that it should then be possible to actually write some proper unit tests for new scripts and have these run by the copy script functionality then only copy if all tests pass but also remove the tests from the copy process so they're aren't run in place. It would be an undertaking but it should be possible.

I'm sure Ableton are going to have some kind of internal tools like this though.

[jbflow]

This project uses a UDP socket to connect from Node and I am currently working on this PR to add a TCP socket so that it's possible to connect from a websocket.

[jbflow]

I'm going to see if my idea of mocking works today with at least a basic implementation to see if I can write a simple unit test, I will publish what I come up with somewhere.

[MartinSahlen]

Got it. By the way, I did actually create a much improved OSC library for python: https://github.com/oslo1989/fastosc, that is much more pythonic and easy to use with decorators for osc addresses and the ability to build routers that are composed of routers so you can easily build more complex OSC apis. It's very mucn inspired by FastAPI and how it enforces good structure and encapsulation. It also creates listeners by default. I like this better as the code is more verbose but still fairly fast to write, compared the other project I shared which does a lot of things that is hard to follow. This is of course more about my preferences but I do think striking a balance to avoid using too much of the dynamic python stuff is better if you want to sustain a project for longer since everything is easy to follow with autocomplete etc.

you can have a look here to see how it's used: https://github.com/oslo1989/ableton-control-surface-osc/blob/main/AbletonControlSurfaceOSC/live/song/tempo.py. I initially created everything from the same codebase bit then I thought that the OSC functionality is probably useful standalone as well.

By the way, when it comes to mocking, there does seem to be something here inside the ableton folder but I could never find much useful as it's pyc files.

Also regarding using the code - you can also just add the code with pip as well, if you're using that: https://pypi.org/project/ableton-control-surface-core.

[jbflow]

Ah nice find, I'll check all this out today, thank you