Feature Request - Remote Control/Usage

[JojoDevel]

Why?

Our microscopes (mostly NikonTi Eclipse) are controlled by computer just next to them running windows not always on the most up-to-date version and sometimes even air-gapped form the internet. Therefore, we usually try to touch these machines as little as possible, install as little software as possible and let them only interface with the microscope.

What?

We would like to run the napari-micromanager on any other computer in our lab or institute and attach remotely to the running microscope control computer. This would allow us to much easier maintain/develop software, control imaging loops easier, and maybe even work remotely from our labs and dial into running image acquisitions from all over the world...

How?

Not sure yet. I imagine a solution with as little programming overhead as possible but keeping the same functionality as right now. So the developer should not feel that everything goes through a remote connection (Though speed will possibly slower).

There is for example, pyRO that allows to execute functions remotely. Maybe we could turn every device in the pymmcore python api into such a remote object. And then the python code just uses calls that are executed remotely. On the microscope computer then a pyro server would run initiated from a micromanager configuration. I never used the framework though.

So, there might be also be better opportunities to implement that. Any ideas?

Hint: We also discussed that on the EPFL Symposium for Smart Microscopy 2024

[tlambert03]

hey @JojoDevel, if it's ok with you, I'm going to move this issue to the pymmcore-plus repo, since ultimately most of the changes to make this possible would need to happen there.

We actually did have a pyro-backed remote communication setup in the beginning, but removed it (in this PR: pymmcore-plus/pymmcore-plus#208) due to excess and unused complexity. However... it shouldn't be too hard to get some sort of proof of principle back up and working again, and it would be great if you could test it a bit to let me know if it is a step in the right direction for your needs.

[tlambert03]

hey @JojoDevel, I extracted the logic from pymmcore-plus/pymmcore-plus#208 and created a new package: https://github.com/pymmcore-plus/pymmcore-remote

when you have a moment, it would be great to get your eyes on it to see if it's going in a direction that would be useful to you. See the readme there for details on installing, and using

[JojoDevel]

Yes, this looks very good 👍 Great work!
Is there a way to get it to full network serialization? So that no shared memory is used and communication goes through network?

But maybe we should move the issue over to the new repo...

[tlambert03]

Yes actually i found a good solution with msgpack-numpy and will use it 👍

I tried briefly to use it with napari micromanager and there are still some remaining issue with serialization of certain custom objects, so I'll keep working on it and let you know when I get a client side gui working with a remote core

[tlambert03]

[I moved this issue again here to pymmcore-remote. I'm sure there will be multiple issues that arise as we test it, and they can hopefully be largely contained in this repo. Though some changes may need to be made in other repos]

[JojoDevel]

Yes actually i found a good solution with msgpack-numpy and will use it 👍

I tried briefly to use it with napari micromanager and there are still some remaining issue with serialization of certain custom objects, so I'll keep working on it and let you know when I get a client side gui working with a remote core

This would be pretty amazing for us and that's exactly what we were looking for 🤩 Then we no longer have to do requests wrapped into a REST API and can use the full ecosystem. But also contribute our devs directly to the napari micromanager gui :)

[tlambert03]

Yeah, I agree, it would be a big win all around for this to be a relative invisible change in the front end.

The biggest issue I'm facing for the gui right now is #4
Client side signal callbacks that invoke server side core methods aren't working, which is going to be necessary for the gui stuff to work. I think it's a threading issue, but still need to figure it out

[tlambert03]

hey @JojoDevel, here's a promising development. proof of principle here using @fdrgsp's https://github.com/fdrgsp/micromanager-gui (but could be napari as well) showing 4 different "client-side" applications open (in 4 different processes, not including the server process itself). anything we do in one window is communicated to all the other windows via the server. So, things are going in a good direction!

Screen.Recording.2024-11-12.at.3.55.26.PM.mov

[JojoDevel]

This looks fantastic 🤩 🚀