Tools for adaptive imaging in the LEB-EPFL
This project was used for the realization of event-driven acquisition (EDA) imaging of mitochondria and caulobacter. The two main modules are Mito2Drp1.py, NetworkWatchdog.py and NNGui.py.
The overall idea of this project is to adapt the imaging speed of a microscope to the presence of events in the images. This is achieved by implementing a feedback loop from the images taken to the microscope control. NetworkWatchdog implements part of the feedback loop, while NNGui can be used to show the resulting data. The steps of the Feedback loop are:
- Micro-Manager saves the images to a network location
- The files are detected by NetworkWatchdog
- NetworkWatchdog computes a decision parameter
- A binary file with this information is written to the network location
- The Matlab program controlling the microscope reads that data
This will result in data with different framerates, which makes for an interesting problem for actually displaying the data. Hence NNGui.
Some of the important settings are in ATS_settings.json. This was used to synchronize settings on different machines that were used to run the different files.
Running EDA requires Micro-Manager 2.0, Python 3.8 and can be run on any operating system supporting these softwares. The software was developped and tested on Windows 10 (version 10.0.19043 Build 19043), in combination with the freely available software Micromanager 2.0 beta (version 2.0.0-beta3 20180905). Custom written code in Python 3.8 was used for the event detection framework. Python modules used for its implementation include tensorflow 2.3, keras2.6, pyqt5 and pyqtgraph 0.11.1. Tensorflow GPU support requires a NVIDIA card with CUDA® architectures 3.5, 5.0, 6.0, 7.0, 7.5, 8.0 and higher than 8.0. Typical installation time on a "normal" desktop computer should take ~30 minutes.
Module that is used to generate the models used by EDA. The training data is data of mitochondria and Drp1 from a SIM microscope. The ground truth is a map of division events generated by Dora Mahecic using a Hessian filter on the mitochondria data and the Drp1 data plus some manual annotaion.
This module looks for new files in a network location. If the files are tif files saved from Micro-Manager, the files will be prepared for the neural network. The neural network gives a heatmap of possible events in the data and the maximum of that value is saved to the binary file.
NNGui provides a GUI to look at the data produced by EDA. It loads a .h5 keras model for inference on the provided images. The images can be tif stacks or folders with the corresponding images in it. Especially images in folders are expected to have Micro-Manager like filenames (img_channel000_position000_time000000000_z000.tif). Stacks are usually interleaved with the two channels that the network takes.
The GUI displays both the original images and the images how they have been prepared for the network. Additionally there is a view for the heatmap produced by the neural network and a plot to show the data and framerates. Views can be hidden to enhance performance and the order in which the data is present in the images can be chosen.
Module to simulate the behavior of EDA imaging on data taken at a fixed framerate.
...