Merge pull request #46 from ifm/develop

Version 1.0.0

.readthedocs.yml

@@ -4,20 +4,25 @@
 # Required
 version: 2
 
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.9"
+    # You can also specify other tool versions:
+    # nodejs: "16"
+    # rust: "1.55"
+    # golang: "1.17"
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
   configuration: doc/manual/conf.py
 
-# Build documentation with MkDocs
-#mkdocs:
-#  configuration: mkdocs.yml
-
 # Optionally build your docs in additional formats such as PDF
 formats:
   - pdf
 
-# Optionally set the version of Python and requirements required to build your docs
+# Optionally declare the Python requirements required to build your docs
 python:
-  version: 3.7
   install:
     - requirements: doc/requirements.txt

README.md

@@ -8,7 +8,7 @@
 - A state machine guarantees consistent behaviour in initialization, active and shutdown phases across plugins and threads.
 - Non-intrusive design. Unlike other frameworks, nexxT tries its best to leave the developers the freedom they need. No directory structures are predefined, no build tools are required, the data formats are not predefined.
 - Rapid prototyping of algorithms using python, both online (i.e. using an active sensor) and offline (i.e. using data from disk).
-- Visualization can be done using python visualization toolkits supporting QT5.
+- Visualization can be done using python visualization toolkits supporting QT6.
 - Efficient pipelines can be built without interacting with the python interpreter by using only C++ plugins.
 - The open source license gives freedom to adapt code if necessary.
 - Cross platform compatibility for windows and linux.
@@ -41,22 +41,48 @@ Assuming that you have a python3.7+ interpreter in your path, the installation i
     python -m pip install pip -U
     pip install nexxT
 
-## Building from source
+## Porting from nexxT 0.x to nexxT 1.x (aka PySide2 to PySide6)
+
+### Python
+
+The main change for nexxT 1.x is the update from QT5/PySide2 to QT6/PySide6. For flexibility reasons, nexxT now provides a meta package nexxT.Qt, which can be used instead of PySide6. So it is now recommended to replace
+
+    from PySide2 import xyz
+    from PySide2.QtWidgets import uvw
 
-Building from source requires a QT5 installation suited to the PySide2 version used for the build. It is ok to use 5.14.0 to build against all versions 5.14.x of PySide2 because of QT's binary compatibility. You have to set the environment variable QTDIR to the installation directory of QT. Note that this installation is only used during build time, at runtime, nexxT always uses the QT version shipped with PySide2.
+with
 
-On linux, you will be caught by the following known bug from QT: https://bugreports.qt.io/browse/QTBUG-80922?focusedCommentId=558603&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-558603. Apply the fix mentioned in the linked comment.
+    from nexxT.Qt import xyz
+    from nexxT.Qt.QtWidgets import uvw
+
+In the future, this approach might be also used to support PyQt6, so using nexxT.Qt is recommended over the also possible direct usage of PySide6. Note that the implementation of nexxT.Qt imports the PySide moduels on demand using sys.meta_path, so unused QT modules are not loaded.
+
+Note the porting guide of PySide6: https://doc.qt.io/qtforpython/porting_from2.html: QAction and QShortcut have been moved from QtWidgets to QtGui.
 
-On linux, you will also need llvm and clang installed (because of the shiboken2 dependency). You might need to set the environment variable LLVM_INSTALL_DIR.
+### C++
 
-The following commands build nexxT from source using the non-recommended pip package of shiboken2-generator.
+For c++, nexxT includes shall be prefixed with nexxT/, for example
+
+    #include "Filters.hpp"
+
+has to be replaced with
+
+    #include "nexxT/Filters.hpp"
+
+## Building from source
+
+Building from source requires a QT6 installation suited to the PySide6 version used for the build. It is ok to use 6.4.0 to build against all versions 6.4.x of PySide6 because of QT's binary compatibility. You have to set the environment variable QTDIR to the installation directory of QT. Note that this installation is only used during build time, at runtime, nexxT always uses the QT version shipped with PySide6.
+
+On linux, you will also need llvm and clang installed (because of the shiboken6 dependency). You might need to set the environment variable LLVM_INSTALL_DIR.
+
+The following commands build nexxT from source using the non-recommended pip package of shiboken6-generator.
 
     git clone https://github.com/ifm/nexxT.git
     cd nexxT/workspace
     python3 -m venv venv
     source venv/bin/activate
     python3 -m pip install pip -U
-    pip install -r requirements.txt --find-links https://download.qt.io/official_releases/QtForPython/shiboken2-generator/
+    pip install -r requirements.txt --find-links https://download.qt.io/official_releases/QtForPython/shiboken6-generator/
     export QTDIR=<path>/<to>/<qt>
     export LLVM_INSTALL_DIR=<path>/<to>/<llvm>
     scons -j 8 ..
@@ -71,6 +97,3 @@ Originally we started with a commercial product from the automotive industry in
 - The data transport layer of *ROS2* seemed not to fulfill our requirements. We have often use cases where we record data to disk and develop algorithms using that data. Because *ROS2* is mainly designed as a system where algorithms run online, its assumptions about the data transport is low latency in prior of reliability. If in question, *ROS2* decides to throw away messages, and this is very bad for the offline/testing usage when you have not such a big focus on algorithm runtime but maybe more on algorithm quality performance. In this use-case it is a reasonable model that slow algorithms shall be able to slow down the computation, but at the time of evaluation this was not easily possible in *ROS2*. A discussion about this topic can be found here: https://answers.ros.org/question/336930/ros2-fast-publisher-slow-subscriber-is-it-possible-to-slow-down-the-publisher/
 - It's not easily possible to start two *ROS2* applications side by side.
 
-## Current Status
-
-The current status is still in an early phase. We use the framework in newer projects, but there is still the chance for API-breaking changes. Some aspects like error handling are still a little rough.

doc/c++/Doxyfile

@@ -790,13 +790,15 @@ WARN_LOGFILE           =
 # spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
 # Note: If this tag is empty the current directory is searched.
 
-INPUT                  = ../../nexxT/src/DataSamples.hpp \
-                         ../../nexxT/src/Filters.hpp \
-                         ../../nexxT/src/Ports.hpp \
-                         ../../nexxT/src/PropertyCollection.hpp \
-                         ../../nexxT/src/Services.hpp \
-                         ../../nexxT/src/Logger.hpp \
-                         ../../nexxT/src/NexxTPlugins.hpp
+INPUT                  = ../../nexxT/src
+#                         ../../nexxT/src/DataSamples.hpp \
+#                         ../../nexxT/src/Filters.hpp \
+#                         ../../nexxT/src/Ports.hpp \
+#                         ../../nexxT/src/PropertyCollection.hpp \
+#                         ../../nexxT/src/Services.hpp \
+#                         ../../nexxT/src/Logger.hpp \
+#                         ../../nexxT/src/NexxTPlugins.hpp \
+#                         ../../nexxT/src/SharedPointerTypes.hpp
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

doc/manual/conf.py

@@ -65,7 +65,7 @@
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = 'en'
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
@@ -230,6 +230,12 @@ def skip(app, what, name, obj, would_skip, options):
                 "DESTRUCTED"]:
         # skip the state machine entries
         return True
+    if name == "Services":
+        if not hasattr(skip, "srvseen"):
+            skip.srvseen = set()
+        if obj in skip.srvseen:
+            return True
+        skip.srvseen.add(obj)
     return would_skip
 
 def remove_apidoc_artifacts(app):

doc/manual/nexxT.rst

@@ -11,6 +11,9 @@ Subpackages
    autogenerated/nexxT.services
    autogenerated/nexxT.filters
    autogenerated/nexxT.examples
+   autogenerated/nexxT.Qt
+   autogenerated/nexxT.QtMetaPackage
+   autogenerated/nexxT.shiboken
 
 Module contents
 ---------------

doc/manual/tutorial.rst

@@ -149,6 +149,31 @@ Now it's time to save the configuration in the tool bar and test it. Initialize
 
 .. image:: example-first-app-running.png
 
+Thread cycles and Deadlocks
++++++++++++++++++++++++++++
+
+When you get error messages like
+
+.. code-block:: console
+
+    nexxT.core.ActiveApplication: This graph is not deadlock-safe. A cycle has been found in the thread graph: main->compute->main
+
+you have tried to create an application which is potentially deadlocking. In the message above, it is stated that
+there is a dependency cycle in the filters of the threads main and compute. This is the corresponding filter graph:
+
+.. image:: example-deadlock.png
+
+The reason for the possible deadlocks is that nexxT by default uses a semaphore to control the number of pending samples in inter-thread connections. When transmitting a sample to another thread which has not yet processed the last transmitted sample, the transmitting thread blocks until the last transmitted sample has been received by the receiving thread. This behaviour might cause deadlocks in the presence of cycles, nexxT detects these cycles and refuses to execute these applications.
+
+There are multiple solutions for this issue:
+
+- move filters to other threads. The above examples gets deadlock safe when moving the filter *filt_gui* from the gui thread (green) to the compute thread (dark-blue).
+- Moving all filters to the main thread is always a solution, but this might be too slow.
+- Use non-blocking connections for specific inter-thread connections (right-click on a connection and select *Set non blocking*). Non-blocking connections do not check for pending data samples, so they cannot cause deadlocks. Non blocking connections are displayed in red in the filter graph:
+
+.. image:: example-deadlock-fixed.png
+
+Note that non-blocking connections come with the risk that a potentially infinite amount of data is pending on inter-thread connections. This might cause high latency or even out-of-memory situations. Therefore, non-blocking connections are not recommended to be used at high data rate connections. Output ports triggered by sporadic events are best suited for non-blocking connections.
 
 Developer Perspectives
 ----------------------
@@ -256,13 +281,13 @@ The onClose(...) method is the inverse of onOpen(...). It releases the widget fr
 C++
 +++
 
-**What are the benefits of using C++ instead of python?** While you can argue that performance is not affected much if the filter only uses a wrapper around a library such as opencv, the `python GIL <https://wiki.python.org/moin/GlobalInterpreterLock>`_ is a factor which might limit performance in a multithreaded application like nexxT. In a nutshell it means, that whenever python code is executed, the interpreter has the GIL locked to prevent other threads from modifying interpreter states. C extensions like numpy, opencv or PySide2 unlock the GIL during long-duration calls. As a consequence, heavily using pure python will slow down other threads because the GIL limits parallel execution. Using C++ filters, it is possible to design operations which are not affected by the GIL at all.
+**What are the benefits of using C++ instead of python?** While you can argue that performance is not affected much if the filter only uses a wrapper around a library such as opencv, the `python GIL <https://wiki.python.org/moin/GlobalInterpreterLock>`_ is a factor which might limit performance in a multithreaded application like nexxT. In a nutshell it means, that whenever python code is executed, the interpreter has the GIL locked to prevent other threads from modifying interpreter states. C extensions like numpy, opencv or PySide6 unlock the GIL during long-duration calls. As a consequence, heavily using pure python will slow down other threads because the GIL limits parallel execution. Using C++ filters, it is possible to design operations which are not affected by the GIL at all.
 
 Filters in C++ are very similar to filters in python. They are defined using a class inheriting from :cpp:class:`nexxT::Filter` and overwriting the same methods just like in python. One difference is the usage of nexxT services like the MainWindow service (see :ref:`tutorial:Display filters`). In C++, these services are of type QObject. Therefore, you need to use `QMetaObject::invokeMethod <https://doc.qt.io/qt-5/qmetaobject.html#invokeMethod>`_ for accessing slots of the services.
 
-The plugin library links against the nexxT runtime library (*libnexxT.so* or *nexxT.dll*) which is provided in nexxT installation directory. It also links against a QT library used for development. Note that during runtime, the QT library bundled with PySide2 will be used regardless of which QT library has been used to develop. To be on the safe side, you should use a matching major.minor version, the patch level should be non-relevant. For example, to compile a plugin for the PySide2 version 5.14.2.3, you can use QT 5.14.0. Plugin libraries do not use shiboken2 for exposing the filters in python, instead they use a QLibrary interface.
+The plugin library links against the nexxT runtime library (*libnexxT.so* or *nexxT.dll*) which is provided in nexxT installation directory. It also links against a QT library used for development. Note that during runtime, the QT library bundled with PySide6 will be used regardless of which QT library has been used to develop. To be on the safe side, you should use a matching major.minor version, the patch level should be non-relevant. For example, to compile a plugin for the PySide6 version 5.14.2.3, you can use QT 5.14.0. Plugin libraries do not use shiboken2 for exposing the filters in python, instead they use a QLibrary interface.
 
-Note that - unlike pure QT - PySide2 does not provide any compatibility guarantees between minor or patch level releases. This means that it is generally not possible to use nexxT with a different PySide2 version than it was compiled against. 
+Note that - unlike pure QT - PySide6 does not provide any compatibility guarantees between minor or patch level releases. This means that it is generally not possible to use nexxT with a different PySide6 version than it was compiled against. 
 
 Each plugin library can announce one or more filter classes.
 

doc/requirements.txt

@@ -1,8 +1,10 @@
 -e .
+-e .
 Sphinx==3.3.1
-sphinx-rtd-theme==0.5.0
+sphinx-rtd-theme==1.1.1
 sphinxcontrib-apidoc==0.3.0
 breathe==4.25.1
 commonmark==0.9.1
 docutils==0.16
 recommonmark==0.7.1
+Jinja2<3.1.0

nexxT/Qt.py

@@ -0,0 +1,9 @@
+"""
+This module is never imported when executing nexxT. Instead, the Qt module is provided through the QtMetaPackage
+which is preventing python from loading unnecessary extensions. However, it is very handy to have this module, so
+pylint and the IDE auto-completers are happy to know the contents of this module
+"""
+import logging
+from PySide6 import QtWidgets, QtCore, QtGui
+
+logging.getLogger(__name__).warning("nexxT.Qt imported not via QtMetaPackage!")

nexxT/QtMetaPackage.py

@@ -0,0 +1,110 @@
+"""
+This module provides a QT meta package such that we are able to write "from nexxT.Qt.QtWidgets import QWidget" and
+nexxT.Qt will serve as an alias for PySide6
+
+It is loosly based on this tutorial:
+https://dev.to/dangerontheranger/dependency-injection-with-import-hooks-in-python-3-5hap
+"""
+import importlib.abc
+import importlib.machinery
+import importlib.util
+import sys
+import types
+
+class QtFinder(importlib.abc.MetaPathFinder):
+    """
+    The meta path finder which will be added to sys.meta_path
+    """
+    def __init__(self, loader):
+        self._loader = loader
+
+    def find_spec(self, fullname, path, target=None):
+        """
+        Attempt to locate the requested module
+
+        :param fullname: the fully-qualified name of the module,
+        :param path: set to __path__ for sub-modules/packages, or None otherwise.
+        :param target: can be a module object, but is unused here.
+        """
+        if self._loader.provides(fullname):
+            return importlib.machinery.ModuleSpec(fullname, self._loader)
+
+class QtLoader(importlib.abc.Loader):
+    """
+    The actual loader which maps PySide modules to nexxT.Qt. The approach is similar to executing a
+    ``from PySide6 import *`` statement in a real Qt.py module, but it is dynamically and prevents from loading unused
+    QT modules, like QtMultimedia, etc.
+    """
+
+    def __init__(self, prefix, qtlib):
+        """
+        Constructor.
+
+        :param prefix: the prefix for the proxied library (e.g. nexxT.Qt)
+        :param qtlib: the PySide library to be used, must be PySide6
+        """
+        self._prefix = prefix
+        self._qtlib = qtlib
+
+    def provides(self, fullname):
+        """
+        Checks whether the queried module can be provided by this laoder or not.
+
+        :param fullname: full-qualified module name.
+        """
+        return fullname == self._prefix or fullname.startswith(self._prefix + ".")
+
+    def create_module(self, spec):
+        """
+        Creates a new module according to spec. It will be empty initially and populated during exec_module(...).
+
+        :param spec: A ModuleSpec instance.
+        """
+        res = types.ModuleType(spec.name)
+        return res
+
+    def exec_module(self, module):
+        """
+        This function is called after create_module and it populates the module's namespace with the corresponding
+        instances from PySideX.
+        """
+        proxyname = self._qtlib + module.__name__[len(self._prefix):]
+        proxymod = importlib.import_module(proxyname)
+        def _copy_attrs(src, dst):
+            """
+            pyqtgraph approach for creating mirrors of the Qt modules
+            """
+            for o in dir(src):
+                if isinstance(getattr(src,o), types.ModuleType):
+                    continue
+                if o == "__path__":
+                    setattr(dst, o, [])
+                    continue
+                if o.startswith("__") and o != "__version__" and o != "__version_info__":
+                    continue
+                if not hasattr(dst, o):
+                    setattr(dst, o, getattr(src, o))
+        _copy_attrs(proxymod, module)
+        if proxyname == "PySide6":
+            module.call_exec = lambda instance, *args, **kw: instance.exec(*args, **kw)
+
+    def _truncate_name(self, fullname):
+        """Strip off _COMMON_PREFIX from the given module name
+        Convenience method when checking if a service is provided.
+        """
+        return fullname[len(self._prefix):]
+
+def setup():
+    libs = []
+    try:
+        import PySide6
+        libs.append(("PySide6", "shiboken6"))
+    except ImportError:
+        pass
+    if len(libs) != 1:
+        raise RuntimeError("nexxT needs PySide6 installed (available: %s).", libs)
+    libs = libs[0]
+    sys.meta_path = ([QtFinder(QtLoader("nexxT.Qt", libs[0])), QtFinder(QtLoader("nexxT.shiboken", libs[1]))] +
+                     sys.meta_path)
+
+setup()