From d4505eb7fbcbf73e31560fc6d52235d7392f65f0 Mon Sep 17 00:00:00 2001
From: Swiftb0y <12380386+Swiftb0y@users.noreply.github.com>
Date: Mon, 15 Jul 2024 14:50:35 +0200
Subject: [PATCH 1/4] initial draft
---
images/controller-engine-v2.svg | 621 +++++++++++++++++++
proposals/2024-05-23_controller_engine_v2.md | 256 ++++++++
2 files changed, 877 insertions(+)
create mode 100644 images/controller-engine-v2.svg
create mode 100644 proposals/2024-05-23_controller_engine_v2.md
diff --git a/images/controller-engine-v2.svg b/images/controller-engine-v2.svg
new file mode 100644
index 0000000..3bde588
--- /dev/null
+++ b/images/controller-engine-v2.svg
@@ -0,0 +1,621 @@
+
+
+
+
diff --git a/proposals/2024-05-23_controller_engine_v2.md b/proposals/2024-05-23_controller_engine_v2.md
new file mode 100644
index 0000000..2cb8a4a
--- /dev/null
+++ b/proposals/2024-05-23_controller_engine_v2.md
@@ -0,0 +1,256 @@
+# Controller Engine V2
+
+* **Owners:**
+ * @Swiftb0y, @acolombier
+
+* **Implementation Status:** `Not implemented`
+
+* **Related Issues and PRs:**
+ * [Create a reactive programming API for
+ controllers](https://github.com/mixxxdj/mixxx/issues/13440)
+ * [QML-based components API for
+ controllers](https://github.com/mixxxdj/mixxx/pull/13459)
+ * [Respect the Midi timestamp when
+ scratching](https://github.com/mixxxdj/mixxx/issues/6951)
+ * [make brake, soft start, and spinback part of the effects
+ system](https://github.com/mixxxdj/mixxx/issues/8867)
+ * [Move the controller screen rendering feature away from
+ `ControllerScriptEngineLegacy`](https://github.com/mixxxdj/mixxx/issues/13203)
+ * [Confusing Midi Sysex
+ handling](https://github.com/mixxxdj/mixxx/issues/12824)
+ * [Ability for controller to share data at
+ runtime](https://github.com/mixxxdj/mixxx/pull/12199)
+ * [Allow Controller Mappings to be located in their own
+ directory](https://github.com/mixxxdj/mixxx/issues/9906)
+ * [support external displays on
+ controllers](https://github.com/mixxxdj/mixxx/issues/8695)
+ * [Handle hot-plugging and graceful recovery of
+ controllers](https://github.com/mixxxdj/mixxx/issues/5614)
+ * [Controller scripts need to be able to
+ cross-communicate](https://github.com/mixxxdj/mixxx/issues/5165)
+
+This document is supposed to serve as an overview for a new controller engine
+for Mixxx. This does not only refer to the runtime "execution engine" of some
+mapping specific code, but almost all aspects from protocol IO, to the metadata
+schema, point-and-click mapping editor, and the scripting API.
+
+## Definitions
+
+* Mapping: a file or set of files which defines everything that’s needed to a
+ controller to communicate with mixxx
+* Controller: Physical self-contained piece of hardware, possibly consisting of
+ multiple sinks and sources
+* Source: a source of data from the controller (midi/HID messages from a
+ particular port/endpoint)
+* Sink: a port/endpoint of the controller that receives data from mixxx (e.g
+ screens, HID output report, write bulk endpoint are example of different sink
+ types)
+* Manifest: Metadata about the controller definition. Execution Engine and
+ Source/sink topology. (equivalent to what we do with the XML in the current
+ legacy engine)
+* Execution Engine: Scripting engine that evaluates the scripting source files
+ of the engine (so essentially what makes the scripting interactive/smart)
+* Module: ES6/QML Module that's part of a mapping, evaluated in an execution
+ engine (its what you expect ;) )
+* Capability: abstract definition of a concept that is not native to mixxx (eg.
+ shift, controlling different decks with the same physical hardware (1/3, 2/4
+ deck switching)). This is needed for “open-ended” cross-mapping communication
+ (avoiding “controller lock-in”).
+* Point-and-click editor: A tool to easily customize a controller mapping by
+ clicking a button on screen and on the hardware surface and those two
+ corresponding to each other.
+
+## Why
+
+The current controller engine suffers from a number of issues primarily
+summarized as a lack of flexibility.
+
+### Pitfalls of the current solution
+
+To summarize the issues linked in the Related Issues and
+PRs section:
+
+ 1. The current controller engine is not able to handle multiple controllers at
+ once. This is important when a single piece of hardware is advertising
+ multiple different control endpoints (eg HID + Bulk for control + screens
+ such as Native Instruments Traktor devices) as well as when multiple pieces
+ of hardware are supposed to present a single unified control surface (such
+ as the modular Behringer CMD-MM1/-PL1/-DC1/-LC1 or modular NI Traktor
+ Kontrol F1/X1/Z1). Being able to bundle multiple different endpoints into
+ the same mapping is important for a good user experience and to allow for
+ more complex mappings.
+ 2. The current controller engine (`ControllerScriptEngineLegacy`) is not able
+ to support modules of any kind. This is important for code organization and
+ reusability. In order to increase mapping code quality and maintainability,
+ it is important to be able to split the mapping code into multiple files and
+ to be able to reuse code between different mappings. This is especially
+ important now that mapping authors are starting to reuse functionality
+ across different hardware devices (see Numark NS6II and Mixtrack variants or
+ similarity between different pioneer controllers)
+ 3. Current mappings modify global state and assume that they are the only
+ mapping running. This needs to be fixed in order to support multiple
+ mappings running at the same time in the same "execution engine".
+ 4. Device hotplug is currently not possible / hard to implement. This is
+ primarily because of the underlying protocol IO layer not willing to support
+ it. This document proposes an alternative approach that lets us more easily
+ switch the underlying library.
+ 5. Allow changing the manifest format. Many people have expressed distaste in
+ the XML manifest format. Careful implementation of the new format should
+ allow experimentation with different formats.
+ 6. Changing out infrastructure like this is virtually impossible. The proposed
+ architecture is designed to be modular and allows for all the required
+ components to be implemented gradually.
+
+## Goals
+
+Goals and use cases for the solution as proposed in [How](#how):
+
+* Allow multiple controllers to be used at the same time.
+* Mappings should be able to organize themselves into modules.
+* IO protocols should easily be changeable and not not be tied to the mapping.
+* Allow easy, gradual implementation of the system.
+
+### Audience
+
+This is primarily targeted at developers and power users who are interested in
+the controller engine. End-users only interested in the point-and-click mapping
+editor should not experience any significant changes.
+
+## Non-Goals
+
+* Replace `ControllerScriptEngine` in the short term until we are confident in
+ the new design.
+* make changes to the scratching code as it is orthogonal to this proposal.
+* Make changes to the effects system as it is orthogonal to this proposal.
+
+## How
+
+Explain the full overview of the proposed solution. Some guidelines:
+
+### File Structure
+
+Built-in mappings reside in `res/controllers/` where each mapping is a directory
+containing at least a manifest file named `manifest.xxx` (where `.xxx` is the
+language-specific file extension (eg. `.xml`, `.yml`, `.json`, etc)).
+Additionally, `res/controllers/lib` contains shared modules that can be used by
+multiple mappings (such as componentsJS).
+
+### Manifest
+
+The manifest is a declarative file that describes the mapping. It contains the
+following information:
+
+* Sources and Sinks
+* Execution Engines
+ * Used Mixxx APIs
+* Capabilities
+
+### Sources and Sinks
+
+Sources and Sinks are the endpoints of the controller. They are defined in the
+manifest and are used to define the IO protocol of the controller. They are
+characterized by their protocol (eg. MIDI, HID, Bulk, etc), their direction (eg.
+input, output, bidirectional) and optionally explicitly their underlying
+implementation (eg. `portmidi`, `rtmidi`, `libremidi`, etc). They will also
+contain a heuristic description to map the source/sink to the physical
+controller (protocol specific, in the case of HID, usb vid&pid could be used for
+example).
+
+### Execution Engines
+
+Execution engines is yet another section of the manifest. It defines the
+scripting engine that will be used to evaluate the mapping. It is defined by its
+type (JS, QML, something else?) along with a specific entry point. In the case
+of JS, this would be a ES6 module that exports a controller class to be
+instantiated. In the case of QML, this would be a QML file that defines a
+controller object.
+
+### Capabilities
+
+Capabilities are the concept that allows for cross-mapping communication while
+avoiding controller-lock-in. They are defined by mixxx and are used to define
+concepts that can't be easily mapped to ControlObjects. For example, a
+capability could be `shift` or `deck-switching`. Capabilities are defined in the
+manifest and can be used by the execution engine to communicate with other
+mappings. Capabilities are essentially "opt-in" APIs.
+
+### Point-and-click editor
+
+In order to still support the point and click style use-case, we add yet another
+section that directly connects patterns of midi messages to a control object.
+This section is again attached to a source/sink. There is no support for
+interacting with an execution engine via this section. This essentially works
+like the current mapping editor, but removes the
+`path.to.some.JS.input.handler` feature as it is not compatible with
+the new architecture. This method is favoured over code generation as that is
+not always possible, nor would it likely produce a good result. Moreover, this
+would let us recycle parts of the current codebase. _No need to reinvent the
+wheel._ Execution engines can still be wired up explicitly to access the
+dispatcher here as well. This is to avoid the need to have a separate dispatcher
+defined in the mapping and for easier integration with "hybrid" mappings. the
+API is essentially already implemented as [Registering MIDI Input Handlers From
+Javascript](https://github.com/mixxxdj/mixxx/pull/12781), though QML would need
+a separate declarative API.
+
+### Wiring it all together
+
+Since this flexible layout results in a many-to-many relationship sources/sinks
+and execution engines, we specify a separate section in the manifest that
+defines how sources and sinks are connected to execution engines.
+
+### Sharing mappings with modules
+
+In order to share mappings that access modules outside the mappings root folder,
+the execution engine must be able to create a list of all files accessed and
+export them along the controller files. This would be done by creating a copy of
+all the required files in the root of the mapping during exporting. That tree is
+then overlaid over the built-in libraries. This is currently only possible
+within a `QQmlEngine` using `QQmlEngine::addUrlInterceptor` and
+`QQmlEngine::addImportPath`.
+
+### Mixxx APIs
+
+In order to obtain optimal developer experience of mixxx APIs in the execution
+engine, each execution engine should have its own implementation that maps
+idiomatically to the strengths of the execution engine. This is especially
+important for QML, as it is a reaktive declarative language that does have many
+more features than JS. This is essentially already the case in our current
+codebase, see `*JSProxy` and `*QMLProxy` classes as examples.
+
+### Migration from `ControllerScriptEngineLegacy`
+
+We can gradually transition to the new architecture by modelling the semantics
+of `ControllerScriptEngineLegacy` as a separate execution engine. This would
+allow us to gradually migrate old mappings to the new architecture.
+
+## Current unknowns
+
+How feasible is the heuristic detection of sources and sinks? How do we handle
+multiple sources/sinks of the same type originating from different controllers?
+Eg how do we avoid that the screen from one piece of hardware is connected to
+the wrong controller?
+
+## Alternatives
+
+1. Shoehorn the new architecture into the existing `ControllerScriptEngine`
+ class. This would not eliminate the one-controller-per-mapping limitation and
+ would not allow us to iterate on the design nor on the API.
+
+## Architecture summary
+
+
+
+## Action Plan
+
+The following action plan is very abstract because the architecture is designed
+to be implemented gradually. Once we have decided on priorities, we can decide
+on a more concrete plan.
+
+1. Define the abstract Manifest outline.
+2. Implement a concrete manifest parser of the basic outline.
+3. Choose any component from the architecture summary diagram and implement it
+ using tests and making instanstiable by via the manifest.
+4. Once 2 connected components are implemented, implement their connection via
+ the manifest.
+5. Repeat until all components are implemented.
From 8afaba406eb53f945553edbc634dbbd7ffea453b Mon Sep 17 00:00:00 2001
From: Swiftb0y <12380386+Swiftb0y@users.noreply.github.com>
Date: Mon, 15 Jul 2024 22:44:08 +0200
Subject: [PATCH 2/4] replace inkscape svg with mermaid diagram
---
images/controller-engine-v2.svg | 621 -------------------
proposals/2024-05-23_controller_engine_v2.md | 52 +-
2 files changed, 50 insertions(+), 623 deletions(-)
delete mode 100644 images/controller-engine-v2.svg
diff --git a/images/controller-engine-v2.svg b/images/controller-engine-v2.svg
deleted file mode 100644
index 3bde588..0000000
--- a/images/controller-engine-v2.svg
+++ /dev/null
@@ -1,621 +0,0 @@
-
-
-
-
diff --git a/proposals/2024-05-23_controller_engine_v2.md b/proposals/2024-05-23_controller_engine_v2.md
index 2cb8a4a..61bb352 100644
--- a/proposals/2024-05-23_controller_engine_v2.md
+++ b/proposals/2024-05-23_controller_engine_v2.md
@@ -238,8 +238,56 @@ the wrong controller?
## Architecture summary
-
+```mermaid
+flowchart LR
+ %% Define front-end web components
+ subgraph io["IO"]
+ co["Control Object"]
+ midiIO["Midi IO"]
+ hidIO["HID IO"]
+ mixxxLib["Mixxx Library"]
+ mixxxUi["Mixxx UI"]
+ end
+ style io fill:transparent,stroke:green,color:#fff
+
+ subgraph layers["Intermediary Layers"]
+ co---midiDispatcher["MIDI Dispatcher"]
+ midiIO---midiDispatcher
+ mixxxUi---uiShift["UI Shift Capability"]
+ end
+ style layers fill:transparent,stroke:yellow,color:#fff
+
+ subgraph proxies["API Proxies"]
+ co---coJSProxy["COJSProxy"]
+ co---coQMLProxy["COQmlProxy"]
+ midiIO---midiJSProxy["MIDI IO JSProxy"]
+ midiIO---midiQMLProxy["MIDI IO QmlProxy"]
+ hidIO---hidJSProxy["HID IO JSProxy"]
+ hidIO---hidQMLProxy["HID IO QMLProxy"]
+ midiDispatcher---midiDispatcherJSProxy["MIDI Dispatcher JSProxy"]
+ mixxxLib---libQmlProxy["Library QmlProxy"]
+ uiShift---CapJSProxy["Capability JSProxy"]
+ end
+ style proxies fill:transparent,stroke:cyan,color:#fff
+
+ subgraph engine["Execution Engine"]
+ coJSProxy---jsEngine["JSExecution Engine"]
+ midiJSProxy---jsEngine
+ hidJSProxy---jsEngine
+ hidQMLProxy---jsEngine
+ midiDispatcherJSProxy---jsEngine
+ CapJSProxy---jsEngine
+ coJSProxy---jsLegacyEngine["LegacyJS Execution Engine"]
+ hidJSProxy---jsLegacyEngine
+ hidQMLProxy---jsLegacyEngine
+ midiJSProxy---jsLegacyEngine
+ CapJSProxy---jsLegacyEngine
+ coQMLProxy---qmlEngine["QmlExecutionEngine"]
+ midiQMLProxy---qmlEngine
+ libQmlProxy---qmlEngine
+ end
+ style engine fill:transparent,stroke:red,color:#fff
+```
## Action Plan
From 3af26acbc777206802b6feeb21b4c1d69af165a6 Mon Sep 17 00:00:00 2001
From: Swiftb0y <12380386+Swiftb0y@users.noreply.github.com>
Date: Sun, 28 Jul 2024 18:13:46 +0200
Subject: [PATCH 3/4] fix copy-paste error in mermaid diagram
Co-authored-by: JoergAtGithub <64457745+JoergAtGithub@users.noreply.github.com>
---
proposals/2024-05-23_controller_engine_v2.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/proposals/2024-05-23_controller_engine_v2.md b/proposals/2024-05-23_controller_engine_v2.md
index 61bb352..9176b6e 100644
--- a/proposals/2024-05-23_controller_engine_v2.md
+++ b/proposals/2024-05-23_controller_engine_v2.md
@@ -274,7 +274,7 @@ flowchart LR
coJSProxy---jsEngine["JSExecution Engine"]
midiJSProxy---jsEngine
hidJSProxy---jsEngine
- hidQMLProxy---jsEngine
+ hidQMLProxy---qmlEngine
midiDispatcherJSProxy---jsEngine
CapJSProxy---jsEngine
coJSProxy---jsLegacyEngine["LegacyJS Execution Engine"]
From 816bfdcd9bcdb890ad6c55f2f5e112720f5ccfb7 Mon Sep 17 00:00:00 2001
From: Swiftb0y <12380386+Swiftb0y@users.noreply.github.com>
Date: Sat, 3 Aug 2024 16:43:20 +0200
Subject: [PATCH 4/4] address latest review comments
---
proposals/2024-05-23_controller_engine_v2.md | 73 ++++++++++++++++++--
1 file changed, 67 insertions(+), 6 deletions(-)
diff --git a/proposals/2024-05-23_controller_engine_v2.md b/proposals/2024-05-23_controller_engine_v2.md
index 9176b6e..ae96c0a 100644
--- a/proposals/2024-05-23_controller_engine_v2.md
+++ b/proposals/2024-05-23_controller_engine_v2.md
@@ -106,7 +106,8 @@ PRs section:
Goals and use cases for the solution as proposed in [How](#how):
* Allow multiple controllers to be used at the same time.
-* Mappings should be able to organize themselves into modules.
+* Mappings should be able to be partitioned / modularized within their subfolder.
+* Code reuse across mappings should be easier and encouraged (complex hardware of the same vendor often shares functionality).
* IO protocols should easily be changeable and not not be tied to the mapping.
* Allow easy, gradual implementation of the system.
@@ -133,7 +134,8 @@ Built-in mappings reside in `res/controllers/` where each mapping is a directory
containing at least a manifest file named `manifest.xxx` (where `.xxx` is the
language-specific file extension (eg. `.xml`, `.yml`, `.json`, etc)).
Additionally, `res/controllers/lib` contains shared modules that can be used by
-multiple mappings (such as componentsJS).
+multiple mappings (such as componentsJS). Mapping folders may be zipped during
+export (see considerations regarding shared modules) for the users convenience.
### Manifest
@@ -145,13 +147,17 @@ following information:
* Used Mixxx APIs
* Capabilities
+In order to reduce boilerplate markup in the manifest, syntactic shorthands may
+be introduced to cover the most common usecases. The most common usecase may be that
+only one source and sink of a particular protocol coupled to a single execution
+engine and/or protocol dispatcher.
+
### Sources and Sinks
Sources and Sinks are the endpoints of the controller. They are defined in the
manifest and are used to define the IO protocol of the controller. They are
-characterized by their protocol (eg. MIDI, HID, Bulk, etc), their direction (eg.
-input, output, bidirectional) and optionally explicitly their underlying
-implementation (eg. `portmidi`, `rtmidi`, `libremidi`, etc). They will also
+characterized by their protocol (eg. MIDI, HID, Bulk, etc) and their direction (eg.
+input, output, bidirectional). They will also
contain a heuristic description to map the source/sink to the physical
controller (protocol specific, in the case of HID, usb vid&pid could be used for
example).
@@ -163,7 +169,11 @@ scripting engine that will be used to evaluate the mapping. It is defined by its
type (JS, QML, something else?) along with a specific entry point. In the case
of JS, this would be a ES6 module that exports a controller class to be
instantiated. In the case of QML, this would be a QML file that defines a
-controller object.
+controller object. Each Execution engine should be run in its own independent
+thread. Data exchange between threads should be non-blocking where possible,
+preferably using Qt Signal/Slots or Non-blocking pipes (since those two are used
+in mixxx already). Scheduling requirements are handled by each engine individually
+(eg refreshrate for a GUI engine or latency requirements of an engine handling IO).
### Capabilities
@@ -289,6 +299,57 @@ flowchart LR
style engine fill:transparent,stroke:red,color:#fff
```
+### Manifest Mockup
+
+While the manifest is supposed to be implemented language-independent, the easiest
+implementation would likely be based on XML, as Qt already contains the necessary
+parsing infrastructure. So it makes sense to use XML for mockups. Inline XML comments
+serve as explanation to the reader of this proposal and are not intended to be part
+of real manifests.
+
+```xml
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+```
+
## Action Plan
The following action plan is very abstract because the architecture is designed