Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for loading controller scripts as JS modules #2868

Merged
merged 29 commits into from
Jul 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
4cf5dd0
add support for loading controller scripts as JS modules
Be-ing Jun 14, 2020
40246b5
ControllerEngine: add warning when module functions are missing
Be-ing Jun 14, 2020
6636d0e
ControllerEngine: reload JS module when the file changes
Be-ing Jun 14, 2020
1bfdaf4
load controller XML and JS module files from subdirectories
Be-ing Jun 14, 2020
537f2a1
add MidiDispatcher module in new res/controllers/lib folder
Be-ing Jun 14, 2020
60590e8
add MidiDispatcher JS library to QRC
Be-ing Jun 15, 2020
a988523
MidiDispatcher: add comment explaining noteOff option
Be-ing Jun 15, 2020
949b11b
MidiDispatcher: rename registerInputCallback to setInputCallback
Be-ing Jun 21, 2020
019bf82
ControllerEngine: show script exception dialog for module functions
Be-ing Jun 21, 2020
a1ac571
MidiDispatcher: throw Errors for invalid inputs
Be-ing Jun 21, 2020
2a9aa02
MidiDispatcher.receiveData: revert to checking if callback is function
Be-ing Jun 21, 2020
dddaab6
MidiDispatcher: add support for 2 byte program change & aftertouch
Be-ing Jun 21, 2020
d9464c0
MidiDispatcher: use noteOff option by default
Be-ing Jun 21, 2020
071b306
ControllerEngine: show error if module has no receiveData function
Be-ing Jun 21, 2020
e1e8145
MidiDispatcher: add JSDoc documentation
Be-ing Jun 21, 2020
dddccd5
MidiDispatcher: rename receiveData to receiveMidiData
Be-ing Jun 30, 2020
6ddc7bb
ControllerEngine: add ConsoleExtension into QJSEngine environment
Be-ing Jul 4, 2020
ef745f1
controllers: rename receiveData function to handleInput
Be-ing Jul 4, 2020
e4a3c58
MidiDispatcher: rename receiveMidiData function to handleMidiInput
Be-ing Jul 4, 2020
6f59d5a
MidiDispatcher: delete debug message
Be-ing Jul 5, 2020
8182c77
ControllerEngine: use millisecond precision for timestamps
Be-ing Jul 5, 2020
20f1485
fix MidiController tests
Be-ing Jul 6, 2020
328611e
MidiDispatcher: use hash function instead of JSONifying Arrays
Be-ing Jul 9, 2020
868fd04
ControllerEngine: #if QT_VERSION check for loadModule method
Be-ing Jul 9, 2020
80f18ec
MidiDispatcher: avoid a string comparison on each input message
Be-ing Jul 9, 2020
abe2624
MidiDispatcher: improve JSDoc documentation
Be-ing Jul 9, 2020
3ef56de
MidiDispatcher: more improvements to JSDoc documentation
Be-ing Jul 10, 2020
ecc740d
ControllerEngine: fix build for Qt < 5.12
Be-ing Jul 10, 2020
f963cc3
remove MidiDispatcher JS module
Be-ing Jul 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/controllers/controller.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ bool Controller::applyPreset(bool initializeScripts) {
if (success && initializeScripts) {
m_pEngine->initializeScripts(scriptFiles);
}

if (initializeScripts) {
m_pEngine->loadModule(pPreset->moduleFileInfo());
Be-ing marked this conversation as resolved.
Show resolved Hide resolved
}
return success;
}

Expand Down Expand Up @@ -143,4 +147,6 @@ void Controller::receive(const QByteArray data, mixxx::Duration timestamp) {
QJSValue incomingDataFunction = m_pEngine->wrapFunctionCode(function, 2);
m_pEngine->executeFunction(incomingDataFunction, data);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should deprecate the use of ".incomingData". This will be a nice performance benefit.

Copy link
Contributor Author

@Be-ing Be-ing Jul 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is what I did, but below you are saying the opposite.

m_pEngine->handleInput(data, timestamp);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the place where the dispatcher should directly call the js callbacks.
For standard button connections we even bypass the js round trip by altering COs directly.
This will also help to clutter the js mapping. This need to provide callbacks for the difficult things only.

The mapping can still optional provide an "incomingData" function to perform difficult tasks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the dispatcher is JS. There are no plans to change this.

}
9 changes: 9 additions & 0 deletions src/controllers/controllerpreset.h
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ class ControllerPreset {
return m_scripts;
}

void setModuleFileInfo(QFileInfo fileInfo) {
m_moduleFileInfo = fileInfo;
}

QFileInfo moduleFileInfo() const {
return m_moduleFileInfo;
}

inline void setDirty(bool bDirty) {
m_bDirty = bDirty;
}
Expand Down Expand Up @@ -178,6 +186,7 @@ class ControllerPreset {
QString m_mixxxVersion;

QList<ScriptFileInfo> m_scripts;
QFileInfo m_moduleFileInfo;
};

typedef QSharedPointer<ControllerPreset> ControllerPresetPointer;
3 changes: 3 additions & 0 deletions src/controllers/controllerpresetfilehandler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,9 @@ void ControllerPresetFileHandler::addScriptFilesToPreset(
preset->addScriptFile(filename, functionPrefix, file);
scriptFile = scriptFile.nextSiblingElement("file");
}

QString moduleFileName = controller.firstChildElement("module").text();
preset->setModuleFileInfo(preset->dirPath().absoluteFilePath(moduleFileName));
Be-ing marked this conversation as resolved.
Show resolved Hide resolved
}

bool ControllerPresetFileHandler::writeDocument(
Expand Down
2 changes: 1 addition & 1 deletion src/controllers/controllerpresetinfoenumerator.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ void PresetInfoEnumerator::loadSupportedPresets() {
m_bulkPresets.clear();

for (const QString& dirPath : m_controllerDirPaths) {
QDirIterator it(dirPath);
QDirIterator it(dirPath, QDirIterator::Subdirectories);
Copy link
Member

@Holzhaus Holzhaus Jun 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can is unrelated, right? I think enabling support for controller scripts in subdirectories is a different matter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was very simple to implement and I don't think there's any reason to remove it now.

while (it.hasNext()) {
it.next();
const QString path = it.filePath();
Expand Down
56 changes: 56 additions & 0 deletions src/controllers/engine/controllerengine.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,10 @@ void ControllerEngine::gracefulShutdown() {
qDebug() << "Invoking shutdown() hook in scripts";
callFunctionOnObjects(m_scriptFunctionPrefixes, "shutdown");

if (m_shutdownFunction.isCallable()) {
executeFunction(m_shutdownFunction, QJSValueList{});
}

// Prevents leaving decks in an unstable state
// if the controller is shut down while scratching
QHashIterator<int, int> i(m_scratchTimers);
Expand Down Expand Up @@ -190,6 +194,8 @@ void ControllerEngine::initializeScriptEngine() {
// Create the Script Engine
m_pScriptEngine = new QJSEngine(this);

m_pScriptEngine->installExtensions(QJSEngine::ConsoleExtension);

// Make this ControllerEngine instance available to scripts as 'engine'.
QJSValue engineGlobalObject = m_pScriptEngine->globalObject();
ControllerEngineJSProxy* proxy = new ControllerEngineJSProxy(this);
Expand Down Expand Up @@ -223,6 +229,55 @@ void ControllerEngine::uninitializeScriptEngine() {
}
}

void ControllerEngine::loadModule(QFileInfo moduleFileInfo) {
#if QT_VERSION >= QT_VERSION_CHECK(5, 12, 0)
m_moduleFileInfo = moduleFileInfo;

QJSValue mod = m_pScriptEngine->importModule(moduleFileInfo.absoluteFilePath());
if (mod.isError()) {
showScriptExceptionDialog(mod);
return;
}

connect(&m_scriptWatcher,
&QFileSystemWatcher::fileChanged,
this,
&ControllerEngine::scriptHasChanged);
m_scriptWatcher.addPath(moduleFileInfo.absoluteFilePath());

QJSValue initFunction = mod.property("init");
executeFunction(initFunction, QJSValueList{});

QJSValue handleInputFunction = mod.property("handleInput");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not stick with the term "incomingData"?
Else I would like to emphasis that it is a catch all callback. "allIncommingData" or similar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is a completely new system.

if (handleInputFunction.isCallable()) {
m_handleInputFunction = handleInputFunction;
} else {
scriptErrorDialog(
"Controller JavaScript module exports no handleInput function.",
QStringLiteral("handleInput"),
true);
}

QJSValue shutdownFunction = mod.property("shutdown");
if (shutdownFunction.isCallable()) {
m_shutdownFunction = shutdownFunction;
} else {
qDebug() << "Module exports no shutdown function.";
}
#else
Q_UNUSED(moduleFileInfo);
#endif
}

void ControllerEngine::handleInput(QByteArray data, mixxx::Duration timestamp) {
if (m_handleInputFunction.isCallable()) {
QJSValueList args;
args << byteArrayToScriptValue(data);
args << timestamp.toDoubleMillis();
executeFunction(m_handleInputFunction, args);
}
}

bool ControllerEngine::loadScriptFiles(const QList<ControllerPreset::ScriptFileInfo>& scripts) {
bool scriptsEvaluatedCorrectly = true;
for (const auto& script : scripts) {
Expand Down Expand Up @@ -263,6 +318,7 @@ void ControllerEngine::reloadScripts() {

qDebug() << "Re-initializing scripts";
initializeScripts(m_lastScriptFiles);
loadModule(m_moduleFileInfo);
}

void ControllerEngine::initializeScripts(const QList<ControllerPreset::ScriptFileInfo>& scripts) {
Expand Down
6 changes: 6 additions & 0 deletions src/controllers/engine/controllerengine.h
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ class ControllerEngine : public QObject {
ControllerEngine(Controller* controller);
virtual ~ControllerEngine();

void handleInput(QByteArray data, mixxx::Duration timestamp);

bool executeFunction(QJSValue functionObject, QJSValueList arguments);
bool executeFunction(QJSValue functionObject, const QByteArray& data);

Expand Down Expand Up @@ -103,6 +105,7 @@ class ControllerEngine : public QObject {
virtual void timerEvent(QTimerEvent* event);

public slots:
void loadModule(QFileInfo moduleFileInfo);
bool loadScriptFiles(const QList<ControllerPreset::ScriptFileInfo>& scripts);
void initializeScripts(const QList<ControllerPreset::ScriptFileInfo>& scripts);
void gracefulShutdown();
Expand Down Expand Up @@ -146,6 +149,8 @@ class ControllerEngine : public QObject {
double getDeckRate(const QString& group);

Controller* m_pController;
QJSValue m_handleInputFunction;
QJSValue m_shutdownFunction;
QList<QString> m_scriptFunctionPrefixes;
QHash<ConfigKey, ControlObjectScript*> m_controlCache;
struct TimerInfo {
Expand All @@ -166,6 +171,7 @@ class ControllerEngine : public QObject {
QJSValue m_byteArrayToScriptValueJSFunction;
// Filesystem watcher for script auto-reload
QFileSystemWatcher m_scriptWatcher;
QFileInfo m_moduleFileInfo;
QList<ControllerPreset::ScriptFileInfo> m_lastScriptFiles;

bool m_bTesting;
Expand Down
13 changes: 13 additions & 0 deletions src/controllers/midi/midicontroller.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,17 @@ void MidiController::commitTemporaryInputMappings() {

void MidiController::receive(unsigned char status, unsigned char control,
unsigned char value, mixxx::Duration timestamp) {
QByteArray byteArray;
byteArray.append(status);
byteArray.append(control);
byteArray.append(value);

ControllerEngine* pEngine = getEngine();
// pEngine is nullptr in tests.
if (pEngine) {
pEngine->handleInput(byteArray, timestamp);
}
// legacy stuff below

// The rest of this function is for legacy mappings
unsigned char channel = MidiUtils::channelFromStatus(status);
Expand Down Expand Up @@ -471,6 +482,8 @@ double MidiController::computeValue(

void MidiController::receive(QByteArray data, mixxx::Duration timestamp) {
controllerDebug(MidiUtils::formatSysexMessage(getName(), data, timestamp));
getEngine()->handleInput(data, timestamp);
// legacy stuff below

MidiKey mappingKey(data.at(0), 0xFF);

Expand Down