Integration Guide
This page outlines how to get Dynamo to work with other software - if that’s of interest - fantastic, read on.
The aim is that this document and its examples are self contained. You’ll need to have played with Dynamo a bit, either the standalone version, or the Revit version, and be familiar with some programming, nothing too arduous though. Until we've had a bunch of people use the document, we expect it'll have bugs. If you have questions, please get in touch and we'll fold the answers in here. For extra kudos, feel free to send us a pull request with suggested improvements
Dynamo is a work in progress. We’re still improving how it works on a regular basis. This will break things from time to time. That said there are a number of folks build tools on top of it already, so there’s no need to wait. If you like, drop us a note and we’ll try and keep in touch before we do anything that causes you to need to do some work. The good news is that this state means that if there are things that don’t work for you we can, and will, look at changing Dynamo to make them better.
We want integrating with Dynamo to be a bit like buying sweets in a pick-and-mix shop. You only pay for what you take. Along this line there are a number of different ways of integrating, each offering more functionality, a tighter level of integration but needing a bit more work.
- Import/Export
- Zero Touch Import
- .NET interop with support utils
- Subclass Dynamo Application
- Fork Dynamo
There's a sample library that is an example of everything we talk about here: Example
When to use: Use Import/Export when you need to get geometry into and out of Dynamo, but don’t need to add new functionality or change the way Dynamo works
Dynamo can import and export geometry as a SAT file. Simply create your geometry and select File, Export as SAT. You can then import this file into another application.
Below shows an example of what this looks like.
[TODO(lukechurch): Add UI screenshot]
Dynamo can import and use .NET libraries with minimal modification and automatically present them as nodes. This is the easiest way of getting started with building an integration.
This document gives an introduction to this process. Please head there and return here when you’re done.
Welcome back. By now you know how to go about writing some C# and have it appear as nodes in Dynamo. We use this mechanism for handling all our nodes in Dynamo.
If you’re writing some simple libraries, that all you’ll need to know. It’ll just work, however, larger applications often need a little more support. Starting with object life cycles.
When to use: When you need to have objects that have state that needs to be cleaned up
Object life cycles are often used to describe the way in which objects are constructed. Lets use the example of building a toy 2D geometry library for WPF.
[TODO(lukechurch): Add Ref to WPF sample project]
We could start off by building ourselves a very simple 2D point class that started a UI window and displayed a point-hog at its location.
We could do this simply with a static method
private static List<Hog> HogList;
Hog HogByPoint(double x, double y)
{
EnsureVisible();
HogList.add(New Hog(x, y));
}
All good. And we can now use this in Dynamo as you can see.
[TODO(lukechurch): Screenshot]
However, there’s a bit of a problem. If we remove the node, the hog doesn’t disappear.
We can fix this by implementing the Dispose method
class Hog : IDisposable
{
public void Dispose()
{
HogList.RemoveElement(this);
}
}
The Dynamo engine contains a fast, ref-counting garbage collector. When geometry is no longer available (e.g. because the node that created it has been deleted), the garbage collector detects this and calls Dispose on it immediately.
This is different to .NET’s garbage collector. The Dynamo garbage collector is deterministic. It will call Dispose on all object no later than the end of the end of the current execution cycle.
By contrast the .NET GC cleans memory when there is a need to do so, caused by e.g. a shortness of available memory (this is a gross simplification of a fascinating but complicated problem – see [TODO(lukechurch) REF] for more).
There is a strict ordering here. Dynamo’s garbage collector will always do its collection first. The only time when the Dynamo dispose call may not get called is on application exit, or because of a crash.
So we now have an object life cycle:
- Object is created by a factory call
- Object is used
- Dynamo calls Dispose (if the IDisposable Interface is implemented)
- At some point later, .NET’s garbage collector will delete the object if the target application is holding no further references to it
If the application experiences an error, the exception is marshalled back into Dynamo and displayed on the node that caused it, see for example the below:
[TODO(lukechurch): Screenshot of error display]
When to use: When you need to reuse the same objects, rather than deleting and recreating them.
Lets go back to our hog example. There’s another difficulty. The hogs don’t move!
Consider the graph below, now change the slider position.
[TODO(lukechurch): Screenshot of error update]
You can see that the hog appears to move, it is no longer in the old position and appears in the new location. However, it’s not really moving, it’s actually deleting the hog at the old position and creating a new one at the new location. We can see this by associating an identity with the hog, and incrementing it everytime we create a new one.
Right now, we can see that every time we move the slider a hog is getting created. The old one is no longer referenced and so gets cleaned up.
We might ask whether this matters? After-all the end state is the same, there is a hog at (10, 10) and not at (0, 0). There are two cases in which it does matter:
- If objects in your system use their identity for other behaviours. Consider the humble reference point in Revit. If you delete and recreate a ref point, then any lines attached to it also get deleted. Here the strategy of deletion and recreation in untenable it destroys other important relationships
- If creating and destroying your objects is really expensive, but mutating them is cheap. At which point you may not want to go through the creation cycle.
The Dynamo engine includes a mechanism called Trace which can be used to re-associate objects with previous executions, this works by the function storing information in a slot in the Thread Local Storage (TLS).
The mechanism below outlines how it works:
First execution
- Call is made (Hog.ByPoint) to the constructor with an empty TLS
- Constructor creates object, extracts an element ID
- Constructor loads the element ID into the TLS before returning from the constructor
- Engine checks the TLS after the constructor returns and saves the ID into the Callsite
Second execution
- Engine loads the ID from the Callsite into the TLS
- Call to (Hog.ByPoint) is made
- Hog.ByPoint reads the TLS, finds the element, looks up that element and updates it rather than creating another
- Hog.ByPoint leaves the element data in TLS
- The Engine maintains the data in the Callsite cache
Implementation notes:
Use of this mechanism will soon require the [RegisterForTrace(ID)] attribute to be marked on the methods. At the moment it is on by default, but this is imposes a performance cost on methods that don’t need the feature
Right now, the engine uses a hard-coded slot ID. This is a temporary limitation that will be removed at the same time as the above. When it is, the engine will use the ID registered with the attribute
[TODO(lukechurch): Arbitrary array binding ]
You've now learned how to make a zero touch library, and how to bind elements created in your host application to objects created in your graph through Trace. The beauty of zero touch is that your library doesn't have to reference any of DynamoCore. But what if your library needs to know when certain things happen in the Dynamo application, like the opening or closing of Workspaces? For example, when a Workspace is closed you might need your zero touch library to know this so that you can conduct some cleanup on the host application's document. For this we've provided a set of static events which are available in the DynamoServices library. These include, WorkspaceAdded and WorkspaceRemoved. Because these are static events, the zero touch library implementor must be sure to un-hook any event handlers during Dispose, or at another appropriate time, to avoid memory leaks.
When to use: When you need to deeply integrate Dynamo into another application, to e.g. meet additional threading requirements
Many larger applications have additional requirements for how plugins that use their API are loaded, and how they use threading. Revit is one example of an application like this.
To understand the structuring lets consider the diagram of the dependencies:
Starting at the top. The host has a dynamic, runtime dependency on Dynamo. this is used to register Dynamo as an extension to the primary application. In Revit this is achieved by an Addin file: https://github.com/DynamoDS/Dynamo/blob/master/tools/install/Extra/DynamoForRevit2015.addin
This loads a specialised version of the application. In Revit this is the DynamoRevit application. This provides a number of extensions to the regular DynamoCore. A key one is the use of IdlePromises for scheduling execution.
The IdlePromise mechanism is used to facilitate executing within the threading constraints of the application. It achieves this by scheduling the execution of the Engine (see below) onto the UI thread within the transaction envelope that Revit needs.
TODO(lukechurch):Ref to Idle Promise class.
The next two elements, the UI Layer and the DynamoCore layer provide the behaviour of the user interface. It is unusual that an application needs to modify the behaviour supplied in these classes. We'll talk more use cases that do require modifications to this in section 5.
The final component in the LHS vertical stack is the Engine itself. This is where the execution is done. It is the code that handles loading the libraries used into the VM and dispatching function calls to them. We haven't come across an integration use case that required modifications to engine itself yet - if you have one, please get in touch!
The code that the engine loads is a selection of user supplied functions. These are the functions that we discussed how to write in the previous sections of the document. In the cases where the nodes have counterpart customised UI implementations, these are kept in a separate library of nodes. These UI Nodes emit AST that calls the implementation in the node libraries themselves.
This process is discussed in more detail in: TODO(lukechurch): Ref to custom UI nodes doc
Whilst this architecture in itself is enough to implement most functionality, it is usually convenient to have an additional services library. In the Revit integration this is provided by the Host Services library here: https://github.com/DynamoDS/Dynamo/tree/master/src/Libraries/Revit/RevitServices
This also contains functionality that is shared between the nodes that implement the behaviour in Revit and the UI elements that need to use similar behaviour.
Some common challenges that occur during the integration of Dynamo into larger applications are Shared Object Ownership, Identity management and dirty object detection.
Most user workflows require that objects that are created in the host are not deleted by Dynamo. Consider for example creating a wall in Revit. Then using the SelectElement to locate the wall. Then deleting the select element node.
If no special support was added, the object would no longer have anything in the graph referring to it and so it would be deleting by calling the Dispose method. This is not likely to be the desirable outcome. This is because logically the object was owned by Revit, not by Dynamo.
In DynamoRevit this is handled by a lifecycle manager. TODO(lukechurch)