-
Notifications
You must be signed in to change notification settings - Fork 263
OSC Support
Open Sound Control, OSC, is an open, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices, according to its specification. OSC allows for name/value pairs to be sent back and forth between different applications and/or devices. In ATF, the underlying network protocol is UDP.
For iOS devices, we recommend using the Lemur application from Liine. For Android devices, we recommend TouchOSC from Hexler.net. In either case, game designers can construct their own user-interfaces (and even macros in Lemur), which then allows the tablet computer to run commands or edit objects in an ATF application that incorporates ATF's OSC support.
The unit of transmission in OSC is the OSC Packet. Any application that sends OSC Packets is an OSC client; any application that receives OSC Packets is an OSC server. Some applications can be both, sending and receiving OSC packets. An ATF application can serve as either an OSC client or server, but generally operates as a server. That is, you can control the ATF application by sending it OSC messages, which include the capability of executing ATF commands. For details, see OscCommandReceiver Component.
A message to an OSC server contains an address, a description of its arguments, and the arguments. Addresses look like URLs. The address for an OSC server is a string of slash "/" separated strings, such as "/resonators/3/frequency". The address's last part specifies a Method, which causes the OSC server to do something. You can think of OSC as a mechanism for making Remote Procedure Calls to applications and devices.
Although the OSC standard addresses shortcomings of MIDI and can replace it, there is nothing in OSC limiting it to musical devices. In the future, the name Open Sound Control may change to Open Systems Control to reflect the fact that the standard is general purpose. For this reason, ATF class names simply use the prefix "OSC". This document also refers to OSC, rather than spelling out its full name.
Not all of the features of OSC 1.0 are supported in ATF. The '\' and '?' wildcard characters are not supported robustly in the OSC address; '\' only works at the end of an address, and '?' is not supported at all. Also, ATF does not support the extended features of OSC 1.1, such as using "//" as a wildcard for paths in the OSC Address.
For a general introduction to OSC on Wikipedia, see Open Sound Control. For the OSC 1.0 specification, see The Open Sound Control 1.0 Specification. For information on version 1.1, see Features and Future of Open Sound Control version 1.1 for NIME. For OSC in general, see http://opensoundcontrol.org/. For OSC best practices, see Best Practices for Open Sound Control.
Guerrilla Games is using the Lemur iPad® application, developed by the Liine company, to communicate with their CoreText Editor. It has integrated its CoreText Editor with ATF's OSC support.
CoreText Editor can do two-way communication with Liine Lemur:
CoreText Editor's NodeTypePaletteItem
from the DomNodeTypes
can be used to populate the OSC addresses and then the properties can be filtered in various ways. This is all client-side.
Guerrilla Games prepared this video to show how they are using OSC:
ATF's OSC support includes these items:
-
IOscService
: Interface for using the OSC protocol to communicate with OSC-compatible devices. This interface allows for sending and receiving name-value pairs, asynchronously. For a discussion, see IOscService Interface. -
OscService
: Component that allows OSC devices to get and set properties on C# objects. This is the main component supporting OSC. -
LemurOscService
: Component that provides OSC support compatible with the Lemur iPad® application. If you want to communicate with the Liine Lemur iPad® application, consider usingLemurOscService
(which derives fromOscService
) instead ofOscService
. For more information on using Lemur, see the Lemur site. For more details on this component, see LemurOscService Component. -
OscCommandReceiver
: Component that listens for OSC messages and can trigger the execution of commands in the application. -
OscCommands
: Component that provides menu commands that may be useful for users ofOscService
. -
OscDialog
: Dialog to display information about OSC operation in the application.
IOscService
is an interface for using the OSC protocol to communicate with OSC-compatible devices. This interface allows for sending and receiving name-value pairs, asynchronously. This interface allows an ATF application to send messages to and receive messages from an OSC application/device.
The interface contains a method and an event:
-
void Send(IEnumerable<Tuple<string, object>> addressesAndData)
: Send the OSC addresses and data object pairs to each destination endpoint, asynchronously. Each pair is an OSC address string and a non-null data payload that is sent to each destination device. Valid types of data are 32-bit floats, arrays of 32-bit floats, 32-bit integers, strings, and arrays of bytes. This method returns immediately. If there is a network slowdown of some kind, it is possible that new values for a particular OSC address replace the old, unsent values. The order of the pairs can also be changed before they are sent. The pairs may not be sent all in one OSC bundle, if the size is too large. -
event EventHandler<OscMessageReceivedArgs> MessageReceived
: Notify listeners that an OSC message has been received. The OSC address and data payload are provided in theOscMessageReceivedArgs
, and listeners can set theOscMessageReceivedArgs.Handled
property totrue
if no further processing should be done on the message.
This event argument class derives from HandledEventArgs
and provides the OSC address and data payload from an OSC message. Setting its Handled
property (from HandledEventArgs
) to true
prevents other listeners from receiving this object. The IOscService.MessageReceived
event uses this class to provide its event information.
OscMessageReceivedArgs
has two fields:
-
Address
: OSC address that the message is directed to. -
Data
: OSC data payload for the address.
OscServices
contains useful extension methods and utilities for working with IOscService
objects.
-
void Send(this IOscService service, string oscAddress, object data)
: Send a single OSC message to each destination endpoint with the given data, asynchronously. -
string FixPropertyAddress(string oscAddress)
: Fix an OSC address to make it compliant. It adds a leading '/' if it's missing and removes illegal characters: spaces, braces, and brackets.
OscService
is the main item used in ATF OSC support.
OscService
processes OSC messages by getting and setting properties on C# objects. OscService
allows you to associate an OSC address with a particular property of a class. Objects with properties that have an OSC address associated with them are known as addressable. For details, see Mapping an OSC Address onto a C Sharp Class Property.
When the IInitializable.Initialize()
method runs for this component, it starts a UDP server. UDP is the main protocol used with OSC.
If you want to communicate with the Liine Lemur iPad® application, you should probably use LemurOscService
instead of OscService
; for details, see LemurOscService Component. Consider also using the OscCommands
component to process commands; for more information, see OscCommands Component. The OscCommandReceiver
component can trigger the execution of commands, so it may also be useful in your application; for details, see OscCommandReceiver Component.
OscService
uses the Bespoke Open Sound Control Library for a UDP server and to send and receive OSC messages. For information, see its description at Open Sound Control. Also see Bespoke Open Sound Control Library.
OscService
receives OSC messages in a separate thread from the main application thread. OscService
can set a C# property when it receives and processes an OSC message. The message's OSC address specifies the class and property. If the OSC address has no property associated with it, the message is ignored. The message's arguments specify the new value of the property. OscService
attempts to convert the arguments to the right type of object for the property. If no arguments are supplied or the wrong type of arguments, the property is not set. You can specify a converter for these values; for a description, see Mapping an OSC Address onto a C Sharp Class Property.
When OscService
receives a message from a device, the device is added to the listeners list, which is kept in the DestinationEndpoints
property. A listener can also be specified in the OSC Dialog created by OscDialog
. For information on what this dialog can do, see OscDialog Class.
Property value changes resulting from messages are done in a transaction context, so the changes can be undone or redone.
For example, an OSC device, such as an iPad® running Lemur, could have a slider control that sends an OSC message whenever the slider position changes. The address of this OSC message could be associated with the property FaderVolume1
in the SoundMixer
class that controls sound use in a game application. Moving the slider on the iPad® changes the FaderVolume1
volume level in SoundMixer
.
By using the OscCommandReceiver
component, you can also send an OSC message to an ATF OSC-enabled application that executes an ATF command, rather than setting a C# property. For more information, see OscCommandReceiver Component.
OscService
sends messages in the main application thread. OscService
sends property value data OSC messages to all listening OSC applications/devices in response to several context change events. OscService
maintains contexts in these properties:
-
SelectionContext
for anISelectionContext
. -
ObservableContext
for anIObservableContext
.
ActiveContextChanged
event in the IContextRegistry
occurs. OscService
imports an IContextRegistry
implementer, such as the ContextRegistry
component.
OscService
sends the value of a property in an OSC message when:
- The selection changes, that is, when the
SelectionChanged
event occurs on theISelectionContext
. Properties for the last selected object(s) are sent to all listeners. - Contents change. This occurs when the
ItemChanged
,ItemInserted
orItemRemoved
events occur in theIObservableContext
. Properties for the changed object are sent to listeners if it was the last selected object.
This means that all OSC receivers broadcast to should be prepared to receive such messages. If these receivers include ATF applications implementing OscService
, the application should map OSC addresses to the desired properties, if any, for which they would be receiving messages. Applications can map OSC addresses onto different properties and classes than the application/device sending the OSC messages. Keep in mind that receiving and processing OSC messages with OscService
results in their associated properties changing. For more details on mapping OSC addresses, see Mapping an OSC Address onto a C Sharp Class Property.
If these selected objects can be adapted as DomNode
s, sending these OSC messages has much better performance than otherwise.
For OscService
to get or set property values, a correspondence between an OSC address and a property in a class must be specified. This is an essential step for any OSC server.
Map addresses by calling one of OscService
's AddPropertyAddress()
methods, which take several forms:
string AddPropertyAddress(PropertyInfo propertyInfo, string oscAddress)
string AddPropertyAddress(Type classType, string propertyName, string oscAddress)
string AddPropertyAddress(string typeName, string propertyName, string oscAddress)
string AddPropertyAddress(PropertyDescriptor descriptor, string oscAddress)
string
for the OSC address. The class and property can be specified in a variety of ways, depending on what's convenient. For instance, the application may have PropertyDescriptor
s handy, because they were used for another purpose.
The OSC address is "fixed" to remove any illegal characters by calling OscServices.FixPropertyAddress()
. The name is also made unique by using a UniqueNamer
object.
The OSC address and class property association is stored in an OscAddressInfo
object.
If you use a PropertyDescriptor
to specify the property, you can also specify a type converter for the property. This allows converting the arguments in the OSC message to the right type for the C# property. For an example of doing this, see AddPropertyAddress Method Example.
OscAddressInfo
contains information for mapping an OSC address to a property on a C# class. Its constructor takes either a PropertyInfo
or PropertyDescriptor
to specify the class and property:
OscAddressInfo(string oscAddress, PropertyInfo propertyInfo)
OscAddressInfo(string oscAddress, PropertyDescriptor descriptor)
OscAddressInfo
has several useful fields, properties, and methods:
-
string Address
: Field for the OSC address. -
Type CompatibleType
: Field for the type of the object that has the C# property. -
Type PropertyType
: Field for the C# property's type. -
List<object> Addressable
: Field containing a list of selected objects that have been converted into OSC-addressable objects of the type of the object that has the C# property, which is the same type as inCompatibleType
. -
string PropertyName
: Get the property's name. -
PropertyDescriptor PropertyDescriptor
: Get thePropertyDescriptor
, if one was used to create thisOscAddressInfo
. It may benull
. -
object CommonToAddressable(object common)
: Attempt to convert a given object into an object that is compatible with the object holding the property, whoseType
is inCompatibleType
. TheOscService.Addressable
field contains objects returned from this method. -
object GetValue(object compatible)
: Get an object with the value of the C# property on the given object. -
void SetValue(object compatible, object data)
: Set the value of the associated C# property on the given object.
OscService
implements IOscService
:
-
void Send(IEnumerable<Tuple<string, object>> addressesAndData)
: Send the OSC addresses and data objects to each destination endpoint inDestinationEndpoints
, asynchronously. EachTuple
is a pairing of the OSC address and an object for the property value associated with that address. If there are multipleTuple
s with the same OSC address, only the last one is sent. This avoids sending values unnecessarily for an object whose property is changing very quickly, as for a property whose value is set by a slider control, for example. This method returns immediately, queuing the data to be sent. -
event EventHandler<OscMessageReceivedArgs> MessageReceived
: When a message has been received, delegates subscribing to this event are called with anOscMessageReceivedArgs
providing message information, as described in OscMessageReceivedArgs Class. TheOscCommandReceiver
component uses this mechanism; for details, see OscCommandReceiver Component. Instead of using this,OscService
subscribes to theMessageReceived
event on the UDP server object
OscService
offers the following properties:
-
int ReceivingPort
: Get or set the port number that is used to receive UDP or TCP/IP messages. The default is 8000. -
IPAddress ReceivingIPAddress
: Get this application's receiving IP address, which is only valid afterIInitializable.Initialize()
is called. When setting, you can use the static methodGetLocalIPAddresses()
to determine which valid receiving IP addresses are available. -
IPEndPoint ReceivingEndpoint
: Get or set the local receiving endpoint, which is a combination of theReceivingIPAddress
andReceivingPort
. -
IPAddress PreferredReceivingIPAddress
: Get or set the preferred receiving IP address. In the case where there are multiple valid local IP addresses to choose from, this is the one that is used. When setting, the value is ignored if it's not a currently valid local IP address. -
IPEndPoint DestinationEndpoint
: Get or set the primary destination endpoint (IP address and port number). Setting this adds the value to the list in theDestinationEndpoints
property. Additional connected devices can be communicated with if they initiate contact. This property cannot be null. If the IP address portion of the endpoint isIPAddress.None
("255.255.255.255"), no messages are sent, and a large amount of processing is possibly saved. This property can be set in the dialog created by theOscDialog
class; for details, see OscDialog Class. -
IEnumerable<IPEndPoint> DestinationEndpoints
: Get the current list of known IP endpoints (IP address and port number) to broadcast to. This always includesDestinationEndpoint
, but may also include additional endpoints: when a message is received, its sender is automatically added to this list. -
IEnumerable<OscAddressInfo> AddressInfos
: Get all theOscAddressInfo
s for OSC addresses that map onto C# properties. -
string StatusMessage
: Get a user-readable status message that describes the current status ofOscService
. It includes the IP addresses broadcast to, the number of OSC messages received, and the number of OSC messages sent.
OscService
methods available include:
-
IEnumerable<IPAddress> GetLocalIPAddresses()
: A utility method for getting local IP addresses that could work for OSC communications. The first address in the enumeration is the best guess, but there can be multiple valid network adapters to choose from. You can use this method to get an address for theReceivingIPAddress
property. -
object ObservableToCommon(object observable)
: Convert a type of object reported by an event in theIObservableContext
into a type of object that is compatible with an object in the selection context (after passing throughSelectedToCommon()
). The default is to assume that no conversion is necessary. For some applications, it may be necessary to convert the object, for example, adapting it to aDomNode
:
protected override object ObservableToCommon(object observable)
{
return observable.As<DomNode>();
}
-
object SelectedToCommon(object selected)
: Convert a type of object from anISelectionContext
into a type of object that is compatible with an object fromIObservableContext
(after passing throughObservableToCommon()
). The default is to assume that no conversion is necessary. For some applications, it may be necessary to convert the object, for example, adapting it to aDomNode
. -
IEnumerable<Tuple<string, object>> GetCustomDataToSend(object common)
: Get a set of OSC addresses and associated data to send in messages, along with the other messages sent after a context change, as discussed in Getting C Sharp Properties and Sending Messages. TheLemurOscService
component overrides this method. -
object PrepareDataForSending(object data, object common, OscAddressInfo info)
: Transform the property data, if necessary, before sending it to a connected OSC device. The default is to not change the data; theLemurOscService
component overrides this method. -
void SendSynchronously(IList<Tuple<string, object>> addressesAndData)
: Send pairs of OSC addresses and property data objects to the current destination endpoints immediately, on the current thread. TheLemurOscService
component overrides this method. -
void SendPacket(IList<Tuple<string, object>> addressesAndData, int first, int count)
: Send a single packet synchronously to each destination device inDestinationEndpoints
, with the given pairs of OSC addresses and data objects. -
void SetValue(string oscAddress, object addressable, object value, OscAddressInfo info)
: Set a value on a property with an associated OSC address. -
IEnumerable<OscAddressInfo> GetInfos(object selected)
: Get theOscAddressInfo
s for the given object, indicating the properties that have OSC addresses associated with them.
LemurOscService
provides OSC support that is compatible with the Lemur iPad® application, from the Liine company. For more information on Lemur, see the Lemur site.
LemurOscService
derives from OscService
and overrides several of its methods to perform special processing:
-
IEnumerable<Tuple<string, object>> GetCustomDataToSend(object common)
: This updates the "interface" value, which can switch GUI screens in Lemur. -
object PrepareDataForSending(object data, object common, OscAddressInfo info)
: This converts any Boolean values to floats, which is what Liine Lemur uses for Boolean values, even though OSC has a Boolean type. -
void SendSynchronously(IList<Tuple<string, object>> addressesAndData)
: This method limits the number of messages sent in a packet to 16 to deal with a Liine Lemur limitation.
The Add2DPointProperty()
method shows how to use OscService.AddPropertyAddress()
:
public string Add2DPointProperty(ChildInfo childInfo, AttributePropertyDescriptor childAttributeDesc, string oscAddress)
{
oscAddress = OscServices.FixPropertyAddress(oscAddress);
var xCoordDesc = new ChildListFloatingPointArrayDesc(childInfo, childAttributeDesc, 0);
AddPropertyAddress(xCoordDesc, oscAddress + "/x");
var yCoordDesc = new ChildListFloatingPointArrayDesc(childInfo, childAttributeDesc, 1);
AddPropertyAddress(yCoordDesc, oscAddress + "/y");
return oscAddress;
}
Add2DPointProperty()
adds OSC addresses (one for the x-coordinate and one for the y-coordinate) for a list of DOM children that have attributes that are arrays of float
s. Each array of float
s represents a 2D point where the first float
is the x-coordinate and the second float
is the y-coordinate.
ChildListFloatingPointArrayDesc
is a private class derived from ChildAttributePropertyDescriptor
:
private class ChildListFloatingPointArrayDesc : ChildAttributePropertyDescriptor
{
public ChildListFloatingPointArrayDesc(ChildInfo childInfo, AttributePropertyDescriptor childAttributeDesc, int coordinateIndex)
: base(
childAttributeDesc.Name,
childAttributeDesc.AttributeInfo,
childInfo,
childAttributeDesc.Category,
childAttributeDesc.Description,
childAttributeDesc.IsReadOnly,
null, //editor
childAttributeDesc.Converter)
{
...
Note the last parameter in the call to the base method, which provides a converter for the type. This converter is used to convert the OSC message arguments into a float
value for the coordinate.
OscCommandReceiver
listens for OSC messages that can trigger the execution of commands in the application. All commands that are known to ICommandService
can be triggered. This allows an OSC application/device to control an ATF application. The format for such an OSC address is:
{source} /{menu name}/{command name} {source}
with spaces and illegal characters removed.
In the application's TypeCatalog
, the OscCommandReceiver
component can only see commands that were previously registered with the ICommandService
. Therefore, to see all commands, put OscCommandReceiver
last in the TypeCatalog
.
OscCommandReceiver
uses the OscService.MessageReceived
event to be notified when OSC messages arrive. OscCommandReceiver
imports a ICommandService
implementer, which it employs to execute commands received.
OscCommands
is a component that provides menu commands that may be useful for users of OscService
. It implements two commands:
- OSC Info: Display the status of the OSC service and list the available OSC addresses and associated properties in a dialog, using the
OscDialog
class. You can see all OSC address mappings this way. For an illustration and explanation of this dialog, see OscDialog Class. - Copy OSC Address: Copy the OSC address of the clicked on property in the property editor to the clipboard. This menu item only appears when the property has an associated OSC address.
OscCommands
implements IContextMenuCommandProvider
so that context menus can display these commands where appropriate.
OscDialog
derives from System.Windows.Forms.Form
and creates the OSC information dialog, shown in this illustration:
The dialog's ListView
shows all mapped OSC addresses along with their associated property name, data type, and C# class. It also provides information, such as the application's local IP address and port number that you might need to configure other OSC applications/devices.
The dialog performs several other functions:
- Modify the application's local IP address and port number.
- Modify the destination's local IP address and port number. Use this to send OSC messages to a device at this IP address. This adds this destination to the
OscService.DestinationEndpoints
, property, which holds the current list of known IP endpoints (IP address and port number) to broadcast to. For more information on this and otherOscService
properties, see OscService Properties. - Clicking the To Clipboard button formats and copies information in the top part of the dialog, such as the application's local IP address, to the clipboard.
- Clicking the *OK* button sets the application's and destination's local IP addresses and port numbers that were entered in the dialog's fields.
The Lemur application allows constructing your own user-interface and macros, and then you can use an iPad® to run commands or edit objects in an OSC-enabled ATF application.
In addition to the configuration mentioned in OscDialog Class, you need to do the following to use Lemur with an OSC-enabled ATF application.
To create a button in Lemur that can execute commands in the ATF OSC-enabled application:
- Set the button's behavior mode to "Pad".
- For every ATF application command you want to use, set the OSC address to be the command name, which usually looks something like "/File/Save". If the command name has spaces in it, remove the spaces, because these are removed by
OscService
, as mentioned in Mapping an OSC Address onto a C Sharp Class Property. - Set the trigger condition to be when the value goes from 0 to positive. This prevents the command from being executed twice on each button click.
- Home
- Getting Started
- Features & Benefits
- Requirements & Dependencies
- Gallery
- Technology & Samples
- Adoption
- News
- Release Notes
- ATF Community
- Searching Documentation
- Using Documentation
- Videos
- Tutorials
- How To
- Programmer's Guide
- Reference
- Code Samples
- Documentation Files
© 2014-2015, Sony Computer Entertainment America LLC