diff --git a/README.md b/README.md index 39af52c..dc6181a 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,195 @@ -# README # +# Low-Level C# SDK -This README would normally document whatever steps are necessary to get your application up and running. +Welcome to the Weart Low-Level C# SDK documentation. -### What is this repository for? ### +The SDK allows to connect to the Weart middleware and perform various actions with the TouchDIVER devices: +* Start and Stop the middleware operations +* Calibrate the device +* Receive tracking data from the devices +* Send haptic effects to the devices -* Quick summary -* Version -* [Learn Markdown](https://bitbucket.org/tutorials/markdowndemo) +# Architecture +C++ SDK Architecture -### How do I get set up? ### +# Setup +The minimum setup to use the weart SDK consists of: +* A PC with the Middleware installed +* A TouchDIVER device +* A C# project using the Low-Level SDK -* Summary of set up -* Configuration -* Dependencies -* Database configuration -* How to run tests -* Deployment instructions +The SDK can be downloaded as a zip file containing all the necessary files. +To use it in your C# project, unzip it and move the files in a folder inside your project. +Then, add the sdk files in your project. +In Visual Studio, this can be done by including the sdk directory into the project (Right Click on Solution -> Add -> Existing Directory). -### Contribution guidelines ### +To start using the SDK in your project, start the Middleware application and connect a TouchDIVER device to it. -* Writing tests -* Code review -* Other guidelines +Import the main namespaces: -### Who do I talk to? ### +~~~~~~~~~~~~~{.cs} +using WeArt.Core; +using WeArt.Components; +~~~~~~~~~~~~~ -* Repo owner or admin -* Other community or team contact \ No newline at end of file +Create the WeArtClient and start the communication with the middleware: +~~~~~~~~~~~~~{.cs} + // Create weart client + WeArtClient weArtClient = new WeArtClient { IpAddress = WeArtNetwork.LocalIPAddress, Port = 13031 }; + + // Start connection and send start message to middleware + weArtClient.Start(); +~~~~~~~~~~~~~ + +@note The call to WeArtClient.Start() will also start the middleware, so be sure to have connected the devices before + +# Features + +## Start/Stop Client +As seen in the previous section, to start the middleware operations, call the Start() method. + +~~~~~~~~~~~~~{.cs} + weArtClient->Start(); +~~~~~~~~~~~~~ + +To stop the middleware, call the Stop() method. + +~~~~~~~~~~~~~{.cs} + weArtClient->Stop(); +~~~~~~~~~~~~~ + +## Devices calibration +After starting the communication with the middleware, it's now possible to calibrate the TouchDIVER devices. +The calibration allows to set the initial offsets of each thimble relative to the control unit position, in order to improve the finger tracking output. + +The SDK client allows to add callbacks to monitor the calibration procedure status and result. +For example, to start the calibration and print "Success" or "Failed": + +~~~~~~~~~~~~~{.cs} + weArtClient.OnCalibrationResultSuccess += (HandSide hand) => Console.WriteLine("Success!"); + weArtClient.OnCalibrationResultFail += (HandSide hand) => Console.WriteLine("Failed"); + + weArtClient.StartCalibration(); +~~~~~~~~~~~~~ + +## Haptic feedback + +The TouchDIVER allows to perform haptic feedback on the user's finger through its *thimbles*. +Every thimble can apply a certain amount of pressure, temperature and vibration based on the processed object and texture. + +### Haptic Object + +A WeArtHapticObject is the basic object used to apply haptic feedback. +To create one, use the following code: + +~~~~~~~~~~~~~{.cs} + // create haptic object to manage actuation on Right hand and Index Thimble + WeArtHapticObject hapticObject = new WeArtHapticObject(_weartClient); + hapticObject.HandSides = HandSideFlags.Right; + hapticObject.ActuationPoints = ActuationPointFlags.Index; +~~~~~~~~~~~~~ + +The attirbutes handSideFlag and actuationPointFlag accept multiple values. +The next example presents a single haptic object that, when applied a WeArtEffect, will affect both hands and all fingers. + +~~~~~~~~~~~~~{.cs} + hapticObject.HandSides = HandSideFlags.Right | HandSideFlags.Left; + hapticObject.ActuationPoints = ActuationPointFlags.Index | ActuationPointFlags.Middle | ActuationPointFlags.Thumb; +~~~~~~~~~~~~~ + +### Create Effect + +In order to send haptic feedback to the device, an effect must be created by implementing the IWeArtEffect interface. +The class shown here contains the effects without any processing, but it might be extended for other use cases +(e.g. values not directly set, but computed from other parameters). + +~~~~~~~~~~~~~{.cs} +public class TouchEffect : IWeArtEffect { + public event Action OnUpdate; + public Temperature Temperature { get; private set; } = Temperature.Default; + public Force Force { get; private set; } = Force.Default; + public Texture Texture { get; private set; } = Texture.Default; + + public void Set(Temperature temperature, Force force, Texture texture) + { + force = (Force)force.Clone(); + texture = (Texture)texture.Clone(); + + bool changed = !Temperature.Equals(temperature) || !Force.Equals(force) || !Texture.Equals(texture); + + Temperature = temperature; + Force = force; + texture.VelocityZ = 0.5f; + Texture = texture; + + if (changed) + OnUpdate?.Invoke(); + } +} +~~~~~~~~~~~~~ + +After defining the effect class, create the object on which the temperature, force and texture values will be applied: + +~~~~~~~~~~~~~{.cs} +TouchEffect touchEffect = new TouchEffect(); +~~~~~~~~~~~~~ + +### Add or Update Effect + +It's possible to add a new effect to an haptic object, or to update an existing one. + +In the example below, the effect created in the previous section is updated with a new temperature, force and texture. +It is then added to the haptic object if not already present (if the effect is already applied to the thimble, it will update it automatically through the OnUpdate event). + +~~~~~~~~~~~~~{.cs} + // create temperature component + Temperature temperature = Temperature.Default; + temperature.Active = true; // must be active to work + temperature.Value = 0.2f; + + // create force component + Force force = Force.Default; + force.Active = true; + force.Value = 0.7f; + + // create texture component + Texture texture = Texture.Default; + texture.Active = true; + texture.TextureType = TextureType.ProfiledAluminiumMeshFast; + + // effect set proporties + touchEffect.Set(temperature, force, texture); + + // add effect if needed, to thimble + if (hapticObject.ActiveEffect == null) + hapticObject.AddEffect(touchEffect); +~~~~~~~~~~~~~ + +### Remove Effect + +If an effect is not needed anymore, it can be removed from the haptic object with the RemoveEffect method. + +~~~~~~~~~~~~~{.cs} + hapticObject.RemoveEffect(touchEffect); +~~~~~~~~~~~~~ + +## Tracking + +After starting the middleware and performing the device calibration, it's possible to receive tracking data +related to the TouchDIVER thimbles. + +To read these values, create and set a thimble tracker object for monitoring the closure/abduction value of a given finger: +~~~~~~~~~~~~~{.cs} + WeArtThimbleTrackingObject indexThimbleTracking = new WeArtThimbleTrackingObject(weArtClient, HandSide.Right, ActuationPoint.Index); +~~~~~~~~~~~~~ + +Once this object is created, it will start receiving the tracking values. +To access the closure and abduction values, simply use the getters provided by the thimble tracking object. + +The closure value ranges from 0 (opened) to 1 (closed). + +The abduction value ranges from 0 (finger near the hand's central axis) to 1 (finger far from the hand central axis). + +~~~~~~~~~~~~~{.cs} + float closure = indexThimbleTracking.Closure.Value; + float abduction = indexThimbleTracking.Abduction.Value; +~~~~~~~~~~~~~ \ No newline at end of file diff --git a/docs/example.md b/docs/example.md new file mode 100644 index 0000000..6bbbaca --- /dev/null +++ b/docs/example.md @@ -0,0 +1,193 @@ +# Example Project + +An example application is available for download together with the C# and C++ SDK [here](https://weart.it/repository/downloads/sdk-low-level/WEART_Low_Level_API_v1.0.zip). + +The application is implemented as a Universal Windows app in C#. The Visual Studio Solution is present inside an archive, at ```C#/WEART_API_Integration_v1.1.zip```. + +To execute the application, just open the solution with Visual Studio and Run it (by clicking the play button or pressing F5). + +![Example application](./example_app/ExampleApp_Screen.png) + +## SDK Integration + +The SDK is integrated in the example application by copying the sdk source and header files into the ```WEART_API_Integration/WEART_SDK``` directory, and adding them to the visual studio project. +Most of the application code can be found in the ```WEART_API_Integration/MainPage.xaml.cs``` file. + +### Connection to Middleware +The connection is managed using the WeArtClient class together with events to monitor the connection status. To manage the client class, the application uses a WeArtController. + +In particular, upon loading the main window, the application initialises the sdk client and adds a delegate for when the connection +status changes (connected or disconnected). + +~~~~~~~~~~~~~{.cs} +using WeArt.Core; +using WeArt.Components; + +private async void PageLoaded(object sender, RoutedEventArgs e) +{ + WeArtController weArtController = new WeArtController(); + _weartClient = weArtController.Client; + _weartClient.OnConnectionStatusChanged += OnConnectionChanged; + ... +} + +private void OnConnectionChanged(bool connected) +{ + if (connected) { + // Initialise empty effect + CreateEffect(); + } +} +~~~~~~~~~~~~~ + +### Start/Stop middleware +By clicking on the "Start" and "Stop" buttons, it's possible to start and stop the middleware operations. +When clicking on the buttons, the application simply calls the corresponding SDK methods. + +~~~~~~~~~~~~~{.cs} +private void StartClient_Click(object sender, RoutedEventArgs e) +{ + _weartClient.Start(); +} + +private void StopClient_Click(object sender, RoutedEventArgs e) +{ + _weartClient.Stop(); +} +~~~~~~~~~~~~~ + +### Calibration +When the "Calibrate" button is clicked, if the middleware is started, the application will start the calibration procedure. + +In addition, the application register to the calibration status and result events, to inform the user about the calibration progress. + +~~~~~~~~~~~~~{.cs} +private async void PageLoaded(object sender, RoutedEventArgs e) +{ + ... + // handle calibration + _weartClient.OnCalibrationStart += OnCalibrationStart; + _weartClient.OnCalibrationFinish += OnCalibrationFinish; + _weartClient.OnCalibrationResultSuccess += (HandSide hand) => OnCalibrationResult(hand, true); + _weartClient.OnCalibrationResultFail += (HandSide hand) => OnCalibrationResult(hand, false); + ... +} + +// Start calibration +private void StartCalibration_Click(object sender, RoutedEventArgs e) +{ + _weartClient.StartCalibration(); +} + +private void OnCalibrationStart(HandSide handSide) +{ + Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => + { + CalibrationStatusText.Text = $"Calibrating {handSide.ToString().ToLower()} hand..."; + StartCalibration.IsEnabled = false; + }); +} + +private void OnCalibrationFinish(HandSide handSide) +{ + Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => + { + StartCalibration.IsEnabled = true; + }); +} + +private void OnCalibrationResult(HandSide handSide, bool success) +{ + Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => + { + CalibrationStatusText.Text = $"Calibration for {handSide.ToString().ToLower()} hand {(success ? "completed" : "failed")}"; + }); +} +~~~~~~~~~~~~~ + +### Tracking data + +The tracking data is read periodically from multiple WeArtThimbleTrackingObject objects, +declared when the application is opened. + +~~~~~~~~~~~~~{.cs} +private async void PageLoaded(object sender, RoutedEventArgs e) +{ + ... + // Instantiate thimbles for tracking + _leftIndexThimble = new WeArtThimbleTrackingObject(_weartClient, HandSide.Left, ActuationPoint.Index); + _leftThumbThimble = new WeArtThimbleTrackingObject(_weartClient, HandSide.Left, ActuationPoint.Thumb); + + ... + + // schedule timer to check tracking closure value + Timer timer = new Timer(); + timer.Interval = 200; //Milliseconds + timer.AutoReset = true; + timer.Elapsed += OnTimerElapsed; + timer.Start(); + ... +} + +private void OnTimerElapsed(object sender, ElapsedEventArgs e) +{ + Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => + { + ValueIndexRightClosure.Text = _rightIndexThimble.Closure.Value.ToString(); + ValueThumbRightClosure.Text = _rightThumbThimble.Closure.Value.ToString(); + ValueThumbRightAbduction.Text = _rightThumbThimble.Abduction.Value.ToString(); + ValueMiddleRightClosure.Text = _rightMiddleThimble.Closure.Value.ToString(); + + ValueIndexLeftClosure.Text = _leftIndexThimble.Closure.Value.ToString(); + ValueThumbLeftClosure.Text = _leftThumbThimble.Closure.Value.ToString(); + ValueThumbLeftAbduction.Text = _leftThumbThimble.Abduction.Value.ToString(); + ValueMiddleLeftClosure.Text = _leftMiddleThimble.Closure.Value.ToString(); + }); +} +~~~~~~~~~~~~~ + +### Applying effects + +To apply effects to the user hands, the application contains an internal ```TouchEffect``` class, which implements IWeArtEffect. + +The application apply a different effect to the index finger when one of the "Add effect sample" buttons is clicked. + +For example, when clicking the "Add effect sample 1" button, a slight pressure and cold temperature (without texture vibration) +are applied to the right index finger, as shown below: + +~~~~~~~~~~~~~{.cs} +private async void PageLoaded(object sender, RoutedEventArgs e) +{ + ... + // instantiate Haptic Object Right hand for Index Thimble + _hapticObject = new WeArtHapticObject(_weartClient); + _hapticObject.HandSides = HandSideFlags.Right; + _hapticObject.ActuationPoints = ActuationPointFlags.Index; + ... +} + +private void AddEffectSample1_Click(object sender, RoutedEventArgs e) +{ + // create temperature component + Temperature temperature = Temperature.Default; + temperature.Active = true; // must be active to work + temperature.Value = 0.2f; + + // create force component + Force force = Force.Default; + force.Active = true; + force.Value = 0.7f; + + // create texture component + Texture texture = Texture.Default; + texture.Active = true; + texture.TextureType = TextureType.ProfiledAluminiumMeshFast; + + // effect set proporties + _effect.Set(temperature, force, texture); + + // add effect if needed, to thimble + if (_hapticObject.ActiveEffect == null) + _hapticObject.AddEffect(_effect); +} +~~~~~~~~~~~~~ \ No newline at end of file diff --git a/docs/img/architecture.svg b/docs/img/architecture.svg new file mode 100644 index 0000000..509d9a7 --- /dev/null +++ b/docs/img/architecture.svg @@ -0,0 +1,3 @@ + + +
WEART
Middleware
Desktop Tool
WEART...
C# Client
C++ Client
TouchDIVER Devices
TouchDIVER Devices
TCP Socket
TCP Socket
BTLE Communication
BTLE Communication
Text is not SVG - cannot display
\ No newline at end of file diff --git a/docs/img/example_app/ExampleApp_Screen.png b/docs/img/example_app/ExampleApp_Screen.png new file mode 100644 index 0000000..c667377 Binary files /dev/null and b/docs/img/example_app/ExampleApp_Screen.png differ diff --git a/docs/upgrade.md b/docs/upgrade.md new file mode 100644 index 0000000..5d7555c --- /dev/null +++ b/docs/upgrade.md @@ -0,0 +1,15 @@ +# Update from older SDK versions + +To update your application to the latest SDK, download and extract the C# sdk archive, then copy the source/header files in the same place as the older SDK version. + +The new version includes additional files, so it's necessary to add them to the project in order to avoid linking errors. + +This can be done on Visual Studio by right-clicking on the solution, then clicking on Add -> Existing Item and selecting all the SDK files. +On other systems (e.g. cmake) the procedure might be different. + +The SDK is retro-compatible with older versions, so there's no need to update the application code. To see the new feature and fixes added to each version, refer to the Changelog section. + +## Changelog +### Version 1.1 (latest) +* Add calibration procedure start/stop and listener +* Add new default tracking message and values for closure diff --git a/docs/weart.png b/docs/weart.png new file mode 100644 index 0000000..52817bf Binary files /dev/null and b/docs/weart.png differ