Framework

The vision system used by the unball team uses its own computer vision framework. This page provides information about how this framework works and how to use it.

Table of contents:

1. About
2. How it works
3. How to use it
4. Examples

About the framework

The purpose of the framework is to provide a simple, highly customizable interface for the implementation of multiple object tracking algorithms.

How it works

The framework consists of three steps: segmentation, identification and tracking. Both the segmentation and identification steps work by keeping lists of implemented algorithms. These algorithms are to be implemented by the user, and may be vastly different. The framework requires only that they follow a determined structure, that will be described ahead. This structure is, of course, extensible.

The segmentation step requires an algorithm that operates on the raw image given to the framework, and produces another image as the output.

The identification step requires an algorithm that operates on the output of a segmentation algorithm, and produces information about the object(s) whose position(s) are being identified.

In the tracking step each tracked object takes the results from a given identification algorithm uses it to track itself.

The information about each object (the algorithms it uses) is stored in a configuration file, vision.yaml. This file follows a predefined structure, that is described in detail on the next section.

How to use it

Creating a segmentation algorithm:

In order to create a segmentation algorithm, you need to create a class that contains the algorithm's implementation. This class has the following requirements:

1. It needs to publicly inherit from the base class SegmentationAlgorithm;
2. It needs to declare its own type, within its public members, using the macro ALGORITHM_TYPE;
3. It needs to register itself as a segmentation algorithm, within its private members, using the macro REGISTER_ALGORITHM_DEC;
4. It needs to complete the self-registration using the macro REGISTER_ALGORITHM_DEF on the class' .cpp file.
5. It needs to implement the void run() public method.
6. It needs to write its output to the cv::Mat output_image_ field.

If the algorithm requires any logic prior to its continuous execution in the run() method, the class may implement the void init() public method. The class should not implement a constructor.

Creating an identification algorithm:

In order to create an identification algorithm, you need to create a class to contain the algorithm's implementation. This class has the following requirements:

1. It needs to publicly inherit from the base class IdentificationAlgorithm;
2. It needs to declare its own type, within its public members, using the macro ALGORITHM_TYPE;
3. It needs to register itself as a segmentation algorithm, within its private members, using the macro REGISTER_ALGORITHM_DEC;
4. It needs to complete the self-registration using the macro REGISTER_ALGORITHM_DEF on the class' .cpp file.
5. It needs to implement the void run() public method.
6. It needs to write its output to the std::shared_ptr<IdentificationOutput> output_info_ field.

Like the segmentation class , if the identification algorithm requires any logic prior to its continuous execution in the run() method, the class may implement the void init() public method. The class should not implement a constructor.

Creating a tracked object:

In order to have the framework track an object, all you need to do is edit the configuration file, vision.yaml. Here is an example of this file:

vision:
    object_list:
     - "dummy_object"
    dummy_object:
        segmentation_algorithm: "DummySegmentationAlgorithm"
        identification_algorithm: "DummyIdentificationAlgorithm"

The first thing to note here is the object_list field. This field contains the list of objects that are currently being tracked. After this field comes the description for each object. Within its description the object must inform the segmentation and identification algorithms it currently uses. Upon execution, the framework will parse this config file and create all objects on the object_list, as well as their algorithms.

Note that having the description for an object that is not on the object list will not result in error. This object, however, will not be tracked, and its algorithms will not be created if they're not being used by other objects.

Examples

Example 1: Dummy algorithms

The framework contains two dummy algorithms, DummySegmentationAlgorithm