Skip to content

Framework

Jump to bottom
gabrielnaves edited this page Jul 11, 2016 · 13 revisions

The vision system used by the unball team uses its own computer vision framework. This page provides information about how this framework works and how to use it.

Table of contents:

  1. About
  2. How it works
  3. How to use it
  4. Examples

About the framework

The purpose of the framework is to provide a simple, highly customizable interface for the implementation of multiple object tracking algorithms.

How it works

The framework consists of three steps: segmentation, identification and tracking. Both the segmentation and identification steps work by keeping lists of implemented algorithms. These algorithms are to be implemented by the user, and may be vastly different. The framework requires only that they follow a determined structure, that will be described ahead. This structure is, of course, extensible.

The segmentation step requires an algorithm that operates on the raw image given to the framework, and produces another image as the output.

The identification step requires an algorithm that operates on the output of a segmentation algorithm, and produces information about the object(s) whose position(s) are being identified.

In the tracking step each tracked object takes the results from a given identification algorithm and uses it to track itself.

The information about each object (the algorithms it uses) is stored in a configuration file, vision.yaml. This file follows a predefined structure, that is described in detail on the next section.

How to use it

Creating a segmentation algorithm:

In order to create a segmentation algorithm, you need to create a class that contains the algorithm's implementation. This class has the following requirements:

  1. It needs to publicly inherit from the base class SegmentationAlgorithm;
  2. It needs to declare its own type, within its public members, using the macro ALGORITHM_TYPE;
  3. It needs to register itself as a segmentation algorithm, within its private members, using the macro REGISTER_ALGORITHM_DEC;
  4. It needs to complete the self-registration using the macro REGISTER_ALGORITHM_DEF on the class' .cpp file.
  5. It needs to implement the void run() public method.
  6. It needs to write its output to the cv::Mat output_image_ field.

If the algorithm requires any logic prior to its continuous execution in the run() method, the class may implement the void init() public method. The class should not implement a constructor.

Creating an identification algorithm:

In order to create an identification algorithm, you need to create a class to contain the algorithm's implementation. This class has the following requirements:

  1. It needs to publicly inherit from the base class IdentificationAlgorithm;
  2. It needs to declare its own type, within its public members, using the macro ALGORITHM_TYPE;
  3. It needs to register itself as a segmentation algorithm, within its private members, using the macro REGISTER_ALGORITHM_DEC;
  4. It needs to complete the self-registration using the macro REGISTER_ALGORITHM_DEF on the class' .cpp file.
  5. It needs to implement the void run() public method.
  6. It needs to write its output to the std::shared_ptr<IdentificationOutput> output_info_ field.

Like the segmentation class , if the identification algorithm requires any logic prior to its continuous execution in the run() method, the class may implement the void init() public method. The class should not implement a constructor.

Creating a tracked object:

In order to have the framework track an object, all you need to do is edit the configuration file, vision.yaml. Here is an example of this file:

vision:
    object_list:
     - "dummy_object"
    dummy_object:
        segmentation_algorithm: "DummySegmentationAlgorithm"
        identification_algorithm: "DummyIdentificationAlgorithm"

The first thing to note here is the object_list field. This field contains the list of objects that are currently being tracked. After this field comes the description for each object. Within its description the object must inform the segmentation and identification algorithms it currently uses. Upon execution, the framework will parse this config file and create all objects on the object_list, as well as their algorithms.

Note that having the description for an object that is not on the object list will not result in error. This object, however, will not be tracked, and its algorithms will not be created if they're not being used by other objects.

Examples

Example 1: Dummy algorithms

The framework contains two dummy algorithms, DummySegmentationAlgorithm and DummyIdentificationAlgorithm. Let's first take a look at the dummy segmentation algorithm:

dummy_segmentation_algorithm.hpp:

class DummySegmentationAlgorithm : public SegmentationAlgorithm
{
  public:
    ALGORITHM_TYPE(DummySegmentationAlgorithm);

    void run();

  private:
    REGISTER_ALGORITHM_DEC(DummySegmentationAlgorithm);
};

Note the usage of the macros ALGORITHM_TYPE and REGISTER_ALGORITHM_DEC. The first macro defines the algorithm's name_ field, and the second one registers the class as a valid segmentation algorithm.

dummy_segmentation_algorithm.cpp:

#include <vision/dummy_segmentation_algorithm.hpp>

REGISTER_ALGORITHM_DEF(DummySegmentationAlgorithm);

void DummySegmentationAlgorithm::run()
{
    ROS_INFO("Running dummy segmentation algorithm");
    RawImage::getInstance().getRawImage().copyTo(output_image_);
}

Note the usage of the macro REGISTER_ALGORITHM_DEF. This macro completes the algorithm's registration as a valid algorithm. Also, note that the run() method doesn't actually do anything: it only copies the raw image as its output.

Now, let's take a look at the DummyIdentificationAlgorithm class:

dummy_identification_algorithm.hpp:

class DummyIdentificationAlgorithm : public IdentificationAlgorithm
{
  public:
    ALGORITHM_TYPE(DummyIdentificationAlgorithm);

    void run();

  private:
    REGISTER_ALGORITHM_DEC(DummyIdentificationAlgorithm);
};

This declaration is a lot like the segmentation algorithm. The macros in use serve the same purpose. Let's move on.

dummy_identification_algorithm.cpp:

#include <vision/dummy_identification_algorithm.hpp>

REGISTER_ALGORITHM_DEF(DummyIdentificationAlgorithm);

void DummyIdentificationAlgorithm::run()
{
    output_info_ = std::make_shared<IdentificationOutput>();
    output_info_->object_center = cv::Point(10,10);
}

Again, the implementation file resembles the previous segmentation one. A key difference here is the nature of the identification algorithm's output, as it is a shared pointer. This allows overriding of the identification output if you need it.

Clone this wiki locally