Skip to content

Kinesthetic Teaching GUI

Jump to bottom
Ryan Petschek edited this page Aug 1, 2017 · 11 revisions

Prerequisites

Kinesthetic Teaching GUI

Launching the Kinesthetic Teaching GUI

  • Start a roscore
  • If using the simulator, start Gazebo: roslaunch hlpr_gazebo vector.launch
  • Customize and start robot services: roslaunch hlpr_bringup customize_start_robot_services.launch
    • Note: You need to startup all services related to playback and recording. For details how, look at the hlpr_bringup wiki, which has a custom launch file with two flags: kinesthetic_record and kinesthetic_playback. Set these both to true.
  • Start the object locator service: roslaunch object_location object_location_engine.launch (only required if "Locate objects" is checked in the GUI)
  • Launch the GUI: rosrun rqt_kinesthetic_teaching rqt_kinesthetic_teaching
    • If the GUI fails to start, you may have to run rqt --force-discover

Parsing, loading, and inspecting .bag files and folders will now be functional.

Starting additional services

To playback or record keyframes, two action servers must be started. Run the following in new terminals:

  • You don't need this step if you set the correct flags on the customize_start_robot_services.launch file
  • Playback demonstration action server: roslaunch hlpr_record_demonstration start_playback_services.launch
  • Record demonstration action server: roslaunch hlpr_record_demonstration start_record_services.launch

Parsing, loading, and inspecting .bag files

To parse and load a single .bag file containing a demonstration, click the open button or enter the path to the file in the text field and press enter. The .bag file will then be parsed and the keyframe data contained displayed in a tree format at the bottom.

The tree format is as follows:

  • Keyframe number (0-indexed) and ISO 8061 timestamp
  • Robot topic recorded
  • Recorded message fields and their corresponding values

When opening a folder, all .bag and .pkl files that are direct children will be parsed and loaded. The previously mentioned tree format will be preceeded by the file's full path.

Parsing, loading, and inspecting .pkl files

Previewing or playing a .bag file automatically generate an associated .pkl file in the same directory containing information about the motion plan that MoveIt! generated. Since the planned playback trajectory is guarenteed not to change when playing from a .pkl file, it can be used to ensure that the robot won't collide with any obstacles before actual execution. The generated .pkl file will end in _joint.pkl if "Locate objects" was unchecked and _EEF.pkl if checked.

To parse and load a single .pkl file, click the open button or enter the path to the file in the text field and press enter like with .bag files. The .pkl file will then be parsed and displayed in addition to its original source .bag file, if found. The "Plan for #N" displayed directly corresponds to the keyframe number in the associated bag file. Keyframes are sometimes skipped during planning if they are nearly identical.

Tree format:

  • Corresponding keyframe number (0-index) and ISO 8061 timestamp
  • Type (Joint space or EEF space)
  • Target
    • If joint space, details about each joint's state
    • The (desired) pose of the EEF at this keyframe
  • Keyframe data
    • Only appears if a corresponding .bag file with the same name is in the same directory and is formatted identically to the description above.

Previewing and playing back demonstrations

Currently, due to limitations with the playback demonstration action server, only full .bag or .pkl files can be previewed or played from start to finish. Individual keyframes or selected keyframes can not be played.

With a .bag or .pkl file open and the playback demonstration action server started, keyframe preview or playback can be started by clicking the "Preview arm trajectory" or "Play demo" buttons respectively at the bottom of the window.

With a folder of .bag and .pkl files open and the playback demonstration action server started, keyframe playback can be started by selecting a file path name in the tree view and clicking the "Preview arm trajectory" or "Play demo" buttons respectively at the bottom of the window.

Playing back relative to an object

For use when you want keyframe actions to adjust if an object is moved or rotated in the scene.

To activate the object locator, ensure that roslaunch object_location object_location_engine.launch is running. Then, check the "Locate objects" box. This will make the recording and playback servers act on the EEF pose rather than the exact recorded joint states. Uncheck the box for faster recording and playback that only respects the exact joint states recorded.

Before previewing or playing a demonstration when "Locate objects" is checked, you must select an object to play back relative to from the drop down menu to the right of the "Play demo" button. These labels are generated during recording by the RAIL object detector via rail_object_locator.

Note: the object locator will run whenever "Start here", "Preview arm trajectory", or "Play demo" are clicked and may take a minute or more to complete during which the GUI will become unresponsive.

Keyframe recording and saving

Keyframe recording is enabled by clicking the "Save" button at the top of the window. Optionally, a timestamp can be added to the save file's name by checking the "Add timestamp to new saves" box. The output window will be cleared.

Using the UI to record and save demonstrations

To begin adding keyframes, move the robot to its startig position and click the "Start trajectory" or "Start keyframe" record buttons. If "Locate objects" is checked, the GUI may become unresponsive for about a minute while objects are located in the scene. The status area at the bottom right will then display "Keyframe recording started" and the secondary status above the output window will display "1 keyframe(s) recorded" as the starting position has been recorded as a keyframe.

New keyframes will display in the output window in a similar format to when saved files are opened. However, recorded topics and message values cannot (yet) be viewed while a recording is in progress.

The robot's arm can now be moved. It's position will be continually recorded if a trajectory was started. Otherwise, new keyframes can be added by clicking the "Add keyframe" button.

Clicking the "Open hand" and "Close hand" buttons will open / close the robot's gripper and automatically record a new keyframe.

To end the recording, move the robot to its final position and click the "End" button. A keyframe for the final position will be recorded and the .bag file will be written, closed, and then automatically parsed for immediate inspection and playback.

Using speech commands to record and save demonstrations hands-free

With a new keyframe recording opened by clicking the "Save" button, speech commands published to the SpeechListener class' speech topic will be read and acted upon. This allows for hands-free control of keyframe recording.

Clone this wiki locally