Driver Station Speech Telemetry
This tutorial shows how FTC Blocks can play speaking audio on the Driver Station (DS) phone or Driver Hub. ¡En Español también! Aussi en Français! ...and many other languages.
Spoken audio can provide helpful feedback to programmers and robot operators, while the OpMode is running. This can be more useful than audio on the Robot Controller (RC) phone, which is hard to hear in competition. Note: the REV Control Hub has no built-in speaker.
Many thanks to Google engineer Liz Looney for this fun and valuable feature!
Spoken audio plays on the RC phone using TextToSpeech commands, found in the Android menu of Blocks. This tutorial covers speech on the DS devices using other commands in the Telemetry menu.
A different FTC Wiki tutorial shows how FTC Blocks can play short recorded sound files on the RC phone and DS devices.
Open any existing or new Autonomous OpMode that will use this feature. FTC Blocks doesn't offer a Sample OpMode for this DS speaking feature; the "ConceptTextToSpeech" Sample plays spoken text only on the RC phone.
In the left side menu, open Utilities, then Telemetry.
Click and drag out the purple Block "call Telemetry.speak". For now, don't use the larger Block with languageCode and countryCode.
Place that basic Block in your OpMode, where you want the speaking to happen.
Change the default "text" to the actual message that you want. The OpMode will use Android's Text-To-Speech (TTS) feature to convert your typed words into spoken audio.
Unlike the regular typed Telemetry that appears on the DS screen, spoken audio Telemetry does not require a "Telemetry.update" command.
The command Telemetry.speak begins playing the spoken message. The OpMode will immediately proceed to the next programmed command. If the program ends soon, your message might not finished playing. Or, you might want the robot's next action to begin only when the audio speech has finished.
So, you can manually allow time for your audio to play. Unlike the Blocks TTS feature for RC phones, there is no Boolean variable "isSpeaking" that becomes FALSE when the audio has finished playing.
You can estimate or measure the duration of the audio. This works fairly well, assuming the audio begins playing immediately.
The Sleep method is simple and will work, but no other new commands can begin during that time.
Using a Timer is often a better way to wait. Create a new Timer object, then loop based on the elapsed time.
Sometimes it's helpful for a Timer loop to include a text message with robot status.
The loop can also include Blocks for checking sensors, starting or ending other robot actions, etc. These would not be possible during a long Sleep command.
This waiting technique works best in Autonomous programs, where programmed events happen in sequence (linear OpMode).
Using timers in a TeleOp program (iterative OpMode) is different. The above code samples appear in the OpMode's INIT section (before waitForStart), to simply introduce the concept of waiting for audio to finish. Do not use the above samples in the RUN section of a real FTC TeleOp OpMode. See the Advanced Topic below, called Sound triggers in TeleOp.
Note: more uses of Timers in FTC Blocks are described in this FTC Wiki tutorial.
Save and run your Autonomous OpMode. Turn up the audio volume on the Driver Hub or DS phone (not the ringtone volume).
The first one or two playbacks of a TTS or recorded audio clip might be delayed for initial processing. After this, the audio should start promptly and consistently. Consider pre-playing key sounds, for reliable fast response later.
If the words don't sound the way you want, adjust the text. Clearly understood audio is probably more important than the exact spelling or punctuation of the typed text.
Those are the basics! Enjoy this fun and useful feature of FTC Blocks.
- Use in Development
- Numeric and Boolean Values
- Sound triggers in TeleOp
- Languages
- Toggle debug audio
- Boolean for playback completion
Sounds can be a very useful tool for programming and debugging. They can mark the transition from one Autonomous action to another, which often happens too fast to observe and study. Audio provides fast convenient feedback, directly from your running code. For example:
Speak "Drive forward 10 inches and stop." - did that happen?
Speak "Step 1", "Step 2", "Step 3", etc. - did any steps get skipped?
Speak "Color sensor sees Red." (or Blue) - was the sensor correct?
If your OpMode is not behaving as expected, audio feedback can help you find the problem in your code.
Spoken text can be generated from other sources besides typing, using the "create text" Block. Here are two examples that say (in words) numeric and Boolean values.
The left command says the words "One hundred". The right command says "Is the color red? True".
Even better, this feature can say (in words) a value coming directly from an encoder, sensor or gamepad. Some values have a large number of digits -- you don't need to hear all of them.
Note: this code sample illustrates the concept of speaking sensor values. Do not pause a real FTC OpMode for 3 seconds!
The "create text" Block expands the ways that DS Speech can help with development and debugging. And in competition, this can reduce the need for a Coach/Driver to constantly monitor and call out robot data from the DS screen.
Special handling is sometimes needed for playing sounds and speech in a TeleOp or driver-controlled program.
TeleOp programs are iterative OpModes, meaning the code runs in a single overall loop. This loop cycles or repeats very fast, typically every 10 to 50 milliseconds.
If a gamepad button triggers your audio playback, there's a special problem. The overall OpMode loop usually cycles faster than a human releases the button, causing a staccato of repeated starts.
You might think of solving this with a timed pause (described above in Step 2) immediately after detecting a button-press, to allow the human driver enough time to release the button. But simple timed pauses should not be used in an iterative OpMode. The program's main loop needs to run as fast as it can, without significant delays in each cycle.
Instead you could start a timer when the button is first pressed, then ignore that button until the timer has finished. The timer gets checked with each loop of the main program.
Another technique is to set a Boolean variable, or 'flag', to indicate the button was pressed (the first time), then ignore that button while the variable is TRUE. The variable can be reset to FALSE when the button has been physically released; no timer is needed.
You can also use a timer and Boolean together. This is shown as Example #5 called Device timer in TeleOp, at a separate timers tutorial in the FTC Wiki.
All these solutions can also be used when the audio is triggered by a sensor or other event on the robot. However if the audio starts when a touch sensor is pressed, for example, the playback could keep starting continuously if the sensor stays "pressed". Annoying! Use a timer and/or flag to initiate a single start, allowing the iterative OpMode to continue looping without repeating the audio starts.
In Android, you can change the default language for TTS playback. This is not the same as the overall default language of the Android device.
Language set-up requires a one-time download to the DS device, so close the FTC app and open a regular Wi-Fi internet connection.
In the DS device's Settings menu, open "Languages & input".
LEFT IMAGE: Choose "Text-to-speech output", not "Languages".
RIGHT IMAGE: Next to "Google TTS Engine", touch the Settings icon (small gear).
LEFT IMAGE: Select "Install voice data".
RIGHT IMAGE: Choose your preferred TTS language (and country/region dialect).
LEFT IMAGE: Touch the "Download" icon, and wait for the language package to download from the internet.
RIGHT IMAGE: After download, choose a preferred voice; touch each one (on the Voice number) to hear a sample. Then select the one you prefer. The voice can be changed later without download, at this menu.
LEFT IMAGE: Go back to the previous screen, and touch Language.
RIGHT IMAGE: Scroll down to display the preferred TTS language that you downloaded.
LEFT IMAGE: Select your preferred (downloaded) TTS language. This can be changed anytime at this menu, for new languages already downloaded.
RIGHT IMAGE: The previous screen will now list your selected TTS language for this DS device. This is not the same as the overall Android system language.
That's it! Exit the Android Settings menu, "Forget" the internet Wi-Fi, Airplane Mode on, turn Wi-Fi back on. Your settings on this DS device did not affect the language or TTS settings on the RC device.
After setting the new TTS language, you may continue to use the basic purple "Telemetry.speak" Block in the instructions above. Your text can now be typed in that language, including special/native text characters -- copy them from other documents or from, for example, Microsoft Word's Insert Symbol. Numbers and Boolean values will be spoken in that language too!
You may have noticed a larger purple Block with languageCode and countryCode. This allows choosing from multiple TTS languages downloaded to the DS device. Its use may be limited by memory allocation and other system-level factors.
To refresh that large Block's drop-down menu after installing a new Android TTS language, pull that Block again from the menu. A previously placed Block will not update itself with added language codes.
If that Block's drop-down menu still does not offer the code you need, pull the green label away. In the remaining bare grey field, type the country or region code needed. These codes follow the standard ISO conventions.
In some cases, a language code alone will work, with no country code needed. ¡Buena suerte! Bonne chance!
Audio cues for key Autonomous OpMode actions are useful for debugging, but can slow down the overall action sequence. Manually adding, removing and re-adding these spoken Telemetry messages takes a lot of work. Consider using a single Boolean variable, or toggle, to turn on and off all such messages.
When the OpMode is ready for smooth/continuous runs, simply change the toggle (e.g. "useAudio") from TRUE to FALSE at the top of the program. Now there's no need to manually add/delete all the message code from your OpMode.
Note: consider adding timed pauses between robot actions, also subject to the same overall toggle. This can greatly improve your Autonomous program testing.
A more advanced solution is for the Driver/Coach to set this toggle during INIT, using the gamepad. A fun programming exercise!
As noted above, FTC Blocks has no Boolean variable "isSpeaking" for Text-To-Speech Telemetry on the DS device. But that Boolean does exist for Robot Controller (RC) playback. So, you could play the same message on both devices, and use the RC Boolean to know when the messages have stopped playing.
This parallel/Boolean technique assumes that (a) both messages start at the same time, and (b) you have not adjusted the playback speed of the RC text. You might find the RC audio starts slightly later than the DS audio. If you find the "echo" annoying, try a short (e.g. 50 millisecond) Sleep between the two commands, as shown above. Or you could turn down the audio on the RC phone, but you'll miss other RC audio signals and warnings. Note that the REV Control Hub has no built-in speaker -- no echo!
Again, the first one or two playbacks might need extra time for processing. Consider pre-playing key sounds, for reliable fast response later.
Note the above code sample should not be placed in the RUN section of an iterative OpMode, as discussed above in Sound triggers in TeleOp. It could be used in the INIT section, or in a linear (Autonomous) OpMode.
Questions, comments and corrections to: [email protected]
-
TensorFlow 2023-2024