Dictation Result Accuracy (tracking "Um"and "Ugh" with dictation services) #9989
Replies: 3 comments 5 replies
-
|
@elbuhofantasma, can you follow up on this one? |
Beta Was this translation helpful? Give feedback.
-
|
Just some further info. I know that you can track keywords, but not both keywords and dication at the same time. What I was doing before was displaying what the user was saying on a teleprompter, and dictation was picking up "Um" and "Ugh" in both the Dictation Hypothesis and Result events. I cant using both dictation to track what the user is saying and have keywords tracking specific utterances as you cant run both at the same time. I cant just track "Um" with keywords since I need dictation on to capture everything else. I'm assuming this was a change done to the Dictation service itself as older iterations of the toolkit currently have the same issue for me. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @CLaferriereHL I wanted to give you an update that I am looking into this for you, but it will be likely after 7/6/2021 that I will have an answer for you as some people are out of office until that date. Also thank you very much for the additional information, it is very helpful! |
Beta Was this translation helpful? Give feedback.
-
|
Very much appreciated. I'm using the DictationHandler script, and was using the OnDictationResult method. It used to have "Um" and "Ughh" in the output of it. Another thing to note is that sometimes if I say "Um" quickly as a single syllable it will output as "I'm" longer utterances like "Ummmmmmmm" register as nothing in the On Dictation Result output and as "..." in the OnDictationHypothesis output. I'm assuming that the actual Dictation service is getting better at recognizing words as opposed t o sounds and will sometimes equate the Um sound as I'm. |
Beta Was this translation helpful? Give feedback.
-
|
Quick synopsis as well: The app I was working on was speech/presentation practice application where users could read off their notecard as a run through. They could then seleect specific keywords they wanted to track so they could see how many times they said "Um" or whatever key word or phrase they wanted to track for metric purposes. When they were done, A virtual telepromter could be loaded with their speech so they can practice further looking at a virtual audience. It was pretty cool that it could track problematic things like "Um" and awkward pauses and give real time feedback for it. Unforunetly it seems dication has gotten too good and now no longer will output sounds that arent actually words. I apprecaite the help though. If you guys need any screenshots of my setup or code let me know! Enjoy the Holiday weekend! |
Beta Was this translation helpful? Give feedback.
-
|
Hello @CLaferriereHL Thanks for your updates! I wanted to let you know that there is movement on this issue now that people are back from holiday. We will update this discussion topic when we have an answer for you. |
Beta Was this translation helpful? Give feedback.
-
|
Hey guys, any updates to this? |
Beta Was this translation helpful? Give feedback.
-
|
Hello @CLaferriereHL Sorry for the delay, I was been waiting on a little bit more information before I responded so that I could give you a firm answer. I do have an update and something for you to investigate. Unfortunately the Speech Dictation service has removed the the utterances (such as "uhh"/"umm") from the Speech Recognition results. This is done server side as you had initially suspected. I did get proposed a potential solution. The suggestion I received was the Pronunciation Assessment feature in the Speech SDK. I recommend investigating that service to see if it will suit your needs. Is the Pronunciation Assessment feature something that you have already investigated for your project? |
Beta Was this translation helpful? Give feedback.
-
Hey guys, a while ago I worked on an app with a previous iteration of the toolkit. Before, I was using Dictation to track the amount of times a user would say "Um", "Ughh", or other utterances that are technically not words. Back then (~a year ago) the dication handler would return Um and uhh as a part of the return string. However now I'm noticing that it no longer does this with a new upgrade of the toolkit. Like I said I havnt worked on this project for a year, and recently upgraded my toolkit so I'm wondering where this issue lies. I'm assuming in my absence Microsoft got their dication service to recognize and remove utterances. This is server side right so I wouldnt be able to modify or even get an older version of the toolkit to fix this correct?
Beta Was this translation helpful? Give feedback.
All reactions