Issue#13 transcribe

[epg323]

closes issue#13

[MrNanosh]

JS Promises should be spelled with one m. async followed by a comma seems to be a syntax error. ko and ok are parameters but are vague.

[epg323]

Is it possible you are looking at an older commit, this file was deleted 2 weeks ago?

[MrNanosh]

received is misspelled.

[epg323]

see above

[ddfridley]

(ok,ko) came from me and I learned it from someone else.  I like it because it's way shorter than resolve and reject, and they are the reverse of each other, and If someone throws a punch (error) you get knocked out (ko'd). I have code spell checker installed in VSC to help with spelling -but it has it's disadvantages too, like it doesn't recognize variable name.   It would give you a little squiggly under Prommise and under receivec

…

On 6/15/2020 11:53 AM, MrNanosh wrote: ***@***.**** commented on this pull request. ------------------------------------------------------------------------ In app/models/transcribe.js <#209 (comment)>: > + +const Joi = require('joi') +const MongoModels = require('mongo-models') + +const schema = Joi.object({ + _id: Joi.object(), + path: Joi.string(), + subject: Joi.string().required(), + description: Joi.string().required(), + component: Joi.object(), + userId: Joi.string(), +}) + +class Transcribe extends MongoModels { + static create(obj) { + return new Prommise(async, (ok, ko) => { JS Promises should be spelled with one m. async followed by a comma seems to be a syntax error. ko and ok are parameters but are vague. ------------------------------------------------------------------------ In app/models/transcribe.js <#209 (comment)>: > + path: Joi.string(), + subject: Joi.string().required(), + description: Joi.string().required(), + component: Joi.object(), + userId: Joi.string(), +}) + +class Transcribe extends MongoModels { + static create(obj) { + return new Prommise(async, (ok, ko) => { + try { + const doc = new Transcribe(obj) + const result = await this.insertOne(doc) + if (result && result.length === 1) ok(result[0]) + else { + const msg = ` unexpected number of results receivec ${results.length}` received is misspelled. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#209 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZJ537F5KLBBQLO5MMGKBDRWZUZJANCNFSM4N3AA4ZA>.

[ddfridley]

the metaTags here are for sharing information to facebook about this page. The transcription information should be put into webComponent.participant[the participant]. It's going to be a little challenging though to figure out which .participant is for which transcription - but I think you can use the userId field to make the association.

[ddfridley]

I admit the description of the structure of the parentIota here is very weak. The best description is in: https://github.com/EnCiv/undebate/blob/master/app/components/data-components/merge-participants.js

If you go down to the part where it says "what we are trying to create"
Also - I see that, at least as documented, the userId is not there. We may need to find some other way to associate the transcription to the participant - or we may just need have the mergeParticipants component add the userId.

[epg323]

The only thing I can come up with is we can match by url , example: "https://res.cloudinary.com/hf6mryjpf/video/upload/v1566510654/5d5b73c01e3b194174cd9b92-1-speaking.webm"

I do see this being highly inefficient. I think the best method will be as you suggested , a userid.

[ddfridley]

I have been thinking about Hartford and how to have a second and third round of undebates - meaning first round is BP's questions, and second round is hartfords questions. I think that userId will not be sufficient. I am thinking that when participants are merged - mergeParticipant into parent needs to get the participant Iota's _id. Then when merging transcription you can look for the corresponding _id to the one you transcribed. This also solves the problem that a participants re-records their answers - which creates a new partcipant records, - but there isn't a translation for it yet. If you update the last forEach loop of merge-participants-into-parent.js you'll get both participantId, and userId.

  limitedLatestParticipants.forEach(participantDoc => {
    parentIota.webComponent.participants[audience + nextIndex++] = { participantId: participantDoc._id.toString(), userId: participantDoc.userId, ...participantDoc.component.participant}
  })

this code is not tested though.

note that a weirdism of Mongo is that _id is an object. In this project I have made up a convention that that is the only place where it's an object. So parentId, and userId, and now participantId are all strings. I've just had too much trouble with inconsistency about this in past projects. If you know of other conventions or anything - I'd be excited to talk about it.

[epg323]

Would the path field(""path": "/schoolboard-demo"") be a better way to match?

[ddfridley]

If there are multiple candidates in an undebate - there will be multiple transcription records - one for each user. This is different than smpreview where there was only one preview for the whole page. I think you are going to have to do something like:

let transcribes=childIotas.reduce((transcribes,iota)=>iota.component && iota.component.component === 'Transcription' ? (transcribes.push(iota),transcribes) : transcribes)

transcribes will be a list of the iota's that are transcriptions.

[ddfridley]

Here is a challenge - I think that after you do https.get resp.body will be the wav file data and all you need to do is audioString = resp.body.toString('base64'). But I'm not sure and their may be another layer of structure with resp.body that you need to drill into. But lets try to save the time of writing to a file and a reading it back out. There may be format issues to resolve but I bet we can figure it out buy looking into what comes back in resp.body and maybe comparing that to what we see in audiostring if you run it now.

[ddfridley]

Danger! you should not check in private_key and things into github. I see you added it to .gitignore but that was probably after there was a git add .

You will need to move the file out of the directory, git another git add ., and then a git commit -m "removed enciv-transcribe.json" in order to get rid of it.

I'm not sure how this code getting the keys into the api call - are they set in env variables somewhere? We'll need to document it somehow so people can set it up. (like me).

Also - you are going to need to get new keys - these are public.

[epg323]

I was thinking of adding installing instructions into the readme after we do the merge to master. To get a key you need to put your credit card info, i hope this isnt a barrier for new developers.

[ddfridley]

I suggest that variable names of lists/arrays end in s like audioBytes

[MrNanosh]

since main is called within the closure of a function which is tied to a variable called request it is better to not use a const name called request here. It is confusing but also might cause errors.

[MrNanosh]

Private keys should go in some sort of env file that is untracked. This should be scrubbed from the git history.

[ddfridley]

in your .bashrc file create:

export GOOGLE_TRANSCRIPTION_PROJECT_ID="..."
export GOOGLE_TRANSCRIPTION_KEY_ID="..."
export GOOGLE_TRANSCRIPTION_KEY="..."

then you can run:

source .bashrc
heroku config:set GOOGLE_TRANSCRIPTION_PROJECT_ID=$GOOGLE_TRANSCRIPTION_PROJECT_ID
heroku config:set GOOGLE_TRANSCRIPTION_KEY_ID=$GOOGLE_TRANSCRIPTION_KEY_ID
heroku config:set GOOGLE_TRANSCRIPTION_KEY=$GOOGLE_TRANSCRIPTION_KEY

If you have troubles - on heroku I had to go in through the web user interface and go to my app, and go to settings, and Reveal Config Vars - and then edited the GOOGLE_TRANSCIPTION_KEY add a newline at the end. (meaning go to the end of the line and hit return so it looks like two lines). I have this same sort of key configuration for gmail from the server and I had to figure that out.

[epg323]

Cloudinary process: We use the cloudinary api to upload the video file to cloudinary and google speech to text api. We use the video url and replace the .mp4 extension with .transcript . Once we do that we can call the url and extract the contents of the .transcript file.

[epg323]

Just reached out to cloudinary, this is what they said: "The transcription gets queued in an async process, so you'll need to wait for that process to finish. I did notice that the documentation doesn't include those details, so I will have them add it but first let me test it."

I will try to get an estimate of the wait time, if the wait time is too long we can just do google transcribe streaming

[ddfridley]

In https://cloudinary.com/documentation/google_ai_video_transcription_addon#:~:text=With%20the%20Google%20AI%20Video,best%20possible%20speech%20recognition%20results.it says: The google_speech parameter value activates a call to Google's Cloud Speech API, which is performed _asynchronously after your original method call is completed_. Thus your original method call response displays a pending status: ... "info": {    "raw_convert": {       "google_speech": {         "status": "pending"       }     }  } ... When the google_speech request is complete (may take several seconds or minutes depending on the length of the video), a new raw file is created in your account with the same public ID as your video or audio file and with the .transcript file extension. If you also provided a notification_url in your method call, the specified URL then receives a notification when the process completes: Here is the documentation on notifications: https://cloudinary.com/documentation/notifications We are going to have to create athis.app.post(...) handler in server.js.  But we should keep putting feature specific code in there - but lets get it working first and then clean it up. I've got time to talk this through on Friday if anyone's available.

…

On 7/16/2020 7:53 PM, epg323 wrote: Just reached out to cloudinary, this is what they said: "The transcription gets queued in an async process, so you'll need to wait for that process to finish. I did notice that the documentation doesn't include those details, so I will have them add it but first let me test it." I will try to get an estimate of the wait time, if the wait time is too long we can just do google transcribe streaming — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#209 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZJ536VOBU6IA6GIRU5KULR364LFANCNFSM4N3AA4ZA>.