Project Planning - Proof of Concept

[victormaricato]

• Specify projects scope and tasks

• Core High-level Functions

  • Start Transcribing Robot
  • Start Filter and break-down Transcribing robot output
  • Start Natural Language Understanding Robot
  • Start Image Gathering Robot
  • Start Video Robot
  • Start to Export Video

• Transcribing Robot

  • Ask input file (.raw format)
    • Get select filepath
    • Check if file exists
    • Data structure (Key-Value)
  • Make Google Speech-To-Text API return a transcript of a simple audio
    • Make it returns with Timestamp
    • Get the timestamps and phrases data structure and logic working
  • Make Google Speech-To-Text API return transcript of a complex audio
  • Break-down sentences in Speech-To-Text output.
    • Define an approach for splitting sentences. (Check if Google NLU can do this)
  • Save output (sentences) on data structure

• NLU Robot

  • Load data structure
  • Send sentences programatically for Google NLU API.
  • Retrieve keywords and entities output from API.
  • Save on data structure.

• Image Gathering Robot

  • Get Images from Keywords using Google Image API
    • Check Google Image API parameters for getting only Copyright-free images
    • Images and Sentences should be ordered according to transcription order of appearance.
  • Use ImageMagick for resizing and preparing downloaded images.

• Video robot

  • Images should be added according to order.
  • Define how to match transcription/audio keyword with image
    • Possible approach is to check Word2Vec of keywords and checking if they're semantically different from each other
    • Another possible approach is to define images timeouts. Or define Word "Countouts" (for each 10 words, change the image to another that corresponds to the keyword of that snippet).
  • Render video
  • Save/Export video as .mp4