Task: Support pose estimation on video

[brukew]

Description

This would support pose estimation on video inputs (via file or Video object). This can currently be done by looping through the video frames and running pose estimation on it, but a built-in method should be implemented for simplicity.

I imagine users may use pose estimation primarily on Video rather than individual images so this is needed.

Tasks

• Figure out the exact implementation (Representation + Access)
• Implement
• Test
• Revise docs and tutorial

Freeform Notes

Thinking about how this could be implemented

• Representation
  • VideoPose object
  • List of ImagePose objects for each frame
• Access Points
  • instance method in Video
  • Video input into API

[fabiocat93]

Hey @brukew , some additional considerations:

• Exploit temporal info for smoother pose estimation:
  -- The pose of a person in frame X can provide useful context for frame X+1.
  -- Consider leveraging past frame data to improve tracking accuracy, reduce jitter, and interpolate missing data.
  -- As a possible technique, I know Kalman filtering and interpolation. There may be something smarter, though.
• Track individuals across frames using spatial information (e.g., useful for multi-person tracking scenarios).
  -- Consider incorporating facial recognition or spatial+tmporal info for better identity mapping.
• Implement a low-resource version of the algorithm, maybe allowing skipping some frames (e.g., you analyze 1 frame every N)