Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Face Detection MediaStreamTrack constraints and VideoFrame metadata #706

Open
ttoivone opened this issue Nov 15, 2022 · 2 comments
Open

Comments

@ttoivone
Copy link

Request for Mozilla Position on an Emerging Web Specification

Other information

Updated PR based on feedback and the new WebCodecs VideoFrameMetadata dictionary is here:
mediacapture-extensions #78.

@zcorpan
Copy link
Member

zcorpan commented Apr 12, 2023

cc @jan-ivar

@jan-ivar
Copy link
Member

jan-ivar commented May 15, 2024

Sorry for the late response. I don't have a strong opinion here.

This is not facial recognition, rather it detects where faces are (eyes, mouth). Its purpose would be optimizing quality and power consumption of video effects like background replacement and other augmentation popular in video conferencing.

This seems somewhat useful in providing parity with non-web applications in providing hardware assistance (from drivers/OS etc) with detecting outlines of heads in camera video, where available.

Privacy-wise the segment information does not appear to reveal anything about a videoframe that couldn't be deduced through processing by the application itself. It might reveal specifics about a particular camera driver or OS, but this seems minor compared to other information exposed by an active camera and its device label.

It would only benefit applications that are aware of and utilize this API. I don't have any info on how prevalent hardware is, nor existing usage in other browsers. Absent that, a criticism might be why this functionality is tied to a camera source and that time might be better spent on a broader API that can process web-provided frames as well, like the one mentioned in the explainer. But if that's where the technology exists, that's where it is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Unscreened
Development

No branches or pull requests

3 participants