Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added facelandmark orientation #160

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

rs-mnl
Copy link

@rs-mnl rs-mnl commented Dec 16, 2024

Hello! This is a nice project you have and I would love to contribute...

I noticed the docs for face landmarking have not quite as robust of an interface as the pose detection and wanted to try and bring them to parity.

When I tried out the example face landmarking project it crashed a little bit for me and so I tried to improve the performance using similar patterns as the pose detection example, such as using reanimated 3. You might not want this, or you might want it in a different way so I'm happy to adjust the contribution based on your priorities. Right now, if you download this branch it will have an updated example project for iOS that improves performance of the drawn facemesh and exposes some more features like FPS throttling and device/sensor orientation/forcing as in the pose detection module.

Please be aware, there is no android code yet, only iOS. I tried getting the examples working on android emulators and get stuck on this runtime error for all example projects at the moment:

 LOG  Worklets loaded successfully
 ERROR  Error: Exception in HostObject::get for prop 'FaceLandmarkDetection': java.lang.UnsatisfiedLinkError: dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found, js engine: hermes

It is my first time creating a react-native module for android so it is probably for some stupid fault of mine but I will keep learning...

Anyway, the purpose of opening this incomplete merge request is to ask if you would be open to a contribution in this direction and if so do you have any advice or comments based on these two commits? I saw the recent release so maybe you're already doing something along these lines or the project has other significant developments.

Thanks again for creating this, hope to be able to help.

@cdiddy77
Copy link
Owner

Wow this is awesome, thank you. I will take a look in the next few days.

@cdiddy77
Copy link
Owner

FYI I have never gotten android working under emulation, nor either iOS.

@rs-mnl
Copy link
Author

rs-mnl commented Dec 30, 2024

I got an android device and am now able to build/run the app on both platforms!

Hopefully in the next few days I will be able to extend the native android module similar to the iOS changes, and then update the docsite...

@rs-mnl rs-mnl force-pushed the added_facelandmark_orientation branch from ff0d14f to c7a5853 Compare January 9, 2025 16:10
@rs-mnl
Copy link
Author

rs-mnl commented Jan 9, 2025

Hi!

Based on my testing with a Pixel 4a, the Android code works now and the facelandmarking module now has feature parity across both platforms and the docusaurus page for the Face Landmarking module reflects the additional orientation interfaces.

Please let me know how I might be able to help with the testing process or if you have any other thoughts regarding this example project, such as whether it's worthwhile to also bump the react-native-vision camera versions from 4.0.3 to 4.5.3 (as in pose detection), or anything else you may be considering.

@rs-mnl rs-mnl marked this pull request as ready for review January 9, 2025 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants