Vision camera v3 (#1121)

* Bump vision-camera

* Refactor patch

* Move patched orientation into patch functions file

* Update react-native-vision-camera+3.4.1.patch

* Switch to MacOS 13 runner

Vision camera requires XCode 15 to compile.

* Add step to specify XCode 15

* Higher level of logging

* Increase test timeout

* Add comment

* Remove navigation to obs without evidence for signed out user

* Patch for location permission not working on iOS

* Increase setup timeout

* Increase some more timeouts

* Revert back to less logging in CI

* Does it have to do with timeouts?

* Trace log level

* Update README.md

* Disable Homebrew’s auto update and install cleanup

* Setup ruby step

* Install pods only if not cached

* Revert "Install pods only if not cached"

This reverts commit 42a2ea0.

* Run simulator in headless mode, record all logs

* Increase timeouts again

* Revert "Remove navigation to obs without evidence for signed out user"

This reverts commit 2b4718f.

* Add boolean to run use effect only once

* Did merge wrong code

* There is one more permission gate when entering obs edit now

* Add permission gate dismissal to signed out user test

* Add comment, rename state

* Lower action timeout

* Update react-native-vision-camera to 3.6, update Reanimated to v3 (#838)

* Bump camera and plugin

* Upgrade Reanimated and libs using it

* Update vision-camera mock

* Remove superfluous patch

* Fix type

* Add mocks for e2e tests because bottom sheet does not work on AOSP

* Update package-lock.json

* Update vision camera

* Bump vision camera

* Update vision camera patch version

* Remove superfluous patch

* Update vision camera

* Update vision-camera and plugin

* Use latest vision plugin = rebased v3

* Run npm clean-start

* No longer needed

* Duplicate prop

* Switch back to v3 code

* Upgrading Detox fixes issue with iOS tests failing

Because of previous lack of permissions.

* Update to latest detox version

* Npm i force

* Vision 3 e2e testing, (#1126)

* Remove jest detox config

* Revert e2e timeout increase

* Revert jest config timeout

* Use macos 14 runner which has XCode 15 per default

* Use latest bottom sheet

* Revert "Use latest bottom sheet"

This reverts commit c66cd09.

* Adding comma back in

* Remove spaces

* Use latest vision-camera and plugin

* Use release version of vision-camera

* Remove force flag

* Update react-native-worklets-core

* Upgrade reanimated

* Update babel.config to allow nested worklets

* Run frame processor async

* Remove enableGpuBuffers flag

* Remove no longer needed comment

* Update comments about version

* Update vision-camera and plugin

* Code format

* Rename param

* Use latest-plugin

* Remove fps param from camera view

* Change confidenceThreshold to number

* API change for results structure

* Refactor fps to be checked inside hook

As of vision-camera 3.9.1 our camera feed is stuttering when calling the runAtTargetFps function inside the useFrameProcessor hook.
So, what I did here is to track the time and skip the frames that are received before the target fps is reached.

* Remove updates of non-camera related

* Update comment

* Code format

* Snapshot updates

* Revert changes to ios e2e

* Add a log for the average time it takes a frame to be processed

* Use release version of plugin

* Add a patch for runAsync to work in release builds

* Update react-native-worklets-core to 0.4.0

* Use latest plugin version

* Add a shift method to worklet arrays

* Use latest vision-plugin

This makes use of .shift() in worklet array and depends on the previous patch.

* Set result timestamp and show age of result in debug mode

* Fix an error with timestamp being undefined

* Remove log

* Use latest vision plugin

* Change result timestamp to pink

* Comment

---------

Co-authored-by: Ken-ichi Ueda <[email protected]>

.github/workflows/e2e_ios.yml

@@ -51,6 +51,11 @@ jobs:
       with:
         node-version: 18
 
+    - name: Setup Ruby version according to .ruby-version with cached gems
+      uses: ruby/setup-ruby@v1
+      with:
+        bundler-cache: true        
+
     - name: Cache node modules
       uses: actions/cache@v3
       id: cache
@@ -98,8 +103,13 @@ jobs:
       run: echo "MOCK_MODE=e2e" >> "$GITHUB_ENV"
 
     # Install prerequisites for detox and build app, and test
-    - run: brew tap wix/brew
-    - run: brew install applesimutils
+    - name: Install macOS dependencies
+      env:
+        HOMEBREW_NO_AUTO_UPDATE: 1
+        HOMEBREW_NO_INSTALL_CLEANUP: 1
+      run: |
+        brew tap wix/brew
+        brew install applesimutils
 
     - name: Build test app
       run: npm run e2e:build:ios

README.md

@@ -9,7 +9,7 @@ See [CONTRIBUTING](CONTRIBUTING.md) for guidelines on contributing to this proje
 
 ### Requirements
 
-* Xcode 13 or above
+* Xcode 15 or above
 * [Android and iOS environment setup](https://reactnative.dev/docs/environment-setup) described in the RN docs
 
 ### Install packages and pods

babel.config.js

@@ -22,14 +22,11 @@ module.exports = {
         tests: "./tests"
       }
     }],
-    // Reanimated 2 plugin has to be listed last https://docs.swmansion.com/react-native-reanimated/docs/fundamentals/installation/
-    // react-native-vision-camera v3
-    // "react-native-reanimated/plugin",
-    // react-native-vision-camera v2
+    // Reanimated plugin has to be listed last https://docs.swmansion.com/react-native-reanimated/docs/fundamentals/installation/
     [
       "react-native-reanimated/plugin",
       {
-        globals: ["__inatVision"]
+        processNestedWorklets: true
       }
     ]
   ],

e2e/signedIn.e2e.js

@@ -30,7 +30,7 @@ describe( "Signed in user", () => {
       await expect( uploadNowButton ).toBeVisible();
       await uploadNowButton.tap();
     } else {
-      // Press Upload now button
+      // Press Save button
       const saveButton = element( by.id( "ObsEdit.saveButton" ) );
       await expect( saveButton ).toBeVisible();
       await saveButton.tap();

ios/Podfile.lock

@@ -317,7 +317,7 @@ PODS:
     - React-Core
   - react-native-webview (11.23.1):
     - React-Core
-  - react-native-worklets-core (0.2.0):
+  - react-native-worklets-core (0.4.0):
     - React
     - React-callinvoker
     - React-Core
@@ -435,33 +435,10 @@ PODS:
     - React-Core
   - RNPermissions (3.10.0):
     - React-Core
-  - RNReanimated (2.17.0):
-    - DoubleConversion
-    - FBLazyVector
-    - FBReactNativeSpec
-    - glog
-    - RCT-Folly
-    - RCTRequired
-    - RCTTypeSafety
-    - React-callinvoker
+  - RNReanimated (3.7.1):
+    - RCT-Folly (= 2021.07.22.00)
     - React-Core
-    - React-Core/DevSupport
-    - React-Core/RCTWebSocket
-    - React-CoreModules
-    - React-cxxreact
-    - React-jsi
-    - React-jsiexecutor
-    - React-jsinspector
-    - React-RCTActionSheet
-    - React-RCTAnimation
-    - React-RCTBlob
-    - React-RCTImage
-    - React-RCTLinking
-    - React-RCTNetwork
-    - React-RCTSettings
-    - React-RCTText
     - ReactCommon/turbomodule/core
-    - Yoga
   - RNScreens (3.21.1):
     - React-Core
     - React-RCTImage
@@ -479,11 +456,12 @@ PODS:
   - SDWebImageWebPCoder (0.8.5):
     - libwebp (~> 1.0)
     - SDWebImage/Core (~> 5.10)
-  - VisionCamera (2.15.6):
+  - VisionCamera (3.9.1):
     - React
     - React-callinvoker
     - React-Core
-  - VisionCameraPluginInatVision (2.2.0):
+    - react-native-worklets-core
+  - VisionCameraPluginInatVision (3.0.0):
     - React-Core
   - Yoga (1.14.0)
 
@@ -768,7 +746,7 @@ SPEC CHECKSUMS:
   react-native-sensitive-info: d44e909d065f9c0e15734245e5dd6a24b82e3dcd
   react-native-slider: cc89964e1432fa31aa9db7a0fa9b21e26b5d5152
   react-native-webview: d33e2db8925d090871ffeb232dfa50cb3a727581
-  react-native-worklets-core: 7ad416a8965086b98b07964f7f6932560a54a14c
+  react-native-worklets-core: 2efe80a3ee87fe5e6fefa814e0e20c2708d3ad25
   React-perflogger: c944b06edad34f5ecded0f29a6e66290a005d365
   React-RCTActionSheet: fa467f37777dacba2c72da4be7ae065da4482d7d
   React-RCTAnimation: 0591ee5f9e3d8c864a0937edea2165fe968e7099
@@ -796,16 +774,16 @@ SPEC CHECKSUMS:
   RNGestureHandler: 6e4dc6b7ab3a385386d4e36228bd065e5a611394
   RNLocalize: d4b8af4e442d4bcca54e68fc687a2129b4d71a81
   RNPermissions: 0332875c444efe864dd97071dc848529bd7cc692
-  RNReanimated: f186e85d9f28c9383d05ca39e11dd194f59093ec
+  RNReanimated: e626b8c31f56bf320fd27ee5ca0d5349792f2a8d
   RNScreens: d3675ab2878704de70c9dae57fa5d024802404cc
   RNShareMenu: cb9dac548c8bf147d06f0bf07296ad51ea9f5fc3
   RNStoreReview: 31dbfd0dac2eea9675f0b84f1dd3261c2110c337
   RNSVG: d00c8f91c3cbf6d476451313a18f04d220d4f396
   RNVectorIcons: fcc2f6cb32f5735b586e66d14103a74ce6ad61f8
   SDWebImage: a7f831e1a65eb5e285e3fb046a23fcfbf08e696d
   SDWebImageWebPCoder: 908b83b6adda48effe7667cd2b7f78c897e5111d
-  VisionCamera: 523b49054bee9dace64189ab6631cb41e8b83fe0
-  VisionCameraPluginInatVision: 7e09a4ca0b34dd81afd4b68aa26a27eff5bb8fd4
+  VisionCamera: 609b194489f336792caa5eda305a3fdd4ba5d44c
+  VisionCameraPluginInatVision: edd58cf80291675d1a1523a3d8d3b2c2f1bff26a
   Yoga: e29645ec5a66fb00934fad85338742d1c247d4cb
 
 PODFILE CHECKSUM: 77ed9526d4011b245ce5afa1ea331dea4c67d753