-
Notifications
You must be signed in to change notification settings - Fork 0
AI box app
here we use the word "AI box" for the module with a camera that Segway provides. It's the same device as what may be called "camera module" or "Segway Pilot".
the software necessary is available on either Linux, Windows or macOS. It's up to you to use any of them.
Install scrcpy from the Github repo. It provides a screen mirroring feature to control the device as if it was an Android phone.
We're going to use adb to connect to the AI box (which is an Android device) and install the app. If you haven't installed the android platform tools when installing scrcpy above, download the Android platform tools from this page. Here is the documentation that's specific to adb for reference. If you already use Android Studio and likely have the Android platform tools installed, look at these suggestions.
Plug a USB cable into the AI box device (it's supposed to be a USB-C) and plug it into your computer on the other end. Open a new terminal window to make sure the PATH is updated, then run scrcpy :
scrcpy
It should open a new window with what the device displays.
If it doesn't, you can check if adb (on which scrcpy relies to connect to the device) detects Android devices:
adb devices
Which will probably return something like
List of devices attached
RFCNA0RZA0A device
It should show a list of at least one device. If none shows up, either the device is not powered or the USB connection is failing.
If the scrcpy started properly, you should see a window open up and display an Android screen. In the most likely case, you will see an app currently running, probably in full screen mode. If you use your cursor to swipe up from the bottom edge or to swipe left from the right edge, you might see the controls to go to the home screen. There is also a shortcut key, [TODO Vincent] find it.
If you managed to get to the home screen, you'll notice that the app that took the full screen is popping back after a few seconds. The only way to get rid of it seems to run the following command with adb, based on the device you got above:
adb -s MTS29M1002UK shell am force-stop com.segway.robot.scooter
Replace MTS29M1002UK with the device name you got when you ran adb devices. This command force stops the constantly running app on that specific device.
Use the apk file transferred on Slack and install it using adb:
adb -s MTS29M1002UK install /path/to/flamingo-aibox-v1.apk
Where you replace the sample device name MTS29M1002UK with the correct one, and you use the correct path to the apk file.
There might be some security message pop up. We know we want that Flamingo AI Box app installed, you can "Install anyway".
Then you can start the app with adb (or you can navigate the Android device with the scrcpy interface. With adb, the command is:
adb -s MTS29M1002UK shell am start -n com.flamingoscooters.android.aibox/com.flamingoscooters.android.aibox.AiBoxActivity
You should now see the AI box app open, called "Flamingo AI Box" and displaying a bunch of buttons to control it. It's asking for some permissions. Accept the permission to use files.
There are a few steps to take before you can control the app remotely via Bluetooth. Mainly connecting to WiFi, starting the camera and pairing your Android phone.
Via scrcpy, in the "Flamingo AI Box app", tap the button to "open the camera". You should see a permission request for the camera access, accept it "While using the app".
Go through the phone settings and connect to the warehouse WiFi network, with the option to auto-connect.
Use any version of the Android admin app equal or above AA3.0.62.
Enable the Bluetooth on both devices (Flamingo AI box app on AI box / Admin app on the Android device), via the button in the apps. On the AI Box app, it's on the main screen of the app (the only screen anyway). On the Android admin app, you have to open the left panel, and pick the "AI Box control" item in the list.
On the AI box app, a pop up message will suggest to enabled Bluetooth, then to make the device discoverable for 1200 seconds (20 minutes) so you have plenty of time to pair the devices.
On the Android admin app, after you tap on "Enable BT", it should show a bottom sheet with a list of devices. Depending on how many other phones and devices are around (probably plenty in the warehouse), you will see a long list of devices without a name. I have no idea what name the AI box device will have so you might have to try and guess. On the device Bluetooth settings, it might stated (it is the case on my phone).
Once you manage to pair the phone and the AI box, it should display on both devices that it's connected to each other. At that point, you can start playing with the Android admin app.
You can try a few actions. Tap "Start recording". If the camera is not yet started on the AI box, it should now start. You should see on the AI box that it will display "Saved image" continuously. At that moment, it is at least stored on the device, and if connected to the warehouse WiFi, it will also upload it. You can change the recording rate if you want, by default I think it's set to about 3 images recorded per second.
You can also try to upload the images, even though we won't really know the outcome right away (as the actual upload results are not displayed on the admin app).
I would like you to try tapping the buttons for "Vehicle speed" and "Vehicle location" and see what happens. If it returns false on the admin app (and it would display "Service unbind" on the AI box screen), then it would mean that the AI box cannot connect to the rest of the IoT. If it does return a speed (or location), then that means we can have a connection to the rest of the IoT.
When you will need to connect to the AI box while out of the warehouse, you won't need to follow this whole process again. You should only need to do the part where you connect the Admin Android app to the AI box app with bluetooth. From the admin app, you open the "AI box control" screen from the left panel. You tap "Enable BT" and it should list again all the nearby bluetooth devices
TODO Vincent: keep writing this
Once connected to the warehouse WiFi, the app should upload automatically the any recorded footage to a storage server.
Technically, take the same solution as the existing Android Admin app that uploads data to Firebase storage. It should then be easy to clean the data and use it to train some object detection model.
Report and show in some dashboard what the app does on each AI box deployed. It could include all these elements: results of the stream analysis (number of frames analysed, number of times a footpath has been detected ...), whether the app is online and has been that day, what it shared to the IoT, the version of the app, what network it is connected to ...
Technical solution could be Firebase Analytics for most of the data listed above with a Google Looker Studio dashboard. Or it could be synchronised with the current prod database (what about the authentication?)