-
Notifications
You must be signed in to change notification settings - Fork 0
Technical architecture
TODO
TODO
TODO
The BerryNetProvider class definition wraps the following open source library: https://github.com/DT42/BerryNet. BerryNet is a very useful component that leverages Tiny-YOLO to do frame by frame object recognition across 80 classes. Our provider allows us to request a snapshot, retrieve the detection results in json format and parse results. The product worked really well and we showcase it in our 4/18 technical checkin. The issue we ran into was training this model to add detection of tram cars. This proved to be difficult and overly complex because we only needed to track the tram and nothing else, and our camera is in a fixed position. Tiny-YOLO is really useful for scenarios where the user would like to classify many different items with a moving camera. In addition to this, due to the pi's limited computational capabilities we could only process around one image every three seconds. This was a little too slow to accurately detect the tram as it departs or lands under the 59th street bridge.
The BerryNetServer main method instantiates an instance of our BerryNetProvider and leverages a loop to analyze snapshots from the USB camera every three seconds. As we create snapshots the images are stored in the pi in the current directory and parsed detections are output to the console. The product worked really well and we showcase it in our 4/18 technical checkin.
TODO
TODO
TODO
TODO
The ImageQueue class definition wraps a standard FIFO queue to pipeline captured image snapshots. The images given to the queue are already represented as Numpy matrices for processing. Currently our final product only queues a single image at a time, but we had tested various queue depths to see what level of separation in our image comparison would give the most accurate and quickest detections.
The LightServer main method leverages Flask to enable an endpoint for controlling the light display. The light display is a laser printed representation of the tram towers and a strip of 60 LEDs representing the current trajectory of a docked or approaching tram. The TramServer will periodically push updates to the LightServer and the LightServer connects over serial to issue the respective updates to the Arduino micro-controller which is controlling the LED strip. We designed it this way so that the light strip display could be connected to a machine running the LightServer and be placed anywhere for people to see and interact with. The LED strip is folded in half on the display and each set of 30 LEDs are used to show the same trajectory to users on either side of the display.