Skip to content

Commit

Permalink
Merge pull request #363 from dtischler/main
Browse files Browse the repository at this point in the history
Akida inspection edits 2
  • Loading branch information
dtischler authored Feb 21, 2024
2 parents 217834b + 0cb79f4 commit ce6d597
Showing 1 changed file with 29 additions and 28 deletions.
57 changes: 29 additions & 28 deletions image-projects/brainchip-akida-industrial-inspection.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Created By: Peter Ing
Public Project Links:

Object Detection - [https://studio.edgeimpulse.com/studio/349843](https://studio.edgeimpulse.com/studio/349843)

Classification - [https://studio.edgeimpulse.com/studio/349858](https://studio.edgeimpulse.com/studio/349858)

GitHub Repo: [https://github.com/peteing/brainchip_edgeimpulse_inspectionsystem.git](https://github.com/peteing/brainchip_edgeimpulse_inspectionsystem.git)
Expand All @@ -20,68 +21,68 @@ GitHub Repo: [https://github.com/peteing/brainchip_edgeimpulse_inspectionsystem.

In the ever-evolving landscape of modern manufacturing, the efficiency and accuracy of production lines are paramount. The meticulous inspection of products at various stages ensures not only the adherence to quality standards but also the optimization of resources. In this dynamic scenario, the integration of cutting-edge technologies such as computer vision and artificial intelligence has emerged as a game-changer.

Initially, machine vision systems relied on basic image processing techniques and rule-based algorithms. These early systems were capable of performing relatively simple tasks, such as inspecting products for basic defects or checking for the presence of specific features. These systems required cameras with high-cost Industrial PC's to perform CPU based processing that was expensive and power hungry while offering limited performance.
Initially, machine vision systems relied on basic image processing techniques and rule-based algorithms. These early systems were capable of performing relatively simple tasks, such as inspecting products for basic defects or checking for the presence of specific features. These systems required cameras with high-cost Industrial PC's to perform CPU-based processing that was expensive and power hungry, while offering limited performance.

Today the trend has shifted towards using Deep Learning specifically Convolutional Neural Networks on Graphics Processing Units and specialized CNN hardware accelerators. The solutions on the market are still relatively costly and power hungry. Camera and IPC's are available with integrated acceleration built-in for industrial use cases, but are very expensive.
Today the trend has shifted towards using Deep Learning, specifically Convolutional Neural Networks on Graphics Processing Units and specialized CNN hardware accelerators. However, the solutions on the market are still relatively costly and power hungry. Cameras and IPC's are available with integrated acceleration built-in for industrial use cases, but are very expensive.

Neuromorphic processing, inspired by the human brain, diverges from traditional computing with its parallel, adaptive features like Spiking Neural Networks, parallel processing, event-driven computation, and synaptic plasticity. This disruptive technology holds promise for energy-efficient, brain-like information processing, particularly in tasks like pattern recognition and sensory processing. This makes Neuromorphic computing ideal for use in Industrial Inspection systems where it can provide real-time insights into inspections. The benefits include reduced costs and improved performance and being able to adapt the system at the edge to new use cases.
Neuromorphic processing, inspired by the human brain, diverges from traditional computing with its parallel, adaptive features like Spiking Neural Networks, parallel processing, event-driven computation, and synaptic plasticity. This disruptive technology holds promise for energy-efficient, brain-like information processing, particularly in tasks like pattern recognition and sensory processing. This makes Neuromorphic computing ideal for use in Industrial inspection systems where it can provide real-time insights into product quality. The benefits include reduced costs, improved performance, and being able to adapt the system at the edge to new use-cases.

Brainchip Akida represents the state of the art in production-ready Neuromorphic computing ideally suited to edge use-cases. We will be demonstrating the power of the Brainchip Akida in an industrial setting in this guide as part of a standalone inspection system that can be setup along a production line.
Brainchip Akida represents the state of the art in production-ready Neuromorphic computing, ideally suited to edge use-cases. We will be demonstrating the power of the Brainchip Akida in an industrial setting in this guide as part of a standalone inspection system that can be setup along a production line.

The Akida processor is available on a PCI-E card form-factor for integration into your own hardware, or ships as either an Intel or Arm-based developer kit. For the purpose of this project our focus is on the Arm-based developer kit, which consists of a Raspberry Pi Compute Module 4 mounted on a Raspberry PI Compute Module 4 IO board, which is what we are using for this application.
The Akida processor is available on a PCIe card form-factor for integration into your own hardware, or ships as either an Intel or Arm-based developer kit. For the purpose of this project our focus is on the Arm-based developer kit, which consists of a Raspberry Pi Compute Module 4 mounted on a Raspberry PI Compute Module 4 IO board, enclosed in a metal housing.

Many users coming from an Industrial environment have limited experience when it comes to AI and Deep Learning and this can seem daunting. There are very expensive platforms and solutions that help simplify the process, but none can match the ease of use and rapid performance of using Edge Impulse for the AI component of your project.
Many users coming from an Industrial environment have limited experience when it comes to AI and Deep Learning, which can seem daunting. There are very expensive platforms and solutions that help simplify the process, but none can match the ease of use and rapid performance of using Edge Impulse for the AI component of your project.

## Industrial Inspection Use Case

A typical scenario in an industrial manufacturing plant is defect detection. This can be applied to a range of different product types but essentially the requirement is always to determine which products to reject, out of a set of products that are often in motion using some kind of materials handling equipment such as a conveyor.
A typical scenario in an industrial manufacturing plant is defect detection. This can be applied to a range of different product types, but essentially the requirement is always to determine which products to reject, out of a set of products that are often moving along some kind of materials handling equipment such as a conveyor.

To achieve this, classic machine vision techniques using old camera systems running CPU algorithms often included detecting a Region of Interest (ROI) and then focusing on that area, and using tools such as edge and blob detection to find anomalies.
To achieve this, classic machine vision techniques using old camera systems running CPU algorithms often included detecting a Region of Interest (ROI) and then focusing on that area, while using tools such as edge and blob detection to find anomalies.

![](../.gitbook/assets/brainchip-akida-industrial-inspection/workflow.jpg)

Deep learning solves this approach by making use of learning algorithms to simply teach the system what is correct and what isn't. This results in a 2 stage pipeline that first does Object Detection, then cascades the results to a classifier.
Deep learning solves this approach by making use of learning algorithms to simply teach the system what is correct and what is not correct. This results in a 2-stage pipeline that first does Object Detection, then cascades the results to a Classifier.

![](../.gitbook/assets/brainchip-akida-industrial-inspection/pipeline.jpg)

The Object Detector functions as the Region of Interest segmenter, while the classifier then determines if a product is defective or damaged, or passes the quality check. We will proceed to implement such a pipeline together with a custom GUI based app.
The Object Detector functions as the Region of Interest segmenter, while the Classifier then determines if a product is defective or damaged, or passes the quality check. We will proceed to implement such a pipeline together with a custom GUI based app.

Akida Neuralmorphic technology is unrivaled in terms of power usage at a given performance level. Neuromorphic also provides unique features not found in other technologies such as in device edge learning made possible by the Spiking Neural Network architecture.
Akida Neuralmorphic technology is unrivaled in terms of power usage at a given performance level. Neuromorphic also provides unique features not found in other technologies, such as on-device edge learning made possible by the Spiking Neural Network architecture.

## Setting up the Brainchip Akida Developer Kit

![](../.gitbook/assets/brainchip-akida-industrial-inspection/devkit.jpg)

The Akida Raspberry Pi Kit ships with an SD card with a preloaded console-only version of Ubuntu and a set of web based demos. Preparing your Brainchip Raspbery Pi Kit for use as an inspection system means building up an OS image that has a Linux Desktop Window Manager to enable to host a desktop GUI application.
The Akida Raspberry Pi Kit ships with an SD card containing a preloaded, console-only version of Ubuntu and a set of web-based demos. Preparing the Brainchip Raspberry Pi Developer Kit for use as an inspection system with Edge Impulse means building up an OS image that has a Linux Desktop Window Manager, to enable it to run a desktop and GUI application.

### Step 1: Flash Ubuntu OS image

You need to install the Raspberry Pi Imager V1.8.5 or higher and have a spare SD Card ready. An SD card with a minimum size of 16Gb is required.
You need to install the Raspberry Pi Imager V1.8.5 or higher, and have a spare SD Card ready. An SD card with a minimum size of 16Gb is required.

When selecting the OS, instead of selecting the Raspberry Pi OS, a desktop version of Ubuntu needs to be installed.
When selecting the OS, instead of choosing Raspberry Pi OS, a desktop version of Ubuntu needs to be installed.

![](../.gitbook/assets/brainchip-akida-industrial-inspection/imager.jpg)

First, select the "Choose Device" button, and select the Raspberry Pi 4 option. Then select the "Choose OS" button and select "Other General Purpose OS". Choose "Ubuntu Desktop 22.04.3 LTS (64 bit)". A later version can be used, but for stability in production environments the LTS version is recommended. Note that at the time of reading this version may be a later version. Proceed to select the storage and flash the SD card, which may take some time as the OS downloads.
First, click the "Choose Device" button, and select the Raspberry Pi 4 option. Then click the "Choose OS" button and select "Other General Purpose OS". Choose "Ubuntu Desktop 22.04.3 LTS (64 bit)". A later version can be used, but for stability in production environments the LTS version is recommended. Note that at the time of reading this version may be a later version. Proceed to select the storage and flash the SD card, which may take some time as the OS downloads.

### Step 2: Install Dependances on the Device
### Step 2: Install Dependencies on the Device

Once ready, the SD card needs to be inserted into the Brainchip unit which must be connected to a display via the HDMI port, and the keyboard and mouse attached via USB. Note due to the fact that the Raspberry Pi IO board only exposes 2 USB ports it is recommended to use a wireless keyboard and trackpad combo via a dongle connected to a single port. This frees up the other USB port for use with a Web Camera later. You can also use a USB hub to increase the number of USB ports available.
Once ready, the SD card needs to be inserted into the Brainchip unit, which must be connected to a display via the HDMI port, and the keyboard and mouse attached via USB. Note due to the fact that the Raspberry Pi IO board only exposes 2 USB ports, it is recommended to use a wireless keyboard and trackpad combo via a dongle connected to a single port. This frees up the other USB port for use with a USB Webcam later. You can also use a USB hub to increase the number of USB ports available if needed.

When booting up the Ubuntu OS installation for the first time, you will need to follow the prompts to set the time zone and create an initial user. Once you are able to open a command prompt the following dependencies need to be installed:
When booting up the Ubuntu OS installation for the first time, you will need to follow the prompts to set the time zone and create an initial user. Once you are able to open a command prompt, the following dependencies need to be installed:

- **Python 3.8 to 3.11**: There should already be a suitable version of Python installed by default in Ubuntu 22.04.3 LTS
- **Tensorflow CPU**: Follow the instructions here [https://www.tensorflow.org/install/pip](https://www.tensorflow.org/install/pip) to install TensorFlow but make sure to do a CPU installation
- **Akida Execution Engine**: This component of the MetaTF framework provides access to the Akida hardware or a software simulator if you don't have hardware present. It also provides a Keras-like API via Python for programmatic access to the Akida Hardware.
- **Akida PCIe Driver**: Lastly the Akida PCIe driver is required to ensure that the Akida Execution Engine can directly access the Akida co-processor.
- **Python 3.8 to 3.11**: There should already be a suitable version of Python installed by default in Ubuntu 22.04.3 LTS.
- **Tensorflow CPU**: Follow the instructions here [https://www.tensorflow.org/install/pip](https://www.tensorflow.org/install/pip) to install TensorFlow, but make sure to do a CPU installation.
- **Akida Execution Engine**: This component of the MetaTF framework provides access to the Akida hardware, or a software simulator if you don't have hardware present. It also provides a Keras-like API via Python for programmatic access to the Akida Hardware.
- **Akida PCIe Driver**: Lastly the Akida PCIe driver is required, to ensure that the Akida Execution Engine can directly access the Akida co-processor.

To get started with the process first upgrade `pip` from the command line on the device (you can do this via SSH as well)
To get started with the process, first upgrade `pip` from the command line on the device (you can do this via SSH as well):

```
pip install --upgrade pip
```

Then install TensorFlow, but for CPU-only as we won't be using a GPU in this case. We will use the Akida Execution Engine with the MetaTF Keras-like API later to access the Akida Hardware
Then install TensorFlow, but for CPU-only as we won't be using a GPU in this case. We will use the Akida Execution Engine with the MetaTF Keras-like API later to access the Akida Hardware:

```
pip install tensorflow
Expand All @@ -94,13 +95,13 @@ python3 -c "import tensorflow as tf;
print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
```

Once Tensorflow is running the next step is to install the Akida Execution engine and this is done via `pip` as well, as its installs a Python package:
Once Tensorflow is running, the next step is to install the Akida Execution engine. Tis done via `pip` as well, as it is offered as a Python package:

```
pip install akida
```

After running this command successfully the MetaTF API can be used from within Python, however since no hardware driver is installed when attempting to execute inference later on it will resort to simulated Akida in the backend. This is helpful for testing models for compatibility and not necessarily performance, and can be useful for creating a portable development environment. To make use of the actual hardware the next step is to install the Akida PCIe card and the software.
After running this command successfully the MetaTF API can be used from within Python, however since no hardware driver is installed, when attempting to execute inference later on it will resort to a simulated Akida in the backend. This is helpful for testing models for compatibility, and can be useful for creating a portable development environment. To make use of the actual hardware the next step is to install the Akida PCIe card and the driver.

First ensure the unit is powered down and the Akida PCIe card is correctly seated inside the Raspberry Pi IO Board's PCIe slot, then power up again. You can verify the card is correctly plugged in using the following command:

Expand All @@ -112,11 +113,11 @@ If the card is working and detected you should see something similar to below wh

![](../.gitbook/assets/brainchip-akida-industrial-inspection/coprocessor.jpg)

The Akida driver needs to be downloaded from the Brainchip GitHub repository from the command line, and the `build-essentials` and `linux-headers` packages need to be installed:
The `build-essentials` and `linux-headers` packages need to be installed on the system first, and then the Akida driver can be downloaded from the Brainchip GitHub repository via the command line:

```
git clone https://github.com/Brainchip-Inc/akida_dw_edma
sudo apt install build-essential linux-headers-$(uname -r)
git clone https://github.com/Brainchip-Inc/akida_dw_edma
```

To install the driver and make it available on each boot and fully accessible to all users, first change directory into the repo and then run the install script as the superuser, this is achieved with running the following commands:
Expand Down

0 comments on commit ce6d597

Please sign in to comment.