The objective is to create an end-to-end event-driven AWS project, simulating an animal and species monitoring platform of an environmental preservation park, all with open source tools and the AWS public cloud.
Some tools and frameworks were used to deploy this stack, including:
- Node Red: Creation of the farm or where the sensors are there
- Serverless: Lambda functions
- Terraform: Setup of AWS services and products
- Docker: For containers and microservices
- Slack: In order to receive notifications
The stack itself is intended to be all code-oriented, IaC, so you won't need to create anything in the AWS console, everything is done on command line.
Let's go quickly, here are the prerequisites to start the setup and deployment:
What you will need already configure and install, look on the vendor's websites for instructions on how to install and configure:
- Clone the repository
git clone https://github.com/metao1/iot-event-streaming-aws.git - Creating your Lambda functions
cd iot-lambda-app- Testing the lambda function
npm install
npm test-
Configure your slack webhook, after completing step 3, you should obtain the link to your webhook as follows:
-
Finally, see the details of your functions that were installed:
aws lambda list-functions Look for the ARN of the notification function, the string is similar to the one below:
aws lambda get-function --function-name=iot-telegram-dev-notification | grep FunctionArnSomething like below should be shown
FunctionArn = "arn:aws:lambda:<AWS_REGION>:XXXXXXXXX:function:iot_function"
Take a look at your AWS account's dashboard and see if the functions are already applied for validation.
- Create certificates for IoT devices
Certificates for IoT devices, Things, are necessary to guarantee secure and end-to-end encrypted communication, so in this case I chose to follow AWS's recommendation and create the certificates in the account dashboard. It's not complicated at all, see the link below:
https://docs.aws.amazon.com/iot/latest/developerguide/device-certs-create.html
Download the certificates, you will use them later to configure Node Red.
Get the Certificate ARN
To get the certificate ARN, run the command below:
aws iot list-certificates The certificate ARN looks like arn:aws:iot:REGION:xxxxxxxxx:cert/um_grande_hash
Now it's time to edit the files to upload resources to AWS.
terraform/
└── modules/
├── api_gw/
├── cloudwatch/
├── iot/
└── lambda/
- main.tf
Export AWS account id value to the environment variable.
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export AWS_REGION = "<AWS_REGION>"- Edit the deploy, I setup the files below, to configure the details of the deploy
cd terraform
touch terraform.tfvars # this will create a file for setting terraform varaible files securely
nano terraform.tfvars- terraform.tfvars: edit the variables, especially AWS_ACCOUNT_ID,AWS_REGION with specified values in env variables which you captured in the previous step
- Start terraform in the terraform folder
terraform init - Now is the time to validate the deploy plan
terraform plan -out=plano.out - The time has come to deploy the resources on AWS
terraform apply plano.out Cross your fingers, I would recommend getting a coffee, but it will be quick.
Once created, take a general look at the dashboard to see with your own eyes the work done by the code.
- Create secure communication between IoT devices and our AWS IoT MQTT broker is vital. We need to create relevent certificates on AWS Console (intentinally manually easy step) Head over to AWS Core and select Thing from the left navigation bar as show in the image below.
Click over 'rule_natural' and then select 'certificates' tabs. Click over 'Create certificate' butto to create new certificates.
In the shown dialog, download all certificates. !!Remember to download them now!!
After downloading certificates, we can see the a resource with link in 'Certificates' is created. This is in 'inactive' mode, we need to activate it in next step.
Click on the link, and then select 'attach policy'.
From 'Choose AWS iot policy' drop-down in the dialog, select 'iot_rule_national_policy'
Copy 'certificate arn' that has the following format
"arn:aws:iot:<AWS_REGION>:<ACCOUNT_ID>:cert/<CERT_ID>"Edit the 'terraform.tfvars' and paste the value in the 'iot_cert_arn'
nano terraform.tfvars- terraform.tfvars: pase the variable, iot_cert_arn
- Here you will need docker and docker-compose installed on your computer
cd docker
docker-compose up -d
-
Now , access nodered in your browser at: http://127.0.0.1:1880
-
Install the random node, as it generates the sensor data and simulations
https://flows.nodered.org/node/node- red-node-random
- Import the event flow that is inside the docker folder, the flow.json file
https://nodered.org/docs/user-guide/editor/workspace/import-export
- Configure the node red to access your Thing, which was created in step 3
Let's keep this code and include new features.
Distributed under the MIT license. See the LICENSE file for more information.