Connected Mobility Solution on AWS

Connected Mobility Solution on AWS | 🚧 Feature request | 🐛 Bug Report | ❓ General Question

Note: If you want to use the solution without building from source, navigate to the AWS Solution Page.

If you want to jump straight into building and deploying, click here

Table of Contents

• Connected Mobility Solution on AWS
  • Table of Contents
  • Solution Overview
  • Architecture Diagrams
    • ACDP Architecture Diagram
    • ACDP Deployment Sequence Diagram
    • Module Deployment Sequence Diagram
  • CMS Modules
  • Deployment Prerequisites
    • Clone the Repository
    • Required Tools
    • Required Tool Versions
    • Install Required Tools (OSX/Linux)
      • NVM
      • Node / NPM
      • Yarn
      • Pyenv
      • Python / Pip
      • Pipenv
      • AWS CLI
      • AWS CDK Toolkit
      • Verify Required Tool Installations
    • Manual Steps
      • Create an Amazon Route 53 Hosted Zone
    • Install Solution Dependencies
    • Setup Environment Variables
  • Deploy
    • Prerequisites
    • Build the Solution's Modules
    • Upload Assets to S3
    • Deploy on AWS
    • Monitoring the ACDP Deployment
    • Module Dependencies
      • ACDP Deployment Order
      • CMS Module Deployment Order
        • Deployment Order Diagram
      • Functional Dependencies
        • Functional Dependency Diagram
    • Deploy CMS Modules via Backstage
      • Example Module Deployment via Backstage
  • Cost Scaling
  • Collection of Operational Metrics
  • Uninstall the Solution
  • Developer Guide
    • Dependencies
      • Python
      • Node
    • Logging
      • Lambda Functions
      • Backstage Logs
    • Pre-Commit Hooks
    • Unit Test
  • License

Solution Overview

The Connected Mobility Solution (CMS) on AWS provides the automotive industry customers the capability to communicate between vehicles and the AWS Cloud, manage and orchestrate CMS on AWS deployments from a centralized developer platform, securely authenticate and authorize users and services, onboard vehicles into AWS IoT Core, create vehicle profiles for storing data about registered vehicles, capture and store telemetry data emitted from registered vehicles, and consume data from FleetWise campaigns. Additionally it provides capabilities to query stored vehicle data, create alerts and subscribe to notifications based on data thresholds, visualize vehicle telemetry data through a provided dashboard, and simulate connected vehicle data.

For more information and a detailed deployment guide, visit the Connected Mobility Solution on AWS solution page.

Architecture Diagrams

ACDP Architecture Diagram

ACDP Deployment Sequence Diagram

Module Deployment Sequence Diagram

CMS Modules

For detailed information visit the modules' README

• ACDP
  • Backstage
• Alerts
• API
• Auth
• Auth Setup
• Connect & Store
• Config
• EV Battery Health
• FleetWise Connector
• Provisioning
• Vehicle Simulator
• VPC

Deployment Prerequisites

Clone the Repository

If you have not done so, first clone the repository, and then cd into the created directory. If you have already cloned the repository, ensure you still cd into the solution's directory.

git clone https://github.com/aws-solutions/connected-mobility-solution-on-aws.git
cd connected-mobility-solution-on-aws/

WARNING: If you do not cd into the solution's directory before installing tools, the correct versions may not be installed.

Required Tools

To deploy CMS on AWS, a variety of tools are required. These deploy instructions will install the following to your machine:

• NVM
• Node
• NPM
• Yarn
• Pyenv
• Python
• Pip
• Pipenv
• AWS CLI
• AWS CDK Toolkit

Required Tool Versions

Certain tools also require specific versions. See the table below for the appropriate versions. Following the provided install instructions will install the correct versions.

For tools not listed here, stable versions should work appropriately.

Dependency	Version
NodeJS	18.20.*
Python	3.12.*

Install Required Tools (OSX/Linux)

Install the following tools in the order instructed here. Where appropriate, a script has been provided to aid in install. Otherwise, please visit the installation guide provided by the tool's publisher to ensure a correct installation.

WARNING: If after a successful installation, a command is not found, you may need to restart your terminal.

NVM

Follow the nvm installation guide to install NVM. Ensure your installation properly set your path by running the script below.

nvm --version

Node / NPM

nvm install
nvm use

For more information see the nvm usage guide for installing the correct version of Node. Manually installing Node without the use of nvm is not recommended.

Yarn

Follow the yarn installation guide. Ensure your installation properly set your path by running the script below.

yarn --version

Pyenv

Follow the pyenv installation guide to install Pyenv. You will likely need to manually add Pyenv to your PATH by following the provided instructions. Ensure your installation properly set your path by running the script below.

pyenv --version

Python / Pip

pyenv install -s

For more information see the pyenv usage guide for installing the correct version of Python. Manually installing Python without the use of pyenv is not recommended.

Pipenv

Follow the pipenv installation guide to install Pipenv. You will likely need to manually add Pipenv to your PATH by following the provided instructions. Ensure your installation properly set your PATH by running the script below.

pipenv --version

AWS CLI

Follow the installation instructions laid out in the AWS CLI install page. This install is OS specific, and includes multiple options for both a system wide, and user specific install. Follow the install instructions most appropriate to you. Ensure your installation properly set your PATH by running the script below.

aws --version

AWS CDK Toolkit

Follow the installation guide to install the AWS CDK toolkit. Ensure your installation properly set your PATH by running the script below.

cdk --version

Verify Required Tool Installations

Run the following command to verify the proper installation of all of the tools listed above. If any errors are displayed, attempt to reinstall that tool.

make verify-required-tools

Manual Steps

Some of the modules may need certain manual steps. Here is the list of modules which requires manual steps. Please refer to module READMEs for instructions.

• ACDP
• CMS EV Battery Health
• CMS Predictive Maintenance

Create an Amazon Route 53 Hosted Zone

To deploy ACDP, either an Amazon Route53 Hosted Zone or external DNS provider is required to be setup in your account. When using Route53, you can either use a Public or Private Hosted Zone, but if you use private, you must manually configure a TLS Certificate in ACM. You will provide the Route53 Hosted Zone ID and a fully qualified domain name for this deployment in the following step when you setup your environment variables. Creating a hosted zone is a manual step. For more details, see Working with hosted zones.

Install Solution Dependencies

Now that you have the correct tools, you can install the dependencies used by the solution using make install. After installing, activate the environment which contains the dependencies.

make install

Setup Environment Variables

To deploy the solution, a variety of environment variables are required. These environment variables will be used to provide the values to your deployment. To generate the file which will store these environment variables and provide their values, run the following command:

make create-rc-file
source .cmsrc

IMPORTANT: The source .cmsrc command is essential for getting the configuration settings for CMS set in your terminal.

The ROUTE53_HOSTED_ZONE_ID can be found from the Amazon Route53 Hosted Zone you setup in the previous step. Use the AWS Management Console to find this information. It is located under the 'Hosted zone details' expander.

Deploy

Refer to the deployment diagram for a detailed walk-through of what is deployed.

Prerequisites

Ensure you've followed the steps in the previous deployment prerequisites section.

• Prerequisite tools installed. Refer to the install required tools sections for details.
• Solution dependencies installed. Refer to the install solution dependencies section for details.
• An Amazon Route53 Hosted Zone in the deployment account. Refer to the create an Amazon Route 53 Hosted Zone section for details.
• Environment variables set. Refer to the setup environment variables section for details.

Build the Solution's Modules

The build target manages dependencies, builds required assets (e.g. packaged lambdas), and creates the AWS CloudFormation templates for all modules.

make build-all

NOTE: There is also a build target. Running that instead of build-all will trigger an error for a "missing file".

Upload Assets to S3

The upload target creates the necessary buckets for, and uploads, the global and regional assets. It also uploads the Backstage .zip asset.

make upload

Deploy on AWS

The deploy target deploys all CMS modules, including the ACDP, in an enforced order.

make deploy

Monitoring the ACDP Deployment

After the CDK deployment is completed, browse to CodePipeline in the AWS Management Console and verify that the "acdp-backstage-pipeline" execution completes successfully.

After the pipeline has completed, the deployment can be considered successfully complete and Backstage is ready for use.

Module Dependencies

It is worth noting that many dependencies, be it deployment or functional, can be met by other means rather than the specified module, such as manually creating/providing SSM Parameters, or any other required resources and their respective functionality. Details for how to accomplish this will vary by module and dependency, and are best understood by reviewing the source code for the modules under question.

ACDP Deployment Order

The ACDP deployment is dependent on the VPC and Auth Setup deployments, since it uses them as its own VPC and IdP configuration. These deployments can be separate from the configuration deployments used to configure further CMS module deployments.

NOTE: Backstage currently only support the usage of cms as the AppUniqueId when deploying from Backstage. Since AppUniqueId must be unique per module deployment, to utilize a separate deployment of VPC or Auth Setup for ACDP and CMS module deployments, the VPC and Auth Setup deployments used by ACDP must specify an AppUniqueId other than cms.

CMS Module Deployment Order

All CMS modules have dependencies on the initial three deployments for configuring CMS: VPC, Auth Setup, and CMS Config.

Some CMS on AWS modules have secondary dependencies on other modules and must be deployed in order.

The remaining modules do not have dependencies on other modules and can be deployed in any order after CMS Config.

Deployment Order Diagram

Functional Dependencies

CMS Modules also have functional dependencies that do not inherently align with deployment dependencies.

Functional Dependency Diagram

Deploy CMS Modules via Backstage

Example Module Deployment via Backstage

The following instructions detail how to deploy the CMS Vehicle Simulator module. The same steps can be applied to other modules as well by replacing the URLs and names.

1. Navigate to the Backstage URL in a web browser (FULLY_QUALIFIED_DOMAIN_NAME that was specified during deployment).

2. Sign in to Backstage using the identity provider of choice, configurd via the Auth Setup module and IdentityProviderId.

3. On Backstage, navigate to the Create page available from the Catalog menu in the side bar. Select the CHOOSE button on the CMS Vehicle Simulator card.

4. Fill in the form as required by the Vehicle Simulator template and click the Next button and then the Review button.

5. Click the Create button.

   

6. Monitor the deployment and ensure that the Vehicle Simulator module deploys successfully.

   

Cost Scaling

In general, cost can differ dramatically based on usage of CMS.

For at rest cost and detailed pricing information, see the implementation guide.

Collection of Operational Metrics

This solution collects anonymized operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the implementation guide.

Uninstall the Solution

1. Capture and store the deployment UUIDs of the solution.

   This is used to look for any resources not destroyed by CloudFormation after teardown completes

   make get-acdp-deployment-uuid
   make get-cms-deployment-uuid

   the outputs will be uuidv4 strings, capture and store both:

   XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

2. Delete CMS modules in order.

   make destroy

   NOTE: Backstage might fail to delete due to the ACM certificate creation custom resource. After delete fails, click delete again and select retain on the custom resource. This will not leave any resources in the account.

   

3. Delete the Backstage ACM Certificate (optional)

   Navigate to Amazon Certificate Manager, and delete the Backstage certificate.

4. Manually cleanup the following resources:

   • S3 buckets
   • DynamoDB tables
   • Cognito user pool
   • KMS keys

   Locate the leftover resources using the following command which first requires you to export the DEPLOYMENT_UUID variable using each of the values previously acquired from AWS Systems Manager.

   If you tore down the ACDP stack without capturing the UUIDs, the below command can be run by removing the Solutions:DeploymentUUID Key filter, however the results will include other CMS on AWS stacks if they exist, so use this method with caution.

   export DEPLOYMENT_UUID=<DEPLOYMENT_UUID_VALUE_FROM_SSM>
   
   aws resourcegroupstaggingapi get-resources --tag-filters \
   Key=Solutions:SolutionID,Values=SO0241 \
   Key=Solutions:DeploymentUUID,Values=$DEPLOYMENT_UUID \
   --query "ResourceTagMappingList[*].ResourceARN"

   This query results in a list of ARNs to assist you with locating the resources in the AWS Management Console. Resources can then be manually deleted, or deleted via a script, utilizing the resource ARNs where appropriate.

   WARNING: Some resources may take some time to cleanup after CloudFormation finishes tearing down, and could show in the output even if they no longer exist. For example, Amazon VPC, Fargate, and Amazon ECS resources can remain queryable for up to 30 minutes after deletion.

Developer Guide

Dependencies

To properly manage dependency versions, ensuring consistency and security across solution installations and usages, lock files (Pipfile.lock, yarn.lock, package-lock.json) are included throughout the repository.

For fresh installations, or for simply ensuring you have the correct dependencies as specified in the lock files, the make install target should be used, which will install all lock file dependencies throughout the solution, without checking for or performing dependency upgrades.

To upgrade or add new dependencies, lock file updates should be performed. For this, the make upgrade target should be used, which will check for dependency upgrades based on semver versions specified throughout the solution, and update the lock files accordingly. This will also install upgraded node dependencies, but will not install upgraded python dependencies. To install upgraded python dependencies, a subsequent make install run is necessary.

NOTE: Upgraded or installing Python or Node dependencies individually is also supported. Run make help for a full list of supported make targets to support this behavior.

See below for more details on Python and Node dependency management.

Python

This solution uses pipenv for Python dependency management. For more information, see the pipenv documentation.

make install will install python dependencies from Pipfile.lock files without checking for or performing dependency upgrades. This is to ensure consistent versioning of dependencies as specified in the included lock files.

make upgrade will upgrade Pipfile.lock files across the solution without installing dependencies. To then install upgraded dependencies, run make install.

Node

This solution uses yarn and npm for management of Node dependencies. For more information, see the yarn documentation and npm documentation.

make install will install node dependencies from package-lock.json or yarn.lock files without checking for or performing dependency upgrades. This is to ensure consistent versioning of dependencies as specified in the included lock files.

make upgrade will upgrade package-lock.json and yarn.lock files across the solution while also installing upgraded depdnencies. A subsequent make install run is not necessary for installing upgraded node dependencies, but is recommended for installing upgraded Python dependnecies, ensuring alignment between installed dependencies and the updated lock files.

Logging

By default, this solution implements safe logging which does not expose any sensitive or vulnerable information. CMS on AWS does not currently support a one-step system for enabling more detailed debug logs. To add additional logs to the solution, you are required to alter the source code. Examples of logging implementations can be found in the existing Lambda functions.

Lambda Functions

By default, the solution disabled Lambda event logging, which contains sensitive information. However, this functionality is provided by the AWS Lambda Powertools library which is utilized by each Lambda function. To quickly enable event logging, navigate to the Lambda function in the AWS Management Console and add the following Lambda environment variable:

POWERTOOLS_LOGGER_LOG_EVENT="true"

For other logging options and methods for enabling event logging, see the AWS Lambda Powertools documentation.

Backstage Logs

By default, the solution's deployment instructions deploy the ACDP and Backstage with a log level of "INFO". To enable debug logs for Backstage, change the following environment variable when you deploy the solution:

export BACKSTAGE_LOG_LEVEL="debug"

Pre-Commit Hooks

This solution contains a number of linters and checks to ensure code quality. If you are not planning to commit code back to source, you can run the pre-commit hooks manually using the following command:

make pre-commit-all

Unit Test

After making changes, run unit tests to make sure added customization passes the tests:

make unit-test

License

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.