In this repository you will find a C++14 implementation of CIoTA; a lightweight framework that utilizes the blockchain concept to perform distributed and collaborative anomaly detection for devices with limited resources.
From, Tomer Golomb, Yisroel Mirsky and Yuval Elovici.
The Internet of Things (IoT) is the next evolution of the Internet[1]. Leading IoT experts believe that by 2020 there will be more than 50 billion devices connected to the Internet, and these devices will offer a variety of applications and services for both daily and critical uses [2]. The vision for these IoT devices is that they will autonomously communicate with one another to improve services and our daily lives.
Like any new architecture or technology, the IoT improves our lives but introduces disruptive elements as well. One known issue associated with IoT devices is that they tend to have vulnerabilities, which in some cases go unpatched by the manufacturer. An attacker can exploit these vulnerabilities for nefarious purposes[3]. Since IoT devices have been integrated into both daily and critical applications, their security is a significant concern.
One solution for protecting an IoT device is to install an intrusion detection system (IDS) [4]. If the IDS is anomaly-based, the system has the potential to detect new and emerging threats that target the device. However, there are two fundamental challenges with anomaly-based IDSs:
-
Adversarial Attacks - During the training phase, all observations are considered benign and are used to train a model which captures the device's normal behavior. After the training phase, the model enters an execution phase where it is used to detect when newly observed behaviors deviate from the norm. Therefore, an attacker who exploits a device before the execution phase can evade detection entirely.
-
False Positive - It is likely that false positives will occur if an anomaly detection model is trained on the observations of just a few devices. This is because some legitimate behaviors are rare and event-based, and therefore may not appear in the training data. For example, the motion detection logic of a smart camera or the response generated by a smoke detector sensing a fire.
However, consider the following scenario. Assume that all IoT devices of the same type simultaneously begin training their own anomaly detection model, based on their own locally observed behaviors. In this scenario, it is unlikely that the majority of these IoT devices would be exploited before the majority completes their training phases.
Using this underlying assumption, CIoTA (pronounced as syota) which is a lightweight, scalable framework that utilizes the blockchain concept to perform distributed and collaborative anomaly detection on resource limited devices, such as IoT devices.
The blockchain is a protocol for maintaining a distributed ledger. The ledger is a chain of blocks which is collectively agreed upon by the majority of participants in the network[5]. Each block is accepted into the chain if it can achieve consensus represented as the satisfaction of specific criteria (e.g., the proof of work criterion in bitcoin[6].). In the context of a device from a specific model, CIoTA uses the blockchain protocol to collaborate among devices of the same type to create a trusted anomaly detection model.
This is accomplished by merging locally trained models into a single global model. A block is a set of locally trained models from different devices. Each device validates the integrity of a block in progress by merging the models and by performing self-attestation. When the block reaches a size limit, it is closed and a new block begins. One closed block in the chain represents a trusted model which has been validated by the majority of devices in the system, and is therefore ready to be used (executed on-site). However, one may wish to track a chain of models, since a stronger model can be created by combining multiple blocks together.
The anomaly detection model used by CIoTA is an extensible Markov model (EMM). The EMM tracks a program's regular memory jump sequences and can be incrementally updated and merged with other models. To introduce new benign concepts into the merged model, we accept new states to the EMM only when there is a consensus among the models.
To understand how CIoTA works lets explore the following analogy: Let's assume that there is an agency called CIoTA which has many agents, each of which is in the enemy's territory. The mission of the agents is to (1) detect malicious acts, and (2) gather intel about what is happening and share a collection of intel (as rumors) with other agents. Since an agent is in the enemy's territory, the agent may receive false intel which introduces noise into the rumors conveyed to the other agents. Therefore, an agent will only trust a rumor (some intel) if that agent knows that at least Pc other agents have heard the same rumor. Finally, an agent accepts the most recent set of L trusted rumors as the latest description of the territory.
This scenario is implemented in the CIoTA framework as follows. Each IoT device has an agent which maintains a local model ML that is used to detect malicious behaviors in a particular application. An agent records new intel by updating ML with observations on the application's behavior. An agent shares its Intel (ML), in the form of a rumor, by adding ML to the chain's partial block, and then sending the chain to neighboring agents in the network. Other agents will only accept this partial block if it is longer than their partial block, and if they can attest that it is safe (by verifying it against their own local model). Thus, the partial block only grows if the majority of agents have verified that it contains a safe model. Once the partial block contains L reports, it is closed as a completed block. Therefore, an agent receives the latest intelligence from its fellow agents by replacing ML with the combined model contained within the newest closed block. Finally, to ensure that the rumors have indeed come from a specific agent, rumors are signed by a private key and verified with the respective public key.
The following describes an agent's three parallel processes:
This repository depend on two libraries ( + one for GUI utilities):
- matrixssl which is a lightweight TSL stack for embedded devices.
- spdlog which is a fast header only logging library.
For graphical interface, you need to install the following dependency as well: 2. Qt which is a Cross-platform software development for embedded devices.
To install spdlog, simply use apt-get:
$ sudo apt-get install libspdlog-dev
To install matrixssl, execute the "install_matrixssl.sh" file:
$ ./install_matrixssl.sh
To install Qt, simply use apt-get:
$ sudo apt-get install qt5-default qtdeclarative5-dev
Inside the header (and the src) folder you can find the code for CIoTA, the code is divided to the following directories:
- CppUtils - contains several classes and utilities.
- NetworkingUtils - contains utils that are networking related.
- Blockchain - lightweight implementation of the blockchain protocol.
- CIoTA - Include the implementation of CIoTA itself.
- ExecutableUtils - Include calsses that simplify CIoTA execution.
- JumpTracer - implementation of the extended markov chain (EMM).
The headers files are stored in the header directory and further documentation can be found at these folders. I recommend for anyone interested in working with this code to read the documentation in each folder to understand how the system and its components work.
Apart from the implementation itself, the test folder contains tests for many of these classes. These tests are consists of several files.
include a tests unit for the NetworkingUtils package. This test require two machine to simulate message and packets transfer between these two machines. The test can be executes as following:
$ ./NetworkingUtilsTest
Usage "NetworkingUtilsTest <rule>"
<rule> can be either "server" or "client" (case sensitive)
To simulate server, execute:
$ ./NetworkingUtilsTest server
To execute client, execute:
$ ./NetworkingUtilsTest client
After executing, simple follow the tests instructions.
TestSuite contains a list of unit tests for all packages. It composes from three different tests:
- Unit Tests - check each package alone using mock implementations.
- Synchronous simulation - Test CIoTA without networking. It simulates 4 agents in a synchronous manner (meaning that at each point of time only one agent is running) and checks that their functionality and state change as expected.
- Asynchronous simulation - Test CIoTA without networking. It simulates 4 agents in a asynchronous manner (meaning that all agents are running each on different thread) and check that their functionality and state change as expected.
To execute the unit tests, simple specify "suite" as parameter:
$ ./TestSuite "suite"
To execute the synchronous simulation, simple specify "sync" as parameter:
$ ./TestSuite "sync"
To execute the asynchronous simulation, simple specify "async" as parameter:
$ ./TestSuite "async"
Please note that these tests, and especially the simulations, are time-consuming and might take several minutes to complete. For ease of use, you can execute the ./runTestSuite that will execute all three tests:
$ ./runTestSuite
We prepared a mock example for CIoTA. The mock example executes the same simulation and tests that the simulation in the TestSuite tests uses, however, the mock include, unlike the TestSuite, networking utils, and illustrate CIoTA real execution with mock settings. The mock is consist of three executables:
- MockServer - this is a Mock example for CIoTA command line command and control (C&C) server that observed the network and receives notifications on attacks.
- MockGUIServer - this is a Mock example for CIoTA GUI command and control (C&C) server that observed the network and receives notifications on attacks. The decision to separate MockGUIServer from MockServer is to allow using this code without installing qt libraries.
- MockAgent - this is a mock example for CIoTA's agents on a fixed and predefined control-sequence.
The usage of the MockServer is as follows:
$ ./MockServer
Usage "MockServer [completeBlockSize]".
completeBlockSize is the number of devices in the mock simulation.
The usage of the MockGUIServer is as follows:
$ ./MockGUIServer
Usage "MockGUIServer [completeBlockSize]".
completeBlockSize is the number of devices in the mock simulation.
The usage of the MockAgent is as follows:
$ ./MockAgent
Usage "MockAgent [ID] [completeBlockSize] [ServerIp]".
ID is identifier for this device, needs to be unique between machines that executes the mock.
completeBlockSize is the number of devices in the mock simulation.
ServerIp is the address of CIoTA C&C server (the IP of the machine that executes MockServer).
As a proof of concept, CIoTA is composed of two entities: C&C server and agents. Agents, by their nature, are command line tools and works without a user interface. However, C&C server might include user interactions, and therefore we supply two possible executables for it. The agents require the C&C server for alerts and for connecting to the network. Therefore executing the C&C is a preliminary step to executing the agents.
All these three executables can be configured using INI files. The format and possible values in the INI files are as following:
Tracer section contains all the configurable options for the tracer. The possible options are:
| Option | type | Description |
|---|---|---|
| application | string | path to the application executable. |
| mask | Address size (32/64 bit) | mask that used to reduce the model size and ignore noises. |
| filterFrom | Address size (32/64 bit) | filter all address lower then the given value. |
| filterTo | Address size (32/64 bit) | filter all address bigger then the given value. |
| windows | size_t | How many events are considering as sequence to test. |
| threshold | size_t | Threshold for the anomaly score, anomaly score under the given threshold will rise an alert. |
| trainInterval | uint | How much time the model should be in training mode. |
The CIoTA section contains all the configurable options for the CIoTA protocol. The possible options are:
| Option | type | Description |
|---|---|---|
| multicastGroup | string (ip) | The IP used by the multicast protocol. |
| port | ushort | Port for the multicast protocol. |
| controlIP | string (ip) | The IP of the command (C&C) server. |
| controlPort | ushort | Port of the command (C&C) server. |
| pConsensus | size_t | The percent of records inside a block needed to accept a record. |
| shareInterval | uint | The interval between two broadcast attempts. |
| receiveInterval | uint | The interval between handling two broadcasts. |
| alpha | ubyte | The similarity percent between models needed to accept untrusted model. |
| completeBlockSize | size_t | The size needed for a block to consider as a complete block. |
| capacity | size_t | The amount of memory allocated for the chain. |
| fragmentSize | size_t | The maximum size of a broadcast packet. |
| EntityKey | string | Unique key to identify the entity (which agent or if server). |
The Log section contains all the configurable options for the Logging CIoTA. The possible options are:
| Option | type | Description |
|---|---|---|
| loggerSize | int | The size to use for the logger. |
| loggerRotation | int | How many files to use in the logger. |
We provides "default_conf.ini" as a default configuration file for CIoTA.
The "CIoTA_CLI_Server" is a command line tool to monitor CIoTA states. CIoTA_CLI_Server usage is as follow:
$ ./CIoTA_CLI_Server
Usage "CIoTA_CLI_Server configurationFile [LogFilePath]".
configurationFile is the INI file containing CIoTA configurations.
LogFilePath is optional argument that specify the log file path.
The "CIoTA_GUI_Server" is a graphical user interface tool to monitor CIoTA states. CIoTA_GUI_Server usage is as follow:
$ ./CIoTA_GUI_Server
Usage "CIoTA_GUI_Server configurationFile [LogFilePath]".
configurationFile is the INI file containing CIoTA configurations.
LogFilePath is optional argument that specify the log file path.
The following is an image of the graphical user interface:
The "CIoTA_Agent" is a utility that should run on the target devices, its usage:
$ ./CIoTA_Agent
Usage "CIoTA_Agent configurationFile [LogFilePath]".
configurationFile is the INI file containing CIoTA configurations.
LogFilePath is optional argument that specify the log file path.
We export CIoTA implementation as static libraries that can be used to expand the current functionalities.
To implement a custom detection policy, you should implements your policy as a function of the following signature:
void policy(void);
To make the agent use your function, you should pass it as a reference to the startAgent through the AgentParameterPack structure. For further details, we recommend you to read the README file inside the /header/CIoTA and the /header/AnomalyDetection folders.
To execute the agent with a custom anomaly detection model, you should implement the ModelUtilities interface. The ModelUtilities is an adapter that bridge CIoTA to the anomaly detection model implementation. After implementing the ModelUtilities based on your custom implementation of the anomaly detection model, you should perform the following steps:
- Create a new compression and encryption library.
- Create instance of model utilities.
- Initiate a new MulticastServerDescriptor instance.
- Initiate a new ExtendedAnomalyListener instance.
- Initiate a new CIoTABlockchainApplication.
- Initiate a new Agent.
- Execute the agent.
For ease of use we implemented the method runAgent that implements these steps.
Example (from the MockAgent.cpp file)
AES_SEALER sealer(&idBuf);
SHA256_FUNC hash{};
MockCompressionLibrary compressionLibrary;
CryptoLibrary<SHA256_OUTPUT_SIZE> cryptoLibrary(&hash, &sealer);
CIoTA::startAgent<JumpTracer, SHA256_OUTPUT_SIZE>(
&pack,
&utilities,
nullptr,
&cryptoLibrary,
&compressionLibrary
);
For further details, we recommend you to read the README file inside the /header/CIoTA and the /header/AnomalyDetection folders.
In each of the header folders, we wrote a README file that contains all the documentation of that folder; we recommend reading all of them for better understanding the code structure and architecture.
To build CIoTA's whole package, you can simply execute the following:
$ mkdir build
$ cd build
$ cmake ..
$ make
The above build several executables, the tests, mocks, CIoTA_Agent and CIoTA_control_Server. You can build only the selected executables through the CMake interface. The build targets are:
- TestsSuite
- NetworkingUtilsTest
- MockAgent
- MockServer
- MockGUIServer
- CIoTA_CLI_Server
- CIoTA_GUI_Server
- CIoTA_Agent
- CIoTAlib
CIoTA includes partial logging, as we implemented it during the later part of the development. To control the logging level you can change the DLOG_LEVEL parameter. DLOG_LEVEL = 0, is no logging at all. DLOG_LEVEL = 5, is massive logging. We recommend keeping it on 4 (default) to understand what going on.
This project is licensed under the MIT License - see the LICENSE.txt file for details
Tomer Golomb [email protected]
[1] KHAN, Rafiullah, et al. Future internet: the internet of things architecture, possible applications and key challenges. In: Frontiers of Information Technology (FIT), 2012 10th International Conference on. IEEE, 2012. p. 257-260.
[2] EVANS, Dave. The internet of things: How the next evolution of the internet is changing everything. CISCO white paper, 2011, 1.2011: 1-11.
[3] SCHNEIER, Bruce. The Internet of things is wildly insecure—and often unpatchable. Schneier on Security, 2014, 6.
[4] GARCIA-TEODORO, Pedro, et al. Anomaly-based network intrusion detection: Techniques, systems and challenges. computers & security, 2009, 28.1: 18-28.
[5] SWAN, Melanie. Blockchain: Blueprint for a new economy. " O'Reilly Media, Inc.", 2015.
[6] NAKAMOTO, Satoshi. Bitcoin: A peer-to-peer electronic cash system. 2008.