Skip to content

Commit

Permalink
[VulnCheck] Initial Implementation of VulnCheck Connector
Browse files Browse the repository at this point in the history
  • Loading branch information
maddawik committed Jan 12, 2025
1 parent 714b94d commit 77281d0
Show file tree
Hide file tree
Showing 28 changed files with 2,278 additions and 0 deletions.
5 changes: 5 additions & 0 deletions external-import/vulncheck/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
src/config.yml
src/__pycache__
src/logs
src/*.gql
src/.venv
19 changes: 19 additions & 0 deletions external-import/vulncheck/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
FROM python:3.12.5-alpine
ENV CONNECTOR_TYPE=EXTERNAL_IMPORT

# Copy the connector
COPY src /opt/opencti-connector-vulncheck

# Install Python modules
# hadolint ignore=DL3003
RUN apk update && apk upgrade && \
apk --no-cache add git build-base libmagic libffi-dev libxml2-dev libxslt-dev

RUN cd /opt/opencti-connector-vulncheck && \
pip3 install --no-cache-dir -r requirements.txt && \
apk del git build-base

# Expose and entrypoint
COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
227 changes: 227 additions & 0 deletions external-import/vulncheck/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
# OpenCTI VulnCheck Connector

The VulnCheck Connector for OpenCTI is a standalone Python process designed to
integrate VulnCheck's extensive threat intelligence into the OpenCTI platform.
VulnCheck aggregates data from a wide range of sources, providing actionable
insights into vulnerabilities, exploits, ransomware activity, threat actors,
and more.

Table of Contents

<!--toc:start-->
- [OpenCTI VulnCheck Connector](#opencti-vulncheck-connector)
- [Introduction](#introduction)
- [Installation](#installation)
- [Requirements](#requirements)
- [Configuration](#configuration)
- [OpenCTI Configuration](#opencti-configuration)
- [VulnCheck Connector Configuration](#vulncheck-connector-configuration)
- [Deployment](#deployment)
- [Docker Deployment](#docker-deployment)
- [Manual Deployment](#manual-deployment)
- [Usage](#usage)
- [Behavior](#behavior)
- [Data Ingestion](#data-ingestion)
- [Data Volume](#data-volume)
- [NIST and VulnCheck NVD](#nist-and-vulncheck-nvd)
- [Debugging](#debugging)
- [Useful Resources](#useful-resources)
<!--toc:end-->

## Introduction

The VulnCheck Connector retrieves and translates data from VulnCheck into STIX
objects. It supports feeds for Known Exploited Vulnerabilities (KEVs), NVD-2,
ransomware, threat actors, exploits, botnets, initial access indicators, and IP
intelligence. Each data source is processed to create structured, meaningful
objects like vulnerabilities, malware, infrastructure, and relationships.

The VulnCheck Connector helps organizations enrich their threat intelligence
within OpenCTI by automating the ingestion of curated vulnerability and threat
actor data. This enables:

- **Proactive Risk Management**: Identify and prioritize vulnerabilities
actively exploited in the wild, improving patch management and defensive
measures.
- **Threat Actor Profiling**: Gain insights into ransomware groups, botnets,
and advanced persistent threat (APT) groups with detailed metadata and
associated tactics.
- **Contextual Analysis**: Link vulnerabilities to threat actors, exploits, and
malicious infrastructure to build a comprehensive understanding of potential
threats.
- **Enhanced Situational Awareness**: Integrate multi-source threat
intelligence into a unified platform, enabling more informed decision-making
for security operations and incident response teams.

## Installation

### Requirements

- OpenCTI Platform >= 6.4.5
- A Valid [VulnCheck](https://www.vulncheck.com/) API Key

## Configuration

There are several configuration options, which can be set either in
`docker-compose.yml` (for Docker deployments) or in `config.yml` (for manual
deployments).

### OpenCTI Configuration

Below are the parameters you'll need to set for OpenCTI:

| Parameter | config.yml | Docker Environment Variable | Mandatory | Description |
|-------------------|------------|-----------------------------|-----------|----------------------------------------------|
| OpenCTI URL | `url` | `OPENCTI_URL` | Yes | The URL of the OpenCTI platform. |
| OpenCTI Token | `token` | `OPENCTI_TOKEN` | Yes | The API token for authenticating with OpenCTI. |

### VulnCheck Connector Configuration

Below are the parameters you'll need to set for running the connector:

| Parameter | config.yml | Docker Environment Variable | Default | Mandatory | Description |
|-----------------|----------------|------------------------------------|------------------------------------------------------------------------------------------------------------------------|-----------|-----------------------------------------------------------------------------|
| Connector ID | `id` | `CONNECTOR_ID` | / | Yes | A unique `UUIDv4` identifier for this connector. |
| Connector Type | `type` | `CONNECTOR_TYPE` | EXTERNAL_IMPORT | Yes | Specifies the type of connector. Should always be set to `EXTERNAL_IMPORT`. |
| Connector Name | `name` | `CONNECTOR_NAME` | VulnCheck | Yes | The name of the connector as it will appear in OpenCTI. |
| Connector Scope | `scope` | `CONNECTOR_SCOPE` | VulnCheck | Yes | Must be set as `VulnCheck` |
| Log Level | `log_level` | `CONNECTOR_LOG_LEVEL` | info | Yes | Sets the verbosity of logs. Options: `debug`, `info`, `warn`, `error`. |
| API Base URL | `api_base_url` | `CONNECTOR_VULNCHECK_API_BASE_URL` | None | Yes | The base URL for the VulnCheck API (e.g., `https://api.vulncheck.com/v3`). |
| API Key | `api_key` | `CONNECTOR_VULNCHECK_API_KEY` | None | Yes | The API key for authenticating with VulnCheck's API. |
| Data Sources | `data_sources` | `CONNECTOR_VULNCHECK_DATA_SOURCES` | botnets,epss,exploits,initial-access,ipintel,nist-nvd2,ransomware,snort,suricata,threat-actors,vulncheck-kev,vulncheck-nvd2 | Yes | List of data sources to collect intelligence from. |

## Deployment

### Docker Deployment

Before building the Docker container, you need to set the version of pycti in
`requirements.txt` equal to whatever version of OpenCTI you're running.
Example, `pycti==5.12.20`. If you don't, it will take the latest version, but
sometimes the OpenCTI SDK fails to initialize.

Build a Docker Image using the provided `Dockerfile`.

Example:

```shell
# Replace the IMAGE NAME with the appropriate value
docker build . -t [IMAGE NAME]:latest
```

Make sure to replace the environment variables in `docker-compose.yml` with the
appropriate configurations for your environment. Then, start the docker
container with the provided docker-compose.yml

```shell
docker compose up -d
# -d for detached
```

### Manual Deployment

Create a file `config.yml` based on the provided `config.yml.sample`.

Replace the configuration variables (especially the "**ChangeMe**" variables)
with the appropriate configurations for you environment.

Install the required python dependencies (preferably in a virtual environment):

```shell
pip3 install -r requirements.txt
```

Then, start the connector from `vulncheck/src`:

```shell
python3 main.py
```

## Usage

After Installation, the connector should require minimal interaction to use,
and should update automatically at a regular interval specified in your
`docker-compose.yml` or `config.yml` in `duration_period`.

However, if you would like to force an immediate download of a new batch of
entities, navigate to:

`Data management` -> `Ingestion` -> `Connectors` in the OpenCTI platform.

Find the connector, and click on the refresh button to reset the connector's
state and force a new download of data by re-running the connector.

> [!NOTE]
> The VulnCheck Connector is designed to fetch data once every 24 hours. This
> approach ensures the connector remains efficient and minimizes the risk of
> overloading the VulnCheck API or the OpenCTI platform.
## Behavior

The VulnCheck Connector integrates VulnCheck's threat intelligence data into
the OpenCTI platform, converting raw data from various feeds into structured
STIX objects. Below is an overview of its behavior and functionality:

### Data Ingestion

The connector retrieves data from VulnCheck's API and imports it into OpenCTI
as STIX objects. The following types of data are processed:

- **VulnCheck KEV**: Populates OpenCTI with vulnerabilities actively exploited
in the wild, focusing on high-priority risks.
- **NVD-2**: Imports vulnerability information enriched with CVSS scores,
descriptions, and associated CPEs. (VulnCheck NVD-2 is available for
subscribers, NIST NVD-2 is available for other users)
- **Exploits**: Maps exploits to vulnerabilities and generates corresponding
Malware objects in OpenCTI.
- EPSS Enrichment: Adds vulnerabilities along with their EPSS scores and
percentiles, helping prioritize remediation efforts based on exploit
probability.
- **Ransomware**: Creates Malware objects for ransomware families, linking them
to associated vulnerabilities.
- **Threat Actors**: Adds Threat Actor objects with external references,
relationships to targeted vulnerabilities, and descriptive metadata.
- **Botnets**: Ingests infrastructure data associated with botnet activities
and links them to targeted vulnerabilities.
- **Initial Access Indicators**: Maps CPEs and vulnerabilities leveraged for
initial access tactics.
- **IP Intelligence**: Adds infrastructure and IP-related intelligence,
including countries and related vulnerabilities.
- **Snort/Suricata Rules**: Ingests Snort and Suricata rules as Indicators.

### Data Volume

> [!WARNING]
> Users should be aware of the significant resource impact when enabling
> certain data sources within the VulnCheck Connector. Enabling any of these
> data sources will require significant time and resources (primarily RAM) to
> process. After the initial ingestion, operations will become more efficient
> with reduced daily data pulls.
#### NIST and VulnCheck NVD

On the initial run, these feeds ingest every vulnerability available, creating
a massive number of STIX Objects and Relationships (e.g., Software `has`
Vulnerability).

Subsequent runs will only pull vulnerabilities published on the previous day,
significantly reducing the data volume after the initial ingestion. The first
run requires substantial RAM and processing time due to the volume of data
being processed. Please ensure your environment is prepared to support this
before enabling the source.

## Debugging

The connector can be debugged by setting the appropiate log level. Note that
logging messages can be added using
`self.helper.connector_logger,{LOG_LEVEL}("Sample message")`, i. e.,
`self.helper.connector_logger.error("An error message")`.

---

## Useful Resources

OpenCTI documentation for connectors:

- [OpenCTI Ecosystem](https://filigran.notion.site/OpenCTI-Ecosystem-868329e9fb734fca89692b2ed6087e76)
- [Connectors Deployment](https://docs.opencti.io/latest/deployment/connectors/)
- [Connectors Development](https://docs.opencti.io/latest/development/connectors/)
39 changes: 39 additions & 0 deletions external-import/vulncheck/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
version: "3"
services:
connector-vulncheck:
image: opencti/connector-vulncheck:6.2.4
environment:
# Connector's generic execution parameters
- OPENCTI_URL=http://localhost
- OPENCTI_TOKEN=CHANGEME
# Connector's definition parameters REQUIRED
- CONNECTOR_ID=CHANGEME
- CONNECTOR_NAME=VulnCheck
- CONNECTOR_SCOPE=VulnCheck
- CONNECTOR_LOG_LEVEL=error
- CONNECTOR_DURATION_PERIOD=PT5M # ISO8601 format in String, start with 'P...' for Period

# Connector's definition parameters OPTIONAL
# - CONNECTOR_QUEUE_THRESHOLD=500 # Default 500Mo, Float accepted
# - CONNECTOR_RUN_AND_TERMINATE=False # Default False, True run connector once
# - CONNECTOR_SEND_TO_QUEUE=True # Default True
# - CONNECTOR_SEND_TO_DIRECTORY=False # Default False
# - CONNECTOR_SEND_TO_DIRECTORY_PATH=CHANGEME # if CONNECTOR_SEND_TO_DIRECTORY is True, you must specify a path
# - CONNECTOR_SEND_TO_DIRECTORY_RETENTION=7 # Default 7, in days

# Connector's custom execution parameters
- CONNECTOR_VULNCHECK_API_BASE_URL=https://api.vulncheck.com/v3
- CONNECTOR_VULNCHECK_API_KEY=CHANGEME
- CONNECTOR_VULNCHECK_DATA_SOURCES=botnets,epss,exploits,initial-access,ipintel,nist-nvd2,ransomware,snort,suricata,threat-actors,vulncheck-kev,vulncheck-nvd2

# Add proxy parameters below if needed
# - HTTP_PROXY=CHANGEME
# - HTTPS_PROXY=CHANGEME
# - NO_PROXY=CHANGEME
restart: always
# networks:
# - docker_default
# networks:
# default:
# external: true
# name: docker_default
6 changes: 6 additions & 0 deletions external-import/vulncheck/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh

cd /opt/opencti-connector-vulncheck

# Launch the worker
python3 main.py
25 changes: 25 additions & 0 deletions external-import/vulncheck/src/config.yml.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
opencti:
url: 'http://localhost:PORT'
token: 'ChangeMe'

connector:
id: 'ChangeMe'
type: 'EXTERNAL_IMPORT'
name: 'External Import Connector VulnCheck'
scope: 'VulnCheck'
log_level: 'info'
duration_period: 'PT5M' # Interval given for scheduler process in ISO-8601 format
#============================================#
# Optional connector's definition parameters #
#============================================#
#queue_threshold: 500
#run_and_terminate: 'False'
#send_to_queue: 'True'
#send_to_directory: 'False'
#send_to_directory_path: 'ChangeMe'
#send_to_directory_retention: 7

connector_vulncheck:
api_base_url: 'ChangeMe'
api_key: 'ChangeMe'
data_sources: 'botnets,epss,exploits,initial-access,ipintel,nist-nvd2,snort,suricata,ransomware,threat-actors,vulncheck-kev,vulncheck-nvd2'
20 changes: 20 additions & 0 deletions external-import/vulncheck/src/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import traceback

from vclib import ConnectorVulnCheck

if __name__ == "__main__":
"""
Entry point of the script
- traceback.print_exc(): This function prints the traceback of the exception to the standard error (stderr).
The traceback includes information about the point in the program where the exception occurred,
which is very useful for debugging purposes.
- exit(1): effective way to terminate a Python program when an error is encountered.
It signals to the operating system and any calling processes that the program did not complete successfully.
"""
try:
connector = ConnectorVulnCheck()
connector.run()
except Exception:
traceback.print_exc()
exit(1)
4 changes: 4 additions & 0 deletions external-import/vulncheck/src/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
vulncheck-sdk==0.0.7
pycti==6.2.18
pydantic==2.8.2
validators==0.33.0
3 changes: 3 additions & 0 deletions external-import/vulncheck/src/vclib/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .connector import ConnectorVulnCheck

__all__ = ["ConnectorVulnCheck"]
Loading

0 comments on commit 77281d0

Please sign in to comment.