BUG OpenCTI Connector is not working

[CyberAbwehr]

IntelOWL: v5.2.3
OPENCTI: 5.12.14

Cant send observable to OPENCTI
"OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration..."

OPENCTI Secrets are set:
api_key_name
url_key_name

Plugins -> Connectors -> OpenCTI -> perform health check
it is up and running

docker logs intelowl_celery_worker_default

INFO:intel_owl.tasks:Configuring plugin OpenCTI for job 8 with task 3093249f-f5b9-4e2c-8501-9b41924cd6db
INFO:intel_owl.tasks:Starting plugin OpenCTI for job 8 with task 3093249f-f5b9-4e2c-8501-9b41924cd6db
INFO:api_app.connectors_manager.classes:STARTED connector: (OpenCTI, job: #8)
INFO:api_app.connectors_manager.classes:Running connector OpenCTI even if job status is reported_with_fails becauserun on failure is set
INFO:pycti.entities:Listing Threat-Actors with filters null.
ERROR:pycti.api:Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?
ERROR:pycti.api:{'name': 'Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?', 'message': 'Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?'}
Traceback (most recent call last):
  File "/opt/deploy/intel_owl/api_app/classes.py", line 190, in start
    _result = self.run()
  File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 110, in run
    self.opencti_instance = pycti.OpenCTIApiClient(
  File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 217, in __init__
    raise ValueError(
ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
ERROR:api_app.classes:(OpenCTI, job: #8). Unexpected error: 'OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...'
Traceback (most recent call last):
  File "/opt/deploy/intel_owl/api_app/classes.py", line 190, in start
    _result = self.run()
  File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 110, in run
    self.opencti_instance = pycti.OpenCTIApiClient(
  File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 217, in __init__
    raise ValueError(
ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
INFO:api_app.connectors_manager.classes:FINISHED connector: (OpenCTI, job: #8)
INFO:celery.app.trace:Task run_plugin[3093249f-f5b9-4e2c-8501-9b41924cd6db] succeeded in 0.09238294186070561s: None

Testing on the same system with pycti:

root@intelowl:/opt/pycti# python3 test.py
INFO:api:Health check (me)...
INFO:api:Creating Identity
INFO:api:Creating Identity
INFO:api:Creating stix_core_relationship
{'id': 'dcb77b4f-5ddc-45d6-8fd8-9d139bedc817', 'standard_id': 'relationship--65098457-d570-4bdf-9cb3-831e17ef125f', 'entity_type': 'part-of', 'parent_types': ['basic-relationship', 'stix-relationship', 'stix-core-relationship'], 'createdById': None}

pycti-version 5.12.14

Environment

OS: Ubuntu 22.04 LTS
IntelOwl version: v5.2.3
YETI: 2.1.3

What did you expect to happen

Write IOCs to OPENCTI
How to reproduce your issue
Error messages and logs
"OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration..."

[mlodic]

hey!

some things:

• the health check does verify only that the service is UP via a HEAD request, it serves as a way to tell you that it can connect to it but if the integration is broken, it still is.
• we have received several other reports of problems with OpenCTI integration. Because of that we have a specific section of the documentation which explains the problem and how to solve it: https://intelowl.readthedocs.io/en/latest/Advanced-Configuration.html#opencti

[CyberAbwehr]

Still the same issue. ;-)

root@intelowl:/opt/IntelOwl# ./start test build --pycti-version 5.12.14
grep: docker/.env.start.test: No such file or directory
./start: line 92: export: `': not a valid identifier
WARN[0000] The "REPO_DOWNLOADER_ENABLED" variable is not set. Defaulting to a blank string.
[+] Building 14.0s (27/27) FINISHED docker:default
=> [uwsgi internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 3.31kB 0.0s
=> [uwsgi internal] load .dockerignore 0.0s
=> => transferring context: 306B 0.0s
=> [uwsgi internal] load metadata for docker.io/library/python:3.9.16 1.1s
=> [uwsgi internal] load metadata for docker.io/library/node:lts-alpine3.18 1.1s
=> [uwsgi backend-build 1/10] FROM docker.io/library/python:3.9.16@sha256:603ac689b89c2a59791a4e7cd3d727f2a673ac3df02dabbd97b0d85bb1eca4e7 0.0s
=> [uwsgi internal] load build context 0.1s
=> => transferring context: 861.58kB 0.1s
=> [uwsgi frontend-build 1/5] FROM docker.io/library/node:lts-alpine3.18@sha256:3aae0ea51b2952660b4b65988963b78b269cf84cc7f36f208462601a12e1531a 0.0s
=> CACHED [uwsgi frontend-build 2/5] COPY frontend/ . 0.0s
=> CACHED [uwsgi frontend-build 3/5] COPY docker/.env .env.local 0.0s
=> CACHED [uwsgi frontend-build 4/5] RUN npm install npm@latest --location=global && npm install && PUBLIC_URL=/static/reactapp/ npm run build 0.0s
=> CACHED [uwsgi backend-build 2/10] RUN mkdir -p /var/log/intel_owl /var/log/intel_owl/django /var/log/intel_owl/uwsgi /opt/deploy/files 0.0s
=> CACHED [uwsgi backend-build 3/10] RUN apt-get update && apt-get install -y --no-install-recommends apt-utils libsasl2-dev libssl-dev netcat 0.0s
=> CACHED [uwsgi backend-build 4/10] RUN cpan -T Email::Outlook::Message 0.0s
=> CACHED [uwsgi backend-build 5/10] COPY requirements/project-requirements.txt /opt/deploy/intel_owl/project-requirements.txt 0.0s
=> CACHED [uwsgi backend-build 6/10] COPY requirements/certego-requirements.txt /opt/deploy/intel_owl/certego-requirements.txt 0.0s
=> CACHED [uwsgi backend-build 7/10] WORKDIR /opt/deploy/intel_owl 0.0s
=> CACHED [uwsgi backend-build 8/10] RUN pip3 install --no-cache-dir --compile -r project-requirements.txt && pip3 install --no-cache-dir elastic 0.0s
=> [uwsgi backend-build 9/10] COPY . /opt/deploy/intel_owl 0.2s
=> [uwsgi backend-build 10/10] RUN touch /var/log/intel_owl/django/api_app.log /var/log/intel_owl/django/api_app_errors.log && touch /var/log/int 11.1s
=> [uwsgi stage-2 1/1] COPY --from=frontend-build /build /var/www/reactapp 0.2s
=> [uwsgi] exporting to image 0.3s
=> => exporting layers 0.3s
=> => writing image sha256:83611a2a4ef85aece736e1773406c0f1df54c68831fc35702f1edbd64370fe59 0.0s
=> => naming to docker.io/intelowlproject/intelowl:test 0.0s
=> [nginx internal] load .dockerignore 0.0s
=> => transferring context: 306B 0.0s
=> [nginx internal] load build definition from Dockerfile_nginx 0.0s
=> => transferring dockerfile: 430B 0.0s
=> [nginx internal] load metadata for docker.io/library/nginx:1.25-alpine 1.1s
=> [nginx 1/2] FROM docker.io/library/nginx:1.25-alpine@sha256:f2802c2a9d09c7aa3ace27445dfc5656ff24355da28e7b958074a0111e3fc076 0.0s
=> CACHED [nginx 2/2] RUN rm /var/log/nginx/access.log /var/log/nginx/error.log && touch /var/log/nginx/access.log /var/log/nginx/error.log 0.0s
=> [nginx] exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:7bf491e8ac1fccd5216fde650de1fb53e97baa10e68b890c2c9c77bf6ae97e4f 0.0s
=> => naming to docker.io/intelowlproject/intelowl_nginx:test 0.0s

intelowl_celery_worker_default | INFO:pycti.entities:Listing Threat-Actors with filters null.
intelowl_celery_worker_default | ERROR:pycti.api:Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?
intelowl_celery_worker_default | ERROR:pycti.api:{'name': 'Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?', 'message': 'Unknown type "ThreatActorsFiltering". Did you mean "ThreatActorsOrdering", "ThreatActorConnection", "ThreatActorEdge", "ThreatActorGroup", or "ThreatActorGroupEdge"?'}
intelowl_celery_worker_default | Traceback (most recent call last):
intelowl_celery_worker_default | File "/opt/deploy/intel_owl/api_app/classes.py", line 190, in start
intelowl_celery_worker_default | _result = self.run()
intelowl_celery_worker_default | File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 110, in run
intelowl_celery_worker_default | self.opencti_instance = pycti.OpenCTIApiClient(
intelowl_celery_worker_default | File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 217, in init
intelowl_celery_worker_default | raise ValueError(
intelowl_celery_worker_default | ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
intelowl_celery_worker_default | ERROR:api_app.classes:(OpenCTI, job: #10). Unexpected error: 'OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...'
intelowl_celery_worker_default | Traceback (most recent call last):
intelowl_celery_worker_default | File "/opt/deploy/intel_owl/api_app/classes.py", line 190, in start
intelowl_celery_worker_default | _result = self.run()
intelowl_celery_worker_default | File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 110, in run
intelowl_celery_worker_default | self.opencti_instance = pycti.OpenCTIApiClient(
intelowl_celery_worker_default | File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 217, in init
intelowl_celery_worker_default | raise ValueError(
intelowl_celery_worker_default | ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
intelowl_celery_worker_default | INFO:api_app.connectors_manager.classes:FINISHED connector: (OpenCTI, job: #10)
intelowl_celery_worker_default | INFO:celery.app.trace:Task run_plugin[675c63cd-64c8-4ba9-964a-df80f0a39385] succeeded in 0.22445168299600482s: None
intelowl_celery_worker_default | INFO:api_app.models:[STARTING] set_final_status for <-- Job(#10, "145.239.253.78").
intelowl_celery_worker_default | INFO:api_app.models:[REPORT] Job(#10, "145.239.253.78"), status:connectors_completed, reports:{'success': 1, 'failed': 1, 'all': 2, 'pending': 0, 'killed': 0, 'running': 0}
intelowl_celery_worker_default | INFO:api_app.models:<Job: Job(#10, "145.239.253.78")> setting status to reported_with_fails
intelowl_celery_worker_default | INFO:celery.app.trace:Task job_set_final_status[87f22d00-6ec2-4d59-af27-b8fb9d807839] succeeded in 0.008970469236373901s: None
intelowl_celery_worker_default | INFO:celery.app.trace:Task intel_owl.tasks.update[48c6357c-8dc0-421c-95de-df81b0ab0bf8] succeeded in 0.00442303204908967s: None

[mlodic]

ouch. I don't know so, it seems this connector must require additional testing....similar problem was arise here: #1730....

Are you sure to have properly set your SSL certificate, you URL and your token?

[CyberAbwehr]

Secrets
Connector - OpenCTI - url_key_name - http://192.168.1.43:8080
Connector - OpenCTI - api_key_name - ************************

Parameters
Connector - OpenCTI - ssl_verify - false

[CyberAbwehr]

Cant find the value "ThreatActorsFiltering"

[mlodic]

I don't have an OpenCTI instance available to do some tests, if you could help me to find which is the problem...

Then, this is something not referred by our connector so it is likely a bug of the pycti library itself instead of a bug of the IntelOwl's connector

[CyberAbwehr]

I can help you, I have following instances running: IntelOWL DFIR-IRIS MISP YETI OpenCTI Shuffle OpenSearch And if you need a other one let me know. Matteo Lodi ***@***.***> schrieb am Fr. 9. Feb. 2024 um 15:45:

…

I don't have an OpenCTI instance available to do some tests, if you could help me to find which is the problem... Then, this is something not referred by our connector so it is likely a bug of the pycti library itself instead of a bug of the IntelOwl's connector — Reply to this email directly, view it on GitHub <#2122 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMG3C3KDJYM7HYQSMK555BTYSYZBXAVCNFSM6AAAAABDBDAB42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWGA3DANBTHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mlodic]

do you still have problems with opencti?

[CyberAbwehr]

Need to update my systems, after that i will check.

[mlodic]

you are talking about another project...