Create external csms demo

[Abby-Wheelis]

In alignment with #101 (comment), creating a quick script to highlight to CSMS's that they can test with EVerest!

[Abby-Wheelis]

Testing done:

had regular 1-line demo up, ran before setting url then after

awheelis@sdesk-37031s ~ % curl https://raw.githubusercontent.com/abby-wheelis/everest-demo/create-external-csms-demo/demo-iso15118-2-ocpp-201-ext-csms.sh | bash -s -- -1 -m
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5584  100  5584    0     0  13496      0 --:--:-- --:--:-- --:--:-- 13487
Error: no csms url provided.

set with export CSMS_SP1_URL="url"
awheelis@sdesk-37031s ~ % export CSMS_SP1_URL="ws://host.docker.internal/ws/cp001"
awheelis@sdesk-37031s ~ % curl https://raw.githubusercontent.com/abby-wheelis/everest-demo/create-external-csms-demo/demo-iso15118-2-ocpp-201-ext-csms.sh | bash -s -- -1 -m 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5584  100  5584    0     0   118k      0 --:--:-- --:--:-- --:--:--  118k
DEMO REPO:        https://github.com/everest/everest-demo.git
DEMO BRANCH:      main
DEMO VERSION:     v2.0.1-sp1
DEMO CONFIG:      docker-compose.ocpp201.yml
DEMO DIR:         /var/folders/40/kvb685dn3wd4zz7twz3rbjxw0000gr/T/tmp.tSsUJi54WB
DEMO CSMS:        external
CSMS_SP1_URL:     ws://host.docker.internal/ws/cp001
CSMS_SP2_URL:     
CSMS_SP3_URL:     
Cloning EVerest from https://github.com/everest/everest-demo.git into /var/folders/40/kvb685dn3wd4zz7twz3rbjxw0000gr/T/tmp.tSsUJi54WB/everest-demo
Cloning into 'everest-demo'...
remote: Enumerating objects: 1161, done.
remote: Counting objects: 100% (480/480), done.
remote: Compressing objects: 100% (187/187), done.
remote: Total 1161 (delta 384), reused 294 (delta 293), pack-reused 681 (from 2)
Receiving objects: 100% (1161/1161), 3.02 MiB | 2.47 MiB/s, done.
Resolving deltas: 100% (597/597), done.
/var/folders/40/kvb685dn3wd4zz7twz3rbjxw0000gr/T/tmp.tSsUJi54WB/everest-demo /var/folders/40/kvb685dn3wd4zz7twz3rbjxw0000gr/T/tmp.tSsUJi54WB
API calls to CSMS finished, Starting everest
WARN[0000] /var/folders/40/kvb685dn3wd4zz7twz3rbjxw0000gr/T/tmp.tSsUJi54WB/everest-demo/docker-compose.ocpp201.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Running 3/3
 ✔ Container everest-ac-demo-mqtt-server-1  Healthy                                                                                       0.5s 
 ✔ Container everest-ac-demo-nodered-1      Healthy                                                                                       0.5s 
 ✔ Container everest-ac-demo-manager-1      Healthy                                                                                       0.5s 
Successfully copied 5.63kB to everest-ac-demo-manager-1:/ext/source/config/config-sil-ocpp201-pnc.yaml
rm: cannot remove '/ext/dist/share/everest/modules/OCPP201/component_config/custom/EVSE_2.json': No such file or directory
rm: cannot remove '/ext/dist/share/everest/modules/OCPP201/component_config/custom/Connector_2_1.json': No such file or directory
Configured to SecurityProfile: 1, disabling TLS and configuring server to ws://host.docker.internal/ws/cp001
Successfully copied 2.05kB to everest-ac-demo-manager-1:/tmp/
/ext/source /workspace
patching file config/config-sil-ocpp201-pnc.yaml
Please start the software in the loop simulation manually by running
on your laptop: docker exec -it everest-ac-demo-manager-1 /bin/bash
in the container: sh /ext/build/run-scripts/run-sil-ocpp201-pnc.sh
You can now stop and restart the manager without re-creating the container

saw log error before setting, after setting seemed to work. Able to run manager and charge in UI just as expected!

[Abby-Wheelis]

Wanted to test with something else ... maeve requires me to work out the certificate situation ...
So does citrine.

I don't think I can chase that down tonight, but I am reasonably confident that this script does what we think it does because I was able to set the url to the working maeve container and see an error when it was not set - so the environment variable does what we want it to do.

If I manage to find some time in the morning I'll work through as much of the certificates as I can.

[Abby-Wheelis]

maeve failed:

 => ERROR [manager builder 6/8] RUN go mod download                                                                                       3.4s 
------                                                                                                                                         
 > [manager builder 6/8] RUN go mod download:                                                                                                  
3.331 go: cloud.google.com/[email protected]: Get "https://proxy.golang.org/cloud.google.com/go/@v/v0.111.0.mod": tls: failed to verify certificate: [+] Running 0/1te signed by unknown authority                                                                                                  
 ⠹ Service manager  Building                                                                                                            106.2s 
failed to solve: process "/bin/sh -c go mod download" did not complete successfully: exit code: 1

so did citrene:

 => ERROR [citrine build 5/5] RUN npm rebuild bcrypt --build-from-source && npm rebuild deasync --build-from-source                       4.2s
------
 > [citrine build 5/5] RUN npm rebuild bcrypt --build-from-source && npm rebuild deasync --build-from-source:
4.165 npm error code 1
4.166 npm error path /usr/local/apps/citrineos/node_modules/bcrypt
4.166 npm error command failed
4.166 npm error command sh -c node-pre-gyp install --fallback-to-build
4.166 npm error Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3 --napi_version=9 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
4.166 npm error node-pre-gyp info it worked if it ends with ok
4.166 npm error node-pre-gyp info using [email protected]
4.166 npm error node-pre-gyp info using [email protected] | linux | x64
4.166 npm error node-pre-gyp info build requesting source compile
4.166 npm error gyp info it worked if it ends with ok
4.166 npm error gyp info using [email protected]
4.166 npm error gyp info using [email protected] | linux | x64
4.166 npm error gyp info ok 
4.166 npm error gyp info it worked if it ends with ok
4.166 npm error gyp info using [email protected]
4.166 npm error gyp info using [email protected] | linux | x64
4.166 npm error gyp info find Python using Python version 3.11.2 found at "/usr/bin/python3"
4.166 npm error gyp http GET https://nodejs.org/download/release/v18.20.6/node-v18.20.6-headers.tar.gz
4.166 npm error gyp http fetch GET https://nodejs.org/download/release/v18.20.6/node-v18.20.6-headers.tar.gz attempt 1 failed with SELF_SIGNED_CERT_IN_CHAIN
4.166 npm error gyp WARN install got an error, rolling back install
4.166 npm error gyp ERR! configure error 
4.166 npm error gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v18.20.6/node-v18.20.6-headers.tar.gz failed, reason: self-signed certificate in certificate chain
4.166 npm error gyp ERR! stack at ClientRequest.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)
4.166 npm error gyp ERR! stack at ClientRequest.emit (node:events:517:28)
4.166 npm error gyp ERR! stack at _destroy (node:_http_client:882:13)
4.166 npm error gyp ERR! stack at onSocketNT (node:_http_client:902:5)
4.166 npm error gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
4.166 npm error gyp ERR! System Linux 6.10.14-linuxkit
4.166 npm error gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node" "--module_name=bcrypt_lib" "--module_path=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3" "--napi_version=9" "--node_abi_napi=napi" "--napi_build_version=3" "--node_napi_label=napi-v3"
4.166 npm error gyp ERR! cwd /usr/local/apps/citrineos/node_modules/bcrypt
4.166 npm error gyp ERR! node -v v18.20.6
4.166 npm error gyp ERR! node-gyp -v v10.1.0
4.166 npm error gyp ERR! not ok 
4.166 npm error node-pre-gyp ERR! build error 
4.166 npm error node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/usr/local/apps/citrineos/node_modules/bcrypt/lib/binding/napi-v3 --napi_version=9 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
4.166 npm error node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/usr/local/apps/citrineos/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
4.166 npm error node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:517:28)
4.166 npm error node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1098:16)
4.166 npm error node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:303:5)
4.166 npm error node-pre-gyp ERR! System Linux 6.10.14-linuxkit
4.166 npm error node-pre-gyp ERR! command "/usr/local/bin/node" "/usr/local/apps/citrineos/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
4.166 npm error node-pre-gyp ERR! cwd /usr/local/apps/citrineos/node_modules/bcrypt
4.166 npm error node-pre-gyp ERR! node -v v18.20.6
4.166 npm error node-pre-gyp ERR! node-pre-gyp -v v1.0.11
[+] Running 1/2 node-pre-gyp ERR! not ok
 ✔ Service directus  Built                                                                                                              115.7s 
 ⠇ Service citrine   Building                                                                                                           267.8s 
failed to solve: process "/bin/sh -c npm rebuild bcrypt --build-from-source && npm rebuild deasync --build-from-source" did not complete successfully: exit code: 1

[Abby-Wheelis]

Quick test w/ Cintrene by calling our docker compose, then exporting the same url as the patch:

Getting issues with the nodered container in EVerest which won't start, so wiping and trying again

[Abby-Wheelis]

Error:

 ✔ Network everest-ac-demo_default                                                                                                                        Created                                                         0.1s 
 ✔ Container everest-ac-demo-mqtt-server-1                                                                                                                Started                                                         5.1s 
 ✔ Container everest-ac-demo-manager-1                                                                                                                    Started                                                         1.0s 
 ⠴ Container everest-ac-demo-nodered-1                                                                                                                    Starting                                                        1.0s 
 ! nodered The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested                                                                 0.0s 
Error response from daemon: driver failed programming external connectivity on endpoint everest-ac-demo-nodered-1 (2a2d39b9d72dc05c561bf0e58294ab4a7ca16cd3fff017ab14988687707193e8): Bind for 0.0.0.0:1880 failed: port is already allocated

It looks like citrene and everest are trying to use the same port?

In any case, this is looking like a fail for now, running the manager and OCPP is not connecting to the client:

2025-01-28 17:31:25.061493 [ERRO] ocpp:OCPP201    int ocpp::WebsocketTlsTPM::process_callback(void*, int, void*, void*, size_t) :: CLIENT_CONNECTION_ERROR: conn fail: 111
2025-01-28 17:31:25.061698 [ERRO] ocpp:OCPP201    void ocpp::WebsocketTlsTPM::on_conn_fail() :: OCPP client connection to server failed

[Abby-Wheelis]

Trying MobilityHouse with the EVerest implementation (https://github.com/EVerest/ocpp-csms) and it is also erroring out, complaining about some dependencies:

Traceback (most recent call last):
  File "/Users/awheelis/Documents/ocpp-csms/central_system.py", line 6, in <module>
    from central_systems.central_system_v16 import ChargePoint16
  File "/Users/awheelis/Documents/ocpp-csms/central_systems/central_system_v16.py", line 29, in <module>
    from ocpp.v201.enums import (AuthorizationStatusType, GenericStatusType,
ImportError: cannot import name 'AuthorizationStatusType' from 'ocpp.v201.enums' (/Users/awheelis/Library/Python/3.9/lib/python/site-packages/ocpp/v201/enums.py)

[Abby-Wheelis]

This is because ocpp (directly from MobilityHouse) is a requirement of the EVerest scripts, but not by a tagged version ... so they're out of sync