Skip to content
This repository has been archived by the owner on Feb 2, 2022. It is now read-only.

How to...

Jump to bottom
intrasoft-rmalik edited this page Nov 14, 2019 · 6 revisions

Check if CSP local node is correctly installed

This is a checklist to verify the connectivity and proper state of a new CSP local node installation. These checks are carried out between the local, central, and debug nodes. They cannot be carried out on just the local node.

First, verify that the network/server/application configuration is correct.

  1. Run a docker stats to verify that all the services are up

docker stats --all --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}" --no-stream | sort

  1. Check if the integration layers can communicate with each other. Send the following curl from debug to local, from central to local, and from local to debug.

cd /opt/csp/apache2/ssl/server

curl --cacert ../ca/common-external-ca.crt --key csp-external.key --cert csp-external.crt https://integration.debug.env.domain:5443/apiversion/

NB: Remember to use the correct target URL.

  1. Log in to central.env, add new local CSP to CSP_ALL and nuke
    1. After adding the local CSP to CSP_ALL, click the nuke button
    2. Verify that the local CSP received the new team (debug) and that CSP_ALL and CSP_SHARING central trust circles are synced with central.env
  2. Set up LTC::CSP_SHARING for this run
    1. Edit LTC::CSP_SHARING in the local CSP and add only the new team and debug
    2. Do the same in the debug.env CMM(TC)
    3. When the two teams exist in each others LTC::CSP_SHARING, then all automatic sharing will be exclusively between those two CSPs without spamming other CSP nodes
  3. Configure sharing policies to allow sharing of contacts
    1. In the local CSP go to integration-ui and put “share as is” in the “contact” datatype (this is needed for the CMM test cases)
    2. The datatypes “event”, “threat”, “incident” should also be “share as is”
  4. Check if SMTP config is working
    1. Check if the local CSP received any regular reports in the mailbox of the user that was specified as sender in the CSP SMTP configuration. _This is an indication that the services can send/receive emails_
  5. Check MISP configuration
    1. Make sure that the local CSP configured MSIP according to the latest installation manual version. This includes checking configuration values such as MISP.org
    2. Verify that the integration user is present
    3. Ensure that SSO works and at least one oAM user has a corresponding MISP user

Having passed the above checks, the test cases can be carried out.

Tips

  • Do the first negative test case in IntelMQ at the very end
    • The test case breaks IntelMQ and you want to avoid trying to fix it before you run the rest of the test cases
    • Remember to fix the broken IntelMQ configuration
  • When you have to verify that an event/message arrived successfully at your end, then you can share your own screen with the other party for validation.
  • If both parties are using Windows, then you can join the conference that was created during the VCB test cases without having to exit the current video conference.
  • In Chrome, if the browser has not been set up to access the MeliCERTes CA certificate, then:
    • The other party will join the videoconference and the main screen (iframe) will show an error message that https://vc.cert.preprod... is down. The error is due to the unknown certificate warning and fails due to the iframe.
    • The other party should copy the link that is showing in the middle of the screen (the one that starts with https://vc....) and paste it on a new tab. Then they can bypass the warning and it should take them to a bare Jitsi. They can close this tab and refresh the previous one (the normal videoconference one). The iframe will work now.

Reset local CSP VM to a clean state

WARNING: This procedure will reset your local CSP and erase all data
The procedure documented below will reset the CSP VM to its initial clean state. There will be a total loss of all data. Please be very careful if and when to execute these instructions.​

Troubleshooting Checks

The following prerequisites are required for the procedure to be completed

Prerequisites

Required infrastructure/feature Yes/No
Internet connection​ Yes
Command line (terminal) access Yes

Required inputs Input from supported local CSP:

  1. /tmp/spring.log

Resetting CSP

The following commands should be executed to reset:

Execute at CSP VM prompt
docker stop `docker ps -q` && docker kill `docker ps -q` && docker rm `docker ps -q -a`
docker volume rm `docker volume ls -q`
docker rmi `docker images -q` && docker system prune -f && docker volume prune -f && docker network prune -f
rm -fr /opt/cspinst/.clientconf* /opt/csp/* /root/*conf /root/*env
reboot.​

Change domain of local CSP

Prerequisites

Required infrastructure/feature Yes/No
Internet connection​ Yes
Command line (terminal) access Yes
New certificates from PKI team for local and central CSP Yes
CA chain as received from the PKI service Yes
New IP addresses (optionally) for location change Yes

Required inputs Input from central CSP:

  1. content of volumes DATA and ROOT (for the installer)
  2. content of volume CONFIG (for the configuration service) Input from local CSP:
  3. content of /opt/cspinst/.clientconf* (2 hidden files, for the installer)

Solution

Overview
The following steps must be carried out sequentially to change the domain of the local CSP
Step 1 - correcting the CSP domain and IP addresses

  1. in CENTRAL, CSP node information should be modified with respect to domain name and IP addresses recorded
  2. in CENTRAL, CSP installer information should be modified with respect to domain name and IP addresses recorded.
  3. in CSP Node(s), CSP installer information should be modified with respect to domain name and IP addresses recorded.

Step 2 - Update the DNS with the new DNS names and entries for the new domain

Step 3 - updating the received databases to the various locations

Step 4 - recreating internal and external certificates and distributing them

Step 5 - reinstalling modules

Step 6 - registering vhosts and agents for OpenAM

Step 7 - starting CSP

For every step, there is an 'Output' section that contains information for verification that the step has been carried without any errors.

Step 1

In this step, the support team requests the 3 different databases for local update and inspection. This step verifies that configuration stored in databases is updated.

Output: 3 databases updated with correct information, produced by support team and emailed to the CENTRAL admins

Step 2

In this step, the support team and CENTRAL administration create the DNS entries for the new domain. CENTRAL and CSP Node names should be created/updated with the new IP addresses.

Network configuration should be correct at this point; the connectivity via firewall whitelisting must be verified.

Output: no output

Step 3

The CENTRAL team should update the databases found in the various locations with the new updated DBs as received by the support team.

To replace the databases, installer and configuration service must be stopped.

  • For the CSP Node, rename 'cspinstaller.jar' to 'cspinstaller.jar.old' and kill the installer via kill -15 and copy the files over. Rename back to .jar, and reboot for the installer to start again.
  • For the Central, stop the installer via the docker-compose.yml: docker-compose stop and copy the database. When complete, restart the central config and installer using the opposite operation as per the manual.

Output: list of volume directories with changed files should be provided - installer on VM should be running and installer and configuration service on CENTRAL should be running

Step 4

To recreate the certificates on CENTRAL and CSP Node, the following steps must be done:

CSP Node For the CSP node, the Installer should be running. Please verify that the installer used is latest version (currently 4.0.7 commitId 3427f27). You can verify the installer version by hovering over the main page, top-left, on the installer logo icon. A tooltip should appear with the installer version. The commitId appears after the SNAPSHOT/.

The following files should be copied to the /opt/csp/downloads directory:

  1. ca-bundle.crt <-- CA file, you should obtain by downloading https://pki.dfn-cert.de/melicertes-ca/pub/cacert/chain.txt and removing the 1st line(important step)
  2. sslprivate.key <-- private key (should have this)
  3. sslpublic.crt <-- public key (should have received this, signed by the PKI service)

The files should be placed in /opt/csp/downloads. Verify the names and file content.

Note: these files should be for the new domain, not the old one!

When the files are in place, a system call to the CSP Installer, from the command line, should be placed:

curl -v http://localhost:18080/regenerateEnv

This step will generate 3 background jobs, which will read the updated information and:

  1. Setup new internal certificates for the new domain
  2. Setup new external certificates
  3. Recreate the common environment variables on the /root folder. The jobs should complete within the next 1-2 minutes, you should be monitoring logs for completion.

Output: the content of /tmp/spring.log and file /root/common.0.env should be sent to the central team for verification of certificate generation.

CENTRAL setup For the CENTRAL node, the Installer should be running. Please verify that the installer used is latest version. You can verify the installer version by hovering over the main page, top-left, on the installer logo icon. A tooltip should appear with the installer version. The commitId appears after the SNAPSHOT/.

In CENTRAL, the installer is running in a docker container. For the process to work, the certificates should be copied in the /opt/csp/downloads folder within the docker container, not the /opt/csp/downloads of the VM!

To do so, you will need the following sub-steps:

  1. copy the files to the ROOT volume,
  2. exec in the container by executing the command docker exec -ti csp-sa_cfg_client bash
  3. copy the files from /root to the /opt/csp/downloads directory. (rest of the guide for CENTRAL is within the container, not on the outside VM)

For the process, we need **curl **and openssl. These tools are not installed by default in the container, in order to install them execute the following command:

apk update && apk add curl && apk del libressl libressl-dev && apk add openssl

The following files should be copied to the /opt/csp/downloads directory (from the /root folder):

  1. ca-bundle.crt <-- CA file, you should obtain by downloading https://pki.dfn-cert.de/melicertes-ca/pub/cacert/chain.txt and removing the 1st line(important step)
  2. sslprivate.key <-- private key (should have this)
  3. sslpublic.crt <-- public key (should have received this, signed by the PKI service) The files should be placed in /opt/csp/downloads. Verify the names and file content. Note, these files should be for the new domain - not the old one!

When the files are in place, a system call to the CSP Installer, from the command line, should be placed:

curl -v http://localhost:18080/regenerateEnv This step will generate 3 background jobs, which will read the updated information and:

  1. Setup new internal certificates for the new domain
  2. Setup new external certificates
  3. Recreate the common environment variables on the /root folder. The jobs should complete within the next 1-2 minutes, you should be monitoring logs for completion.

Output: the content of /tmp/spring.log and file /root/common.0.env should be sent to the central team for verification of certificate generation.

Step 5

In this step, all modules should be reinstalled by the installer. This is a sequential process, should be executed for both CENTRAL and CSP Node from the 'Updates' screen of the installer.

Prerequisites:

  • Installer is running
  • Services are stopped
  • Updates screen shows modules and actions. In particular it should not show the "stop" action. Use the 'reinstall' button to perform a module delete and install, for each module, EXCEPT the base module (first on the list). This process may take up to 5 minutes per module, but to expedite, you may do CENTRAL and CSP Node in parallel.

Important: OpenAM is the module that has the security configuration of identities. This module needs to be properly initialised.

Removing the OpenAM volumes

Delete the OpenAM volumes:

cd /opt/csp/modules/oam<hashid> docker volume list |grep OAM # 3 volumes should appear docker volume rm # this should happen for all 3 volumes docker volume create # this should happen for all 3 volumes

Removing the Apache volumes

Delete the Apache volume for web agents:

cd /opt/csp/modules/apache<hashid> docker volume list |grep Web docker volume rm docker volume create

Output: the content of files *env and *conf in the /root folder. Note that /root on the CSP Node is on the VM itself, but on the CENTRAL, it is on the ROOT volume. Both outputs should be sent over to the central team for verification.

Step 6

This step applies to both CENTRAL and CSP Node instances

To register again the OpenAM and VHost entries, execute the following command:

curl -v http://localhost:18080/recreateOAMVH

You should receive a HTTP 200 response. Note that in the CENTRAL, this has to be within the docker container not on the outside.

Step 7

For CENTRAL, follow the CENTRAL administration guide to start the CENTRAL CSP.

For the CSP Node, start the CSP instances via "START" button. When the start process is complete, send the central team the logs from /tmp/spring.log.

Verify that the log contains entries with "ACTIONS COMPLETED" (example: fgrep "ACTIONS COMPLETED" /tmp/spring.log)

All entries found should mention OAM : true - APC : true for service <A SERVICE NAME HERE, e.g. misp / rt / intelmq etc>

When this is verified, the CSP should have started and the services should be running and be accessible from their web interfaces.

Clone this wiki locally