Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Docker Compose configuration entirely with environment variables #282

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
229281a
Use env var SSD_DEVICE to pass SSD to container
aromanielloNTIA Mar 22, 2024
ee3b311
Add default SSD_DEVICE=/dev/nvme0n1
aromanielloNTIA Mar 22, 2024
645b034
Configure API container shared memory size with env var
aromanielloNTIA Mar 22, 2024
027b572
remove unnecessary quotes
aromanielloNTIA Mar 22, 2024
4814833
add SSD_DEVICE and API_SHM_SIZE to README
aromanielloNTIA Mar 22, 2024
d3c541b
remove testing changes
aromanielloNTIA Mar 22, 2024
c795c98
remove redundant environment variable setting
aromanielloNTIA Mar 22, 2024
d62ed2d
Merge branch 'master' into all-config-with-env
aromanielloNTIA May 10, 2024
340588c
Automatically get SSD_DEVICE
aromanielloNTIA May 10, 2024
5aa2503
Merge branch 'all-config-with-env' of https://github.com/ntia/scos-se…
aromanielloNTIA May 10, 2024
5875313
Fix SSD_DEVICE default for github actions runner
aromanielloNTIA May 10, 2024
d804d6d
Use env var SSD_DEVICE to pass SSD to container
aromanielloNTIA Mar 22, 2024
c01814a
Add default SSD_DEVICE=/dev/nvme0n1
aromanielloNTIA Mar 22, 2024
e4f343d
Configure API container shared memory size with env var
aromanielloNTIA Mar 22, 2024
80e4bd1
remove unnecessary quotes
aromanielloNTIA Mar 22, 2024
4e55d69
Automatically get SSD_DEVICE
aromanielloNTIA May 10, 2024
eb178f7
add SSD_DEVICE and API_SHM_SIZE to README
aromanielloNTIA Mar 22, 2024
b9c5bf9
remove testing changes
aromanielloNTIA Mar 22, 2024
e29e015
remove redundant environment variable setting
aromanielloNTIA Mar 22, 2024
e6fd715
Fix SSD_DEVICE default for github actions runner
aromanielloNTIA May 10, 2024
aca1017
merge upstream
aromanielloNTIA Jul 25, 2024
82bc5ed
Fix github action
aromanielloNTIA Jul 25, 2024
df2e1dc
Merge branch 'master' into all-config-with-env
aromanielloNTIA Oct 17, 2024
37cc54c
Use "docker compose" in CI
aromanielloNTIA Oct 17, 2024
2f26795
Merge branch 'master' into all-config-with-env
aromanielloNTIA Nov 19, 2024
58612f6
Merge branch 'master' of https://github.com/NTIA/scos-sensor into all…
jhazentia Dec 13, 2024
dab8607
Re-add mistakenly removed env variable
aromanielloNTIA Dec 13, 2024
cc553d0
Update scos-tekrsa branch for testing
aromanielloNTIA Dec 13, 2024
16b12dd
Do not dynamically generate SSD_DEVICE default
aromanielloNTIA Dec 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/github-actions-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ jobs:
source ./env.template
export MOCK_SIGAN=1
export MOCK_SIGAN_RANDOM=1
export SSD_DEVICE=/dev/nvme0n1
docker compose build --no-cache
docker compose up -d

Expand Down
5 changes: 2 additions & 3 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ services:
interval: 6s
timeout: 3s
retries: 1
shm_size: '16gb'
shm_size: ${API_SHM_SIZE}
restart: always
depends_on:
db:
Expand Down Expand Up @@ -80,7 +80,6 @@ services:
- STARTUP_CALIBRATION_ACTION
- RAY_INIT
- RUNNING_MIGRATIONS
- USB_DEVICE
aromanielloNTIA marked this conversation as resolved.
Show resolved Hide resolved
expose:
- '8000'
volumes:
Expand All @@ -95,7 +94,7 @@ services:
- SYS_ADMIN # required for ^ with NVMe drives
devices:
- /dev/bus/usb:/dev/bus/usb:rw
- /dev/nvme0n1:/dev/nvme0n1:ro
- ${SSD_DEVICE}:/dev/nvme0n1:ro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this would work if the SSD_DEVICE is a SATA device which is then mapped to an nvme device in the container? Perhaps it would be better to use ${SSD_DEVICE} for both then pass the SSD_DEVICE environment variable to scos-actions to get the path?

extra_hosts:
- "${MANAGER_FQDN}:${MANAGER_IP}"
logging:
Expand Down
21 changes: 15 additions & 6 deletions env.template
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

# Copy env.template to env and modify


# Mark all the following variables for export
set -o allexport

#Any names here will be added as additional users with the
#specified additional user password
# Any names here will be added as additional users with the
# specified additional user password
ADDITIONAL_USER_NAMES="" # comma separated
ADDITIONAL_USER_PASSWORD=""

Expand All @@ -18,15 +17,15 @@ ADMIN_PASSWORD=password

# set to CERT to enable scos-sensor certificate authentication
AUTHENTICATION=TOKEN
CALIBRATION_EXPIRATION_LIMIT=360

# Default callback api/results
# Set to CERT for certificate authentication
CALLBACK_AUTHENTICATION=TOKEN
# Set to false to disable SSL cert verification in the callback POST request
CALLBACK_SSL_VERIFICATION=false

#Set the number of seconds before timeout in postback when a scheduled
#action completes
# Set the number of seconds before timeout in postback when a scheduled
# action completes
CALLBACK_TIMEOUT=2

# SECURITY WARNING: don't run with debug turned on in production!
Expand Down Expand Up @@ -89,12 +88,22 @@ SSL_CA_PATH=scos_test_ca.crt
SSL_CERT_PATH=sensor01.pem
SSL_KEY_PATH=sensor01.pem

# Device made available to SCOS Plugins, e.g. for getting SMART data
SSD_DEVICE="$(lsblk -I 8 -npdo KNAME)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command isn't returning anything on the sensors or my laptop

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with a plaintext default value which can be changed when configuring a sensor.

Note also that I made changes in scos-actions (NTIA/scos-actions#126) which will result in a default value of /dev/nvme0n1 being used by the SEA action if the environment variable is not provided.


# Size of shared memory for API container (/dev/shm), used for parallel processing
# See https://docs.docker.com/compose/compose-file/compose-file-v3/#shm_size
API_SHM_SIZE=16gb

# Calibration action selection
# The action specified here will be used to attempt an onboard
# sensor calibration on startup, if no onboard calibration data
# is available on startup. The specified action must be available.
STARTUP_CALIBRATION_ACTION=SEA_CBRS_Calibrate_Baseline

# How many seconds until calibration data is considered stale
CALIBRATION_EXPIRATION_LIMIT=3600
aromanielloNTIA marked this conversation as resolved.
Show resolved Hide resolved

# Debug dependant settings
if $DEBUG; then
GUNICORN_LOG_LEVEL=debug
Expand Down