Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update create config + 1.0b5 updates #529

Merged
merged 35 commits into from
Sep 23, 2023
Merged

Conversation

maaikelimper
Copy link
Collaborator

New PR with the branch on my test-server, which combines the updates for update-create-config and the changes for 1.0b5 (wis2box-webapp and new wis2box-api from branch).

.gitignore Outdated
docker/.env

.ipynb_checkpoints
wis2box.env
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate?

@@ -4,7 +4,7 @@ services:
- ${WIS2BOX_HOST_DATADIR}:/data/wis2box:rw
- ./wis2box-management/wis2box/wis2box.cron:/etc/cron.d/wis2box:ro
- ./wis2box-management/wis2box:/app/wis2box
command: ["wis2box", "pubsub" , "subscribe", "--broker", "http://wis2box-minio:9000", "--topic", "wis2box-storage/#", "--verbosity", "INFO"]
command: ["wis2box", "pubsub" , "subscribe"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Options no longer required?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the options were mostly not used inside subscribe.py . The topic used for subscription can not be a configurable at this time as it set in other places (such as acl.conf for mosquitto, notification-topic for minio) and subscribing to any other topic than 'wis2box/#' would breake the wis2box-workflow.

@@ -86,7 +85,7 @@ ERROR - Failed to publish, wsi: ..., tsi: XXXXX
-----------------------------------------------

Data arrived for a station that is not present in the station metadata cache.
To add missing stations, update the file ``metadata/station/station_list.csv`` in the wis2box data directory (see :ref:`setup`).
To add missing stations, use the station-editor in wis2box-webapp (from wis2box-1.0b5) or update the file ``metadata/station/station_list.csv`` in the wis2box data directory (see :ref:`setup`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does updating station_list still update the station collection? I know we had a discussion on this but can't remember what the outcome was.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can still use the station_list.csv to do a bulk-insert the station collection, by executing the command 'wis2box metadata station publish-collection', as documented in setup.rst.

@@ -16,11 +16,19 @@ To add a token to the execution of a wis2box process, use the following command:

.. code-block:: bash

wis2box auth add-token --path processes/wis2box myexecutiontoken
wis2box auth add-token --path processes/wis2box
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add that they can specify a token as well?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I can add this to the docs

@@ -345,12 +404,12 @@ def get(ctx, wsi, verbosity):
results = OrderedDict({
'station_name': station['station_name'],
'wigos_station_identifier': station['wigos_station_identifier'],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get? also for the other keys?

Copy link
Collaborator Author

@maaikelimper maaikelimper Sep 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The station-report from pyoscar always returns the same attributes, I agree it looks inconsistent so sure I can add get

@maaikelimper
Copy link
Collaborator Author

Comment to self: add additional sleep before running integration tests in GHA: to help avoid intermittent CI failures

@maaikelimper
Copy link
Collaborator Author

I noted the cronjob for cleaning data was broken and added a comment to use the correct environment variable. I increased the default retention-period for data from 7 to 30.

@tomkralidis tomkralidis merged commit f3d2dd3 into main Sep 23, 2023
3 checks passed
@tomkralidis tomkralidis deleted the update-create-config-1.0b5 branch September 23, 2023 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants