Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarifying data/state tables from event tables #304

Closed
frgfm opened this issue Oct 19, 2023 · 4 comments
Closed

Clarifying data/state tables from event tables #304

frgfm opened this issue Oct 19, 2023 · 4 comments
Labels
help wanted Extra attention is needed type: improvement New feature or request
Milestone

Comments

@frgfm
Copy link
Member

frgfm commented Oct 19, 2023

I've been thinking about the relevance of some of the early tables we created. I identify the core purpose of the backend as:

  • identifying start & end of wildfires events
  • provide live monitoring during the wildfire (access latest images/prediction)

Now think about the tables & columns involved in each stage:

  • each device needs to be registered in the API (access or device table) with each location & orientation (potentially it's site)
  • when a device detects a wildfire (locally a buffer/confirmation mechanism is possible), using its authentication, it needs to send the image. To simplify, I'd argue that if the device can't send the image, it can be used for training/data collection later, but it's no use for firefighters.
  • on the backend, the image is uploaded to S3 & generates a URL for it. It creates an event at the location of the device, using the current timestamp
  • for monitoring purposes, in the device table, each device should have last_ping, and last_image (url to S3)

Now since we have a lot of noise between events & alerts, I suggest considering the following table with their columns:

  • groups, accesses, users & sites (rename stations?): same as before
  • devices: id, access_id, login, group_id, current_site_id, last_ping, last_image_bucket_key, lat+lon+elevation+azimuth+angle of view
  • alerts: id, device_id, image_bucket_key, detection_localization, created_at (the device gives lat+lon+azimuth & cie)
    Finally, I think we need to rework the event table and simplify it considering that it only aggregates the alerts from a single device (i.e. if we have 4 devices on a site, we can have 4 events at the same time):
  • events: id, device_id, type, first_alert, last_alert, created_at (the last_alert gets updated by detections, and type by firefighters)

(we remove the legacy media, installations & notifications tables. and I think we should merge webhooks & recipients)

In order to see more clearly, I can set an APM like PostHog to be able to explore events. That should help us figure out which tables are still relevant afterward.

What do you think?

@frgfm frgfm added type: improvement New feature or request help wanted Extra attention is needed labels Oct 19, 2023
@frgfm frgfm added this to the 0.2.0 milestone Oct 19, 2023
@frgfm
Copy link
Member Author

frgfm commented Nov 12, 2023

Also, we definitely need to add authentication requirements on notifications & recipients routes if we keep them 😅

@blenzi
Copy link

blenzi commented Nov 12, 2023

Both are protected for admin scope. You probably didn't notice here and here.

image

@frgfm
Copy link
Member Author

frgfm commented Nov 12, 2023

Oh I didn't see the router-level dependency, my bad 👍

@frgfm
Copy link
Member Author

frgfm commented Jan 17, 2025

Closed by #412 & #405 & #340

@frgfm frgfm closed this as completed Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed type: improvement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants