Generate log files (cockpit standard and video subtitles) #582

rafaellehmkuhl · 2023-11-21T17:29:22Z

~~To be merged after #573.~~

ES-Alexander · 2023-11-23T03:59:27Z

src/libs/logging.ts

+    const roll = Number(logPoint.data['roll']).toFixed(2)
+    const pitch = Number(logPoint.data['pitch']).toFixed(2)
+    const heading = Number(logPoint.data['heading']).toFixed(2)
+    const depth = Number(logPoint.data['depth']).toFixed(2)
+    const mode = logPoint.data['mode']
+    const batteryVoltage = Number(logPoint.data['batteryVoltage']).toFixed(2)
+    const batteryCurrent = Number(logPoint.data['batteryCurrent']).toFixed(2)
+    const gpsVisibleSatellites = logPoint.data['gpsVisibleSatellites']
+    const gpsFixType = logPoint.data['gpsFixType']
+    const latitude = Number(logPoint.data['latitude']).toFixed(6)
+    const longitude = Number(logPoint.data['longitude']).toFixed(6)
+    const secondsStart = logPoint.seconds.toFixed(0)
+    const secondsFinish = (logPoint.seconds + 1).toFixed(0)


Can we include units for these somehow? The indicators already include units, so that seems like it should be doable, hopefully :-)

relevant comment

I will try the discussed approach. If it fits, I will change the PR.

On the new approach, each widget register which variables it uses, and this information is used for automatic variable selection (if no user custom selection is made).

src/components/mini-widgets/MiniVideoRecorder.vue

src/libs/logging.ts

patrickelectric · 2023-11-27T15:37:46Z

src/libs/logging.ts

+/**
+ * Variables data can be datalogged
+ */
+export enum DatalogVariable {


Cant we use registerUsage to do this kind of definition for us dynamically ?

If we do that, we lose the ability to add to the logs variables that are not being used by widgets.

So.. why not having the log being done by the abstract vehicle class over registerUsage being used over widgets if the idea is to log everything ?

Logs should be independent from video subtitles.

Logs should be independent from video subtitles.

They are. It's the other way around. The subtitles are dependent on the logs.

So.. why not having the log being done by the abstract vehicle class over registerUsage being used over widgets if the idea is to log everything ?

The registerUsage approach is just to deliver a good default to users. If they never configured which variables to show in the subtitle overlays, the default ones will be the ones they are using.

joaoantoniocardoso

I was imagining that all the widgets would provide their data, and a class would record them when registered.

With the current design, I don't understand how, for example, arbitrary data would be recorded, like from a custom sensor received as a float value, or anything from a generic indicator.

rafaellehmkuhl · 2023-11-27T18:05:44Z

I was imagining that all the widgets would provide their data, and a class would record them when registered.

With the current design, I don't understand how, for example, arbitrary data would be recorded, like from a custom sensor received as a float value, or anything from a generic indicator.

We indeed do not support the use-case you mentioned in this first patch.

The problem with having the widgets registering what to record as a solo approach is that we wouldn't be able to record things that no widget is registering. Remember the widgets are lazy-loaded, so if the widget is not in any view, the variable will not be registered and them the user won't be able to use it.

What we could do is to actually do both: have the pre-registered variables and allow widgets to also provide their own, in a custom way.

… subtitles log

…rors

rafaellehmkuhl · 2023-11-27T19:41:47Z

@joaoantoniocardoso I'm testing the approach I mentioned and the place where it fit's more is indeed the VeryGenericIndicator, but for it to make sense I need to add extra UI on the widget to allow the user to manually register the variable (clicking a button) when it decides on a good widget configuration (otherwise we end up with garbage in the log).

As I still need to finish the joystick pipeline for this week, do you think it's reasonable if we go with this PR as is and open a an issue and follow with a PR next week adding support for the generic from-wiget registering?

joaoantoniocardoso

Tested :)

For the docs, we should mention that the browser will ask the user for permission to download multiple files at once.

rafaellehmkuhl · 2023-11-27T23:08:53Z

Video of the final working version:

Screen.Recording.2023-11-27.at.20.05.32.mp4

ES-Alexander · 2023-12-04T21:22:51Z

Docs key points:

Currently a pre-defined set of variables being logged
Raw log of inputs/outputs being logged automatically at 1Hz on the control station
Subtitle file created when recording a video, by slicing the raw log from the start:end times of the video, formatted as a subtitle file
By default logs variables in active widgets, but not the ambiguous ones (like VeryGenericIndicator)
- It's possible to override the variables by going to the logs configuration page (need to document that image too)
Currently output format determined automatically (TODO: create issue on desired interface, with grid and draggable elements)

joaoantoniocardoso · 2023-12-04T23:21:20Z

Docs key points:

* Currently a pre-defined set of variables being logged

* Raw log of inputs/outputs being logged automatically at 1Hz on the control station

* Subtitle file created when recording a video, by slicing the raw log from the start:end times of the video, formatted as a subtitle file

* By default logs variables in active widgets, but not the ambiguous ones (like VeryGenericIndicator)
  
  * It's possible to override the variables by going to the logs configuration page (need to document that image too)

* Currently output format determined automatically (TODO: create issue on desired interface, with grid and draggable elements)

@ES-Alexander I'd advocate to include something like:

Multiple files will be downloaded (subtitle + video) when recording is finished, so the browser will ask the user for permission to download multiple files, and the user must accept it to use this feature.

Documents bluerobotics/cockpit#582

rafaellehmkuhl changed the title ~~Generate log files~~ Generate log files (cockpit standard and video subtitles) Nov 21, 2023

rafaellehmkuhl force-pushed the generate-log-files branch 5 times, most recently from 2b83c6c to b15506e Compare November 22, 2023 20:23

ES-Alexander added the docs-needed Change needs to be documented label Nov 22, 2023

rafaellehmkuhl force-pushed the generate-log-files branch 3 times, most recently from 30bb006 to f7718a7 Compare November 23, 2023 01:21

ES-Alexander reviewed Nov 23, 2023

View reviewed changes

rafaellehmkuhl force-pushed the generate-log-files branch 3 times, most recently from 6ceca2e to f8c3559 Compare November 24, 2023 03:35

rafaellehmkuhl requested a review from ES-Alexander November 24, 2023 03:35

rafaellehmkuhl marked this pull request as ready for review November 24, 2023 12:20

rafaellehmkuhl requested review from patrickelectric, Williangalvani and joaoantoniocardoso November 24, 2023 12:20

rafaellehmkuhl force-pushed the generate-log-files branch 2 times, most recently from 549093b to 0cdf9c2 Compare November 27, 2023 15:19

patrickelectric requested changes Nov 27, 2023

View reviewed changes

rafaellehmkuhl force-pushed the generate-log-files branch from 0cdf9c2 to e32773c Compare November 27, 2023 17:09

rafaellehmkuhl requested a review from patrickelectric November 27, 2023 17:10

joaoantoniocardoso reviewed Nov 27, 2023

View reviewed changes

rafaellehmkuhl requested a review from joaoantoniocardoso November 27, 2023 18:05

rafaellehmkuhl added 3 commits November 27, 2023 16:10

Create initial logging library

dd9374d

Start data logging on each boot

800e18e

Activate variable log from widgets

32e0470

rafaellehmkuhl added 4 commits November 27, 2023 16:10

Allow creation of .ass overlay telemetry subtitle files from logs

36e3a8b

Create overlay telemetry subtitle on video records

03a2c0a

Add page to allow selection of which variables to show in the overlay…

7be5a7a

… subtitles log

Update Node types to allow usage of structuredClone without type-er…

e3a1dec

…rors

rafaellehmkuhl force-pushed the generate-log-files branch from e32773c to e3a1dec Compare November 27, 2023 19:11

joaoantoniocardoso approved these changes Nov 27, 2023

View reviewed changes

rafaellehmkuhl merged commit bdf4999 into bluerobotics:master Nov 27, 2023
7 checks passed

rafaellehmkuhl deleted the generate-log-files branch November 27, 2023 23:00

This was referenced Dec 5, 2023

Support playback from Cockpit log files + video recordings #619

Open

[Cockpit-devel] updates for first major release beta bluerobotics/ardusub-zola#143

Merged

ES-Alexander added a commit to ES-Alexander/ardusub-zola that referenced this pull request Dec 12, 2023

advanced-usage: add telemetry logging and subtitle generation

28979d0

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to ES-Alexander/ardusub-zola that referenced this pull request Dec 12, 2023

advanced-usage: add telemetry logging and subtitle generation

effd6cb

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to ES-Alexander/ardusub-zola that referenced this pull request Dec 12, 2023

advanced-usage: add telemetry logging and subtitle generation

c848959

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to ES-Alexander/ardusub-zola that referenced this pull request Dec 12, 2023

advanced-usage: add telemetry logging and subtitle generation

e7fb167

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to ES-Alexander/ardusub-zola that referenced this pull request Dec 13, 2023

advanced-usage: add telemetry logging and subtitle generation

03b46d7

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to bluerobotics/ardusub-zola that referenced this pull request Dec 13, 2023

advanced-usage: add telemetry logging and subtitle generation

2eec81e

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to bluerobotics/ardusub-zola that referenced this pull request Dec 13, 2023

advanced-usage: add telemetry logging and subtitle generation

f475d52

Documents bluerobotics/cockpit#582

ES-Alexander added a commit to bluerobotics/ardusub-zola that referenced this pull request Dec 13, 2023

advanced-usage: add telemetry logging and subtitle generation

4c43a39

Documents bluerobotics/cockpit#582

ES-Alexander added docs-complete Change documentation has been completed and removed docs-needed Change needs to be documented labels Dec 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate log files (cockpit standard and video subtitles) #582

Generate log files (cockpit standard and video subtitles) #582

rafaellehmkuhl commented Nov 21, 2023 •

edited

Loading

ES-Alexander Nov 23, 2023

ES-Alexander Nov 23, 2023

rafaellehmkuhl Nov 23, 2023

rafaellehmkuhl Nov 24, 2023

patrickelectric Nov 27, 2023

rafaellehmkuhl Nov 27, 2023

patrickelectric Nov 27, 2023

patrickelectric Nov 27, 2023

rafaellehmkuhl Nov 27, 2023 •

edited

Loading

joaoantoniocardoso left a comment •

edited

Loading

rafaellehmkuhl commented Nov 27, 2023

rafaellehmkuhl commented Nov 27, 2023

joaoantoniocardoso left a comment

rafaellehmkuhl commented Nov 27, 2023

ES-Alexander commented Dec 4, 2023

joaoantoniocardoso commented Dec 4, 2023 •

edited

Loading

Generate log files (cockpit standard and video subtitles) #582

Generate log files (cockpit standard and video subtitles) #582

Conversation

rafaellehmkuhl commented Nov 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rafaellehmkuhl Nov 27, 2023 • edited Loading

Choose a reason for hiding this comment

joaoantoniocardoso left a comment • edited Loading

Choose a reason for hiding this comment

rafaellehmkuhl commented Nov 27, 2023

rafaellehmkuhl commented Nov 27, 2023

joaoantoniocardoso left a comment

Choose a reason for hiding this comment

rafaellehmkuhl commented Nov 27, 2023

ES-Alexander commented Dec 4, 2023

joaoantoniocardoso commented Dec 4, 2023 • edited Loading

rafaellehmkuhl commented Nov 21, 2023 •

edited

Loading

rafaellehmkuhl Nov 27, 2023 •

edited

Loading

joaoantoniocardoso left a comment •

edited

Loading

joaoantoniocardoso commented Dec 4, 2023 •

edited

Loading