feat: implement captions editor integration

[onyedikachi-david]

/claim #361
Fixes #361

[richiemcilroy]

hey @onyedikachi-david - thanks so much for the submission.

it doesn't seem to load for me, I get:

  thread 'main' panicked at /Users/macbookuser/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5:

  failed to execute command: No such file or directory (os error 2)
  is `cmake` not installed?

  build script failed, must exit now
  stack backtrace:
     0:        0x1029b6c1c - std::backtrace_rs::backtrace::libunwind::trace::h43235ac629484ce8
                                 at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/../../backtrace/src/backtrace/libunwind.rs:116:5
     1:        0x1029b6c1c - std::backtrace_rs::backtrace::trace_unsynchronized::h3bc78969d500b9fc
                                 at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
     2:        0x1029b6c1c - std::sys::backtrace::_print_fmt::h66db558f95871955
                                 at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/sys/backtrace.rs:66:9
     3:        0x1029b6c1c - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6e25c13836847e70

Also, we need the captions to be baked into the video frames that come from the renderer. Would you be able to check?

[onyedikachi-david]

Was the error during the export

[richiemcilroy]

@onyedikachi-david it was on startup of the desktop app

[onyedikachi-david]

@onyedikachi-david it was on startup of the desktop app

I am assuming this, not sure:

                    let status = Command::new("ffmpeg")
                        .args([
                            "-i", &output_path.to_string_lossy(),
                            "-i", &srt_path.to_string_lossy(),
                            "-c:v", "copy",
                            "-c:a", "copy",
                            "-c:s", "mov_text",
                            "-metadata:s:s:0", "language=eng",
                            "-movflags", "+faststart",
                            "-map", "0:v",
                            "-map", "0:a",
                            "-map", "1:0",
                            "-y",
                            &temp_output.to_string_lossy(),
                        ])
                        .status()
                        .map_err(|e| {
                            tracing::error!("FFmpeg command failed: {}", e);
                            format!("Failed to run FFmpeg: {}", e)
                        })?;

[onyedikachi-david]

working quite well here:

Give me some time to find out what is wrong

[richiemcilroy]

right now you are:

• in the editor, rendering captions on top of canvas with html
• in the export, adding a subtitle track to the mp4

we need the subtitles to be rendered in the GPU as part of the actual frame. Not a subtitle track, but a physical implementation in the frame. This would allow us to change the text size, text appearance etc in the editor.

You can use a crate like glyphon to help with this.

[onyedikachi-david]

right now you are:

• in the editor, rendering captions on top of canvas with html
• in the export, adding a subtitle track to the mp4

we need the subtitles to be rendered in the GPU as part of the actual frame. Not a subtitle track, but a physical implementation in the frame. This would allow us to change the text size, text appearance etc in the editor.

You can use a crate like glyphon to help with this.

Done, please review. Was a tough one.

[onyedikachi-david]

Screen.Recording.2025-03-19.at.10.46.32.mp4

@richiemcilroy Here is a demo