Transport stream encoding to prevent moov atom hell

[HT-7]

TL;DR:

Partial transport stream video files are playable, whereas partial program stream video files not. Smartphones record in program stream by default, and so does OpenCamera. I suggest OpenCamera to have an option for encoding in transport stream.

---

I have one day observed that broken pieces of some video files recovered from a corrupted file system are still playable. They are in M2TS, from a dedicated camcorder.

An incomplete video file can also be caused by a crash of the camera software or operating system, or sudden power loss due to undervoltage from an aged battery.

However, video files from a smartphone (MP4, H.264) were unplayable. The error log of VLC media player and console output of mplayer indicated "moov atom not found".

Then I informed myself about what this "moov atom" thing is. Turns out this is an issue with so-called program streams, which are fragile to damage since they store vital information necessary for encoding the entire stream in one location, either near the beginning or at the end of the file, the former of which would require the length or size of the video to be known, which is not possible in advance for video recording. Therefore, it is located at the end.

If the video file is truncated, meaning incomplete, the entire stream is useless. Repairing such stream is difficult, requires forensic-grade tools, and is not guaranteed to succeed. As an answer to Video.StackExchange.com question 29073 explains:

The mdat box in a MP4 contains the media payload but it is freeform. It is as good as gibberish on its own. The moov box identifies how many tracks there are, which codecs and their parameters those tracks contain, which portions of the data in the mdat correspond to which tracks, what their timestamps are..etc.

From another answer where only the video could be recovered but the sound track is lost:

I've just covered a mp4 with the above method - although sound is still corrupted, video is all good.

Audio lost ≠ "all good".

---

In comparison, as I understand, transport streams have many moov atoms for individual segments of the video. While this increases overhead size, which leads to slightly larger files at the same quality, it evades the moov atom hell, which is worth it.

The idea of a file becoming a deadweight just because the end is missing is despicable. If this was the case for photos, many photo files I recovered would be useless, and web images would be unviewable until fully downloaded, since they download sequentially. It appears that YouTube puts moov atoms at the beginning, but that is only possible since it processes an existing video file, not a real-time video recording.

Using program streams for mobile video recording was a bad idea from the beginning due to the possibility of corruption. It is OK for, for example, DVDs, since they are a finished package, and the benefit of the saved space granting few additional minutes of duration at the end outweighs the possibility of corruption from physical damage, and the user would realize anyway from the visible damage why their disc would have trouble playing. The moov atom is presumably read once by the DVD player and remains in the RAM for the session. But for a mobile device where video files can get corrupted for a multitude of reasons, the benefits of program stream underweigh.

It seems natural that the intact parts of a video file should simply be playable. After all, intact parts of cassette tapes are playable as well, meaning 1970s technology is more resistant to damage than modern mobile video.