moviepy returns empty frames if frames are not accessed strictly sequentially

[pgunn]

Expected Behavior

See below - the handle to a video file should not start returning zeros. The video file we're using with this is here:
https://caiman.flatironinstitute.org/~neuro/caiman_downloadables/msCam13.avi

We found this while spotting the error in pims, using the moviepy backend, and eventually tracked it down to moviepy when considering directly using moviepy to avoid what we believed was a bug in pims, instead to find that it's actually a bug in moviepy.

Oddly, accessing frame 0 of the movie fixes the handle to the movie

Actual Behavior

mph = moviepy.editor.VideoFileClip(fn)
mph.size
[752, 480]
mph.duration
50.0
mph.fps
20.0
np.array(mph.get_frame(0)).max()
173
np.array(mph.get_frame(2)).max()
206
np.array(mph.get_frame(1)).max()
0
np.array(mph.get_frame(0)).max()
173
np.array(mph.get_frame(1)).max()
188

Steps to Reproduce the Problem

Specifications

• Python Version: 3.11.5
• MoviePy Version: 1.0.3
• Platform Name: Linux (tested on Windows too)

[pgunn]

Following it down into the backend:

mph = moviepy.video.io.ffmpeg_reader.FFMPEG_VideoReader(fn)
>>> mph.get_frame(0).max()
173
>>> mph.get_frame(1).max()
188
>>> mph.get_frame(2).max()
206
>>> mph.get_frame(1).max()
0
>>> mph.get_frame(0).max()
173
>>> mph.get_frame(1).max()
188

I'm looking at the logic here:
https://github.com/Zulko/moviepy/blob/master/moviepy/video/io/ffmpeg_reader.py#L218

But I find the existence of self.last_read and self.lastread as distinct variables to be puzzling. I'm guessing the bug is somewhere around there.

[pgunn]

It looks like, at least with our datafiles, any use of -ss before -i in the call to ffmpeg causes ffmpeg itself to return empty frames, while the second call to -ss (while not performant) returns data correctly.

Found this by modifying moviepy/video/io/ffmpeg_reader.py to print out the commandline its building with various calls, and looking over the if start_time != 0 block in initialize().

Something like:

ffmpeg -ss 10 -i /home/pgunn/caiman_data/example_movies/msCam13.avi -ss 10 -loglevel info -vf scale=752:480 -sws_flags bicubic -pix_fmt rgb24 /tmp/foo_noprelinear.avi

(really providing anything at all, float or integer, with a first -ss, gives us a blank movie.

Without that initial -ss, we get movies that work.

I'm wondering if it might be better to remove the first -ss from these calls, or at least provide an option to.

[pgunn]

I will follow up with the ffmpeg developers.

[pgunn]

https://trac.ffmpeg.org/ticket/10602

[Breakthrough]

But I find the existence of self.last_read and self.lastread as distinct variables to be puzzling. I'm guessing the bug is somewhere around there.

This seems like an oversight in #1170, I tested locally and remaing the fields to all have the same name works correctly. There's also some cases where you can make the variables get out of sync depending on what order you call get_frames/read_frames.

These variables were previously not underscore-prefixed, so one of my projects relies on the presence of lastread. I would request that both names are kept in sync to avoid breaking code that relied on this field. That could be done by using last_read everywhere, and providing a property that aliases it. Submitted PR #2257 to resolve the discrepancy there.