Motion interpolation between AnimatedSprite frames

[panorb]

Context

I was trying around with fires lottie module for the upcoming LD47 game jam.

Problem

File sizes. The generated scene files quickly reach sizes that the godot editor can't handle in real time anymore. So it made me think.

Suggestion

Make a module in support of a solution.

Solution

What if we could give the AnimatedSprite Node the ability to generate the in-between frames in real time via Motion Interpolation, so that we no longer need to generate 60fps in high resulation, which leads to lagg spikes, but maybe even 12 could suffice if the inbetweens are generated in real-time. What do you think of this idea?

[willnationsdev]

Sounds like a decent idea to me. But, that actually sounds like you'd want to add an enhancement to the core engine though; you should probably open a formal proposal. You've got a use case that was obstructing you, clearly identified the problem, and came up with a solution. Meets all the requirements.

If you used a module, you would have to define a whole new type of AnimatedSprite, but if it's an engine change, you can just directly edit the original class to optionally support that behavior. Sounds cleaner to me that way.

[Xrayez]

When looking at the Godot source, the animated sprite drawing boils down to texture drawing:

texture->draw_rect_region(ci, dst_rect, Rect2(Vector2(), texture->get_size()), Color(1, 1, 1), false, normal, specular, Color(specular_color.r, specular_color.g, specular_color.b, shininess));

For motion interpolation to be implemented, I guess one would still need to bake those in-between frames as textures anyway with the current API, so in theory that's quite possible to do given the already mentioned lottie module, perhaps it just needs optimizations, CC @fire.

That said, baking textures at real-time may be costly even if implemented in C++, it would depend on the texture size.

Real-time would probably require generating Images and corresponding ImageTextures from those. Perhaps Godot could implement Image.interpolate_with(image) to facilitate this process (and cache those frames of course, internally), that's talking about CPU-only approach, of course would be more efficient if this could be done via shaders with no sweat, perhaps.

I recall reduz wasn't against adding more useful methods to Image. 🙂

But anything more complex probably requires machine learning techniques anyway, which are certainly plugin territory for now, because it's difficult to possibly predict what could be in-between very distinct frames.

[Xrayez]

Looks like there's a solution for this using https://github.com/Bauxitedev/bitmapflow written in Rust, perhaps @Bauxitedev could elaborate. 🙂

[Bauxitedev]

@Xrayez It's a good idea in theory, unfortunately optical flow algorithms are quite slow, especially for large sprites. The flow vectors would have to be calculated/baked at runtime, which could take several minutes if the game contains a large amount of animated sprites. However, after the flow vectors have been calculated, interpolating the sprites can be done very quickly on the fly, using a simple shader that runs on the GPU. (it's just a shader that pushes the pixels around according to the flow vectors)