Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify UniqueAVFrame handling in audio decoder #571

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
+8 −12
Draft

Simplify UniqueAVFrame handling in audio decoder #571

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions src/torchcodec/decoders/_core/VideoDecoder.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1361,14 +1361,8 @@ void VideoDecoder::convertAudioAVFrameToFrameOutputOnCPU(
static_cast<AVSampleFormat>(avFrameStream.avFrame->format);
AVSampleFormat desiredSampleFormat = AV_SAMPLE_FMT_FLTP;

UniqueAVFrame convertedAVFrame;
if (sourceSampleFormat != desiredSampleFormat) {
convertedAVFrame = convertAudioAVFrameSampleFormat(
avFrameStream.avFrame, sourceSampleFormat, desiredSampleFormat);
}
const UniqueAVFrame& avFrame = (sourceSampleFormat != desiredSampleFormat)
? convertedAVFrame
: avFrameStream.avFrame;
const UniqueAVFrame avFrame = convertAudioAVFrameSampleFormat(
avFrameStream.avFrame, sourceSampleFormat, desiredSampleFormat);

AVSampleFormat format = static_cast<AVSampleFormat>(avFrame->format);
TORCH_CHECK(
Expand All @@ -1395,11 +1389,13 @@ void VideoDecoder::convertAudioAVFrameToFrameOutputOnCPU(
}

UniqueAVFrame VideoDecoder::convertAudioAVFrameSampleFormat(
const UniqueAVFrame& avFrame,
UniqueAVFrame& avFrame,
AVSampleFormat sourceSampleFormat,
AVSampleFormat desiredSampleFormat
AVSampleFormat desiredSampleFormat) {
if (sourceSampleFormat == desiredSampleFormat) {
return std::move(avFrame);
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scotts WDYT - this is wrong, isn't it? This will end up destructing the avFrameStream.avFrame that was passed as a parameter to convertAudioAVFrameToFrameOutputOnCPU?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On this being wrong - I don't know, that depends on what we need for the calling side. What this will do is take ownership away from the input avFrame and give it to the output UniqueAVFrame. The question then is: is that okay for avFrameStream in the calling context? That is, is it okay for the avFrameStream in the calling context to be modified such that it no longer owns the avFrame? Because as a consequence of the move, avFrameStream.avFrame will become nullptr.

And if it's not okay, then should avFrameStream in convertAudioAVFrameToFrameOutputOnCPU() actually be a const reference?


) {
auto& streamInfo = streamInfos_[activeStreamIndex_];
const auto& streamMetadata =
containerMetadata_.allStreamMetadata[activeStreamIndex_];
Expand Down
2 changes: 1 addition & 1 deletion src/torchcodec/decoders/_core/VideoDecoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ class VideoDecoder {
torch::Tensor& outputTensor);

UniqueAVFrame convertAudioAVFrameSampleFormat(
const UniqueAVFrame& avFrame,
UniqueAVFrame& avFrame,
AVSampleFormat sourceSampleFormat,
AVSampleFormat desiredSampleFormat);

Expand Down
Loading