Skip to content

Commit

Permalink
Fix bug where a large file (>1.2GB) would cause a int overflow
Browse files Browse the repository at this point in the history
The reasons was to avoid large buffers getting allocated for small files, but that hurt big files in the end
  • Loading branch information
DavyLandman committed Feb 13, 2024
1 parent e4078ee commit 8e0adb3
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@ protected ByteBuffer refill(ByteBuffer torefill) throws IOException {

private static ByteBuffer constructDecompressedBuffer(ByteBufferInputStream oriStream) {
int compressedSize = oriStream.getByteBuffer().remaining();
ByteBuffer result = DirectByteBufferCache.getInstance().get(Math.min(compressedSize * 2, ZstdDirectBufferDecompressingStream.recommendedTargetBufferSize()));
int bufferSize = ZstdDirectBufferDecompressingStream.recommendedTargetBufferSize();
if (bufferSize > compressedSize) {
bufferSize = Math.min(compressedSize * 2, bufferSize);
}
ByteBuffer result = DirectByteBufferCache.getInstance().get(bufferSize);
result.limit(0); // delay compression for first read
return result;
}
Expand Down

0 comments on commit 8e0adb3

Please sign in to comment.