Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not return io.EOF when n == len(p) #43

Merged
merged 1 commit into from
Nov 4, 2024

Conversation

nirs
Copy link
Member

@nirs nirs commented Nov 3, 2024

io.ReaderAt docs says[1]:

If the n = len(p) bytes returned by ReadAt are at the end of the input
source, ReadAt may return either err == EOF or err == nil.

We chose to return EOF, and this confuses io.CopyBuffer, since it expects to get io.EOF, not an error wrapping io.EOF[2]:

if er != nil {
    if er != EOF {
        err = er
    }
    break
}

We can fix this by never wrapping io.EOF, or by not returning io.EOF when we can avoid it. I chose the second way since it is much simpler.

Fixed by returning EOF only if n < len(p) when reading zeros. This means that reading the last buffer will return len(p), nil, and trying to read the next buffer will return 0, EOF.

To reproduce we need to write zero cluster at the end of the image:

% qemu-img create -f qcow2 test.qcow2 1m
Formatting 'test.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16

% qemu-io -f qcow2 -c 'write -z 960k 64k' test.qcow2
wrote 65536/65536 bytes at offset 983040
64 KiB, 1 ops; 00.00 sec (97.962 MiB/sec and 1567.3981 ops/sec)

% qemu-img map --output json test.qcow2
[{ "start": 0, "length": 983040, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false},
{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}]

Without the fix we fail when reading the last 32k at 1015808:

% ./go-qcow2reader-example read test.qcow2 >/dev/null
ERROR: failed to read standard cluster (len=32768, off=1015808, desc=0x1): EOF

I don't understand why we try to read with a buffer size of 32k, when we use a 64k buffer in the read example. This may be another bug.

[1] https://pkg.go.dev/io#ReaderAt
[2] https://cs.opensource.google/go/go/+/refs/tags/go1.23.2:src/io/io.go;l=449

Fixes: #40

io.ReaderAt docs says[1]:

> If the n = len(p) bytes returned by ReadAt are at the end of the input
> source, ReadAt may return either err == EOF or err == nil.

We chose to return EOF, and this confuses io.CopyBuffer, since it
expects to get io.EOF, not an error wrapping io.EOF[2]:

    if er != nil {
        if er != EOF {
            err = er
        }
        break
    }

We can fix this by never wrapping io.EOF, or by not returning io.EOF
when we can avoid it. I chose the second way since it is much simpler.

Fixed by returning EOF only if n < len(p) when reading zeros. This means
that reading the last buffer will return len(p), nil, and trying to read
the next buffer will return 0, EOF.

To reproduce we need to write zero cluster at the end of the image:

    % qemu-img create -f qcow2 test.qcow2 1m
    Formatting 'test.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16

    % qemu-io -f qcow2 -c 'write -z 960k 64k' test.qcow2
    wrote 65536/65536 bytes at offset 983040
    64 KiB, 1 ops; 00.00 sec (97.962 MiB/sec and 1567.3981 ops/sec)

    % qemu-img map --output json test.qcow2
    [{ "start": 0, "length": 983040, "depth": 0, "present": false, "zero": true, "data": false, "compressed": false},
    { "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": true, "data": false, "compressed": false}]

Without the fix we fail when reading the last 32k at 1015808:

    % ./go-qcow2reader-example read test.qcow2 >/dev/null
    ERROR: failed to read standard cluster (len=32768, off=1015808, desc=0x1): EOF

I don't understand why we try to read with a buffer size of 32k, when we
use a 64k buffer in the read example. This may be another bug.

[1] https://pkg.go.dev/io#ReaderAt
[2] https://cs.opensource.google/go/go/+/refs/tags/go1.23.2:src/io/io.go;l=449

Fixes: lima-vm#40
Signed-off-by: Nir Soffer <[email protected]>
Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, can we have a test?

@AkihiroSuda AkihiroSuda merged commit 94ebcca into lima-vm:master Nov 4, 2024
2 checks passed
@nirs nirs deleted the fix-read-zero-cluster branch November 4, 2024 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failed to read zero clusters
2 participants