Found go routine leak in ListBlobsFlatSegment

[dustintsao]

Which version of the SDK was used?

v0.6.0/v0.9.0

Which platform are you using? (ex: Windows, Linux, Debian)

MacOS

What problem was encountered?

We found go routine leak via uber-go/goleak

How can we reproduce the problem in the simplest way?

We run the following test code.

package whatever 
 import (
    "testing"
    "github.com/stretchr/testify/assert"
    "go.uber.org/goleak"
)
func TestA(t *testing.T) {
    defer goleak.VerifyNone(t)   
 _ , err := containerURL.ListBlobsFlatSegment(context.Background(), azblob.Marker{}, azblob.ListBlobsSegmentOptions{Prefix: "", MaxResults: 5})
assert.Nil(t, err)
}

And found there's a go routine leak as the error shows b

leaks.go:78: found unexpected goroutines:
    [Goroutine 37 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
    goroutine 37 [IO wait]:
    internal/poll.runtime_pollWait(0x3465e38, 0x72, 0xffffffffffffffff)
            /usr/local/Cellar/go/1.13.8/libexec/src/runtime/netpoll.go:184 +0x55
    internal/poll.(*pollDesc).wait(0xc00038e098, 0x72, 0x13e00, 0x13ec4, 0xffffffffffffffff)
            /usr/local/Cellar/go/1.13.8/libexec/src/internal/poll/fd_poll_runtime.go:87 +0x45
    internal/poll.(*pollDesc).waitRead(...)
            /usr/local/Cellar/go/1.13.8/libexec/src/internal/poll/fd_poll_runtime.go:92
    internal/poll.(*FD).Read(0xc00038e080, 0xc0005b6000, 0x13ec4, 0x13ec4, 0x0, 0x0, 0x0)
            /usr/local/Cellar/go/1.13.8/libexec/src/internal/poll/fd_unix.go:169 +0x22b
    net.(*netFD).Read(0xc00038e080, 0xc0005b6000, 0x13ec4, 0x13ec4, 0x203000, 0x34e25e8, 0x0)
            /usr/local/Cellar/go/1.13.8/libexec/src/net/fd_unix.go:202 +0x4f
    net.(*conn).Read(0xc000386010, 0xc0005b6000, 0x13ec4, 0x13ec4, 0x0, 0x0, 0x0)
            /usr/local/Cellar/go/1.13.8/libexec/src/net/net.go:184 +0x68
    crypto/tls.(*atLeastReader).Read(0xc0004a23e0, 0xc0005b6000, 0x13ec4, 0x13ec4, 0x0, 0xc0000648c8, 0xc0000648a0)
            /usr/local/Cellar/go/1.13.8/libexec/src/crypto/tls/conn.go:780 +0x60
    bytes.(*Buffer).ReadFrom(0xc000398258, 0x1d34340, 0xc0004a23e0, 0x100c375, 0x19e81c0, 0x1abbf80)
            /usr/local/Cellar/go/1.13.8/libexec/src/bytes/buffer.go:204 +0xb4
    crypto/tls.(*Conn).readFromUntil(0xc000398000, 0x34e1088, 0xc000386010, 0x5, 0xc000386010, 0xc000064930)
            /usr/local/Cellar/go/1.13.8/libexec/src/crypto/tls/conn.go:802 +0xec
    crypto/tls.(*Conn).readRecordOrCCS(0xc000398000, 0x0, 0x0, 0x3)
            /usr/local/Cellar/go/1.13.8/libexec/src/crypto/tls/conn.go:609 +0x124
    crypto/tls.(*Conn).readRecord(...)
            /usr/local/Cellar/go/1.13.8/libexec/src/crypto/tls/conn.go:577
    crypto/tls.(*Conn).Read(0xc000398000, 0xc00027d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
            /usr/local/Cellar/go/1.13.8/libexec/src/crypto/tls/conn.go:1255 +0x161
    net/http.(*persistConn).Read(0xc00035c240, 0xc00027d000, 0x1000, 0x1000, 0xc00034a000, 0xc000064c20, 0x1006d35)
            /usr/local/Cellar/go/1.13.8/libexec/src/net/http/transport.go:1758 +0x75
    bufio.(*Reader).fill(0xc000096660)
            /usr/local/Cellar/go/1.13.8/libexec/src/bufio/bufio.go:100 +0x103
    bufio.(*Reader).Peek(0xc000096660, 0x1, 0x0, 0x0, 0x1, 0xc000382100, 0x0)
            /usr/local/Cellar/go/1.13.8/libexec/src/bufio/bufio.go:138 +0x4f
    net/http.(*persistConn).readLoop(0xc00035c240)
            /usr/local/Cellar/go/1.13.8/libexec/src/net/http/transport.go:1911 +0x1d6
    created by net/http.(*Transport).dialConn
            /usr/local/Cellar/go/1.13.8/libexec/src/net/http/transport.go:1580 +0xb0d

     Goroutine 38 in state select, with net/http.(*persistConn).writeLoop on top of the stack:
    goroutine 38 [select]:
    net/http.(*persistConn).writeLoop(0xc00035c240)
            /usr/local/Cellar/go/1.13.8/libexec/src/net/http/transport.go:2210 +0x123
    created by net/http.(*Transport).dialConn
            /usr/local/Cellar/go/1.13.8/libexec/src/net/http/transport.go:1581 +0xb32
    ]

Please help to make sure it is a problem in sdk.
Thanks

Have you found a mitigation/solution?

no

[zezha-msft]

It looks like the connection is kept open, but I'm not sure whether that should really be considered a leak. I'd like to enlist the help of our Go expert @jhendrixMSFT: how should we interpret this?

[jhendrixMSFT]

The HTTP response body must always be drained (read/closed). It looks like this doesn't happen for non-successful HTTP status codes.

azure-storage-blob-go/azblob/zz_generated_container.go

Line 635 in 3efca72

defer resp.Response().Body.Close()

Is it the case that err is not nil?

[serprex]

@jhendrixMSFT Doesn't seem that theory is correct: validateResponse closes the body it it deoesn't return nil

azure-storage-blob-go/azblob/zz_generated_responder_policy.go

Line 43 in 4604c9f

func validateResponse(resp pipeline.Response, successStatusCodes ...int) error {

I'm trying to investigate various leaks in azure-storage-blob-go, but I'm having trouble finding out how generated code is generated