System.ObjectDisposedException when processing multiple requests at once

[derole1]

Describe the bug
When the server encounters multiple requests at once, or a lot of traffic, WebApiController encounters an ObjectDisposedException.

To Reproduce

1. Create a WebApiController using sqlite-net SQLiteAsyncConnection and JSON requests and responses.
2. Send a few JSON requests in quick sucession that use async database connections, some requests will throw an internal server error as the HttpContext is being disposed of.

Expected behavior
HttpContext objects should not get disposed of by garbage collection whilst processing requests.

Exception

13:06:48.040 ERR >> [WebServer] [eGF0CV3pdEeq+d1hJRegoA] Unhandled exception.
    $type           : System.ObjectDisposedException
    Message         : Cannot access a disposed object.
    ObjectName      :
    TargetSite      : Void EnsureCanChangeHeaders()
    StackTrace      :
        at EmbedIO.Net.Internal.HttpListenerResponse.EnsureCanChangeHeaders()
        at EmbedIO.Net.Internal.HttpListenerResponse.set_ContentType(String value)
        at EmbedIO.ResponseSerializer.Json(IHttpContext context, Object data)
        at JsonApi.Api.GetDataHandler() in D:\source\repos\JsonApi\JsonApi\Api.cs:line 39
        at EmbedIO.Routing.RouteResolverBase`1.ResolveAsync(IHttpContext context)
        at EmbedIO.Routing.RouteResolverCollectionBase`2.ResolveAsync(IHttpContext context)
        at EmbedIO.Routing.RoutingModuleBase.OnRequestAsync(IHttpContext context)
        at EmbedIO.WebModuleBase.HandleRequestAsync(IHttpContext context)
        at EmbedIO.ExceptionHandler.Handle(String logSource, IHttpContext context, Exception exception, ExceptionHandlerCallback handler, HttpExceptionHandlerCallback httpHandler)
        at EmbedIO.WebModuleBase.HandleRequestAsync(IHttpContext context)
        at EmbedIO.Internal.WebModuleCollection.DispatchRequestAsync(IHttpContext context)
        at EmbedIO.WebServerBase`1.DoHandleContextAsync(IHttpContextImpl context)
    Source          : EmbedIO
    HResult         : -2146232798

Desktop:

• OS: Windows 10 20H2
• Browser: Firefox
• Version: 106.0.1 x64
  Note this happens with all browsers, not just firefox.

Additional context
The WebUI that makes the JSON requests uses fetch, but I dont think this has any relevance to the issue as the same problem occurs when sending requests with Telerik Fiddler.

[derole1]

One observation ive made, if I use the JsonData attribute in the route function parameters to get the json data, and set the response object as the return type and return it instead of using HttpContext.SendDataAsync(), this issue doesnt occur and everything works smoothly. However unfortunately there is no such equivalent for binary data which is unfortunate as that also suffers from the same issue.

My theroy is that when theres multiple requests happening asynchronously, One request finishes and closes the HttpContext whilst the rest are still using it, which causes the ObjectDisposedExceptions. Hope this workaround helps anyone with the same issue in the meantime.

[rdeago]

Hello @derole1, thanks for using EmbedIO!

I agree that there's probably a bug somewhere in EmbedIO's implementation of HttpListener, because a response clearly gets closed ahead of its due time.

That having been said, the root cause of the exception you observe is your use of HttpContext.SendDataAsync(), which must never be called from a WebApiModule.

Here's what happens:

• WebApiModule parses the request, extracts parameter values and calls your controller method;
• your controller method calls SendDataAsync(), thereby sending response headers;
• your controller method returns;
• WebApiModule calls the JSON response serializer to serialize your return value and send it as a response (that's the reason why WebApiModule even exists: to spare you from having to serialize data and call SendDataAsync);
• the serializer tries to set the Content-Type response header, but headers have already been sent, so HttpListenerResponse throws an InvalidOperationException;
• the exception is caught here by WebServerBase<>, that calls this method to generate an HTTP 500 response;
• in the meantime, the client has received your response (the data you sent before returning from the controller method) and has reused the TCP connection to send another request;
• after sending the 500 response, WebServerBase<> closes the HttpContext, which closes the HttpListenerResponse, which in turn closes the underlying connection;
• whichever method handles the second request is bound to get an ObjectDisposedException as soon as it tries to send a response.

(I'm not 100% sure about all the fine details, but I think I got it right.)

I don't even know whether we can fix this, let alone how. It would require some serious reworking of the Net.Internal workspace, but we have to keep it compatible with Microsoft's HttpListener, so our options are quite limited.

As you have guessed, the workaround is... to not cause exceptions in the first place, as silly as it sounds.

Unlike the weird behavior of HttpConnection, EmbedIO failing because you called SendDataAsync from a Web API controller method is not a bug: it's you asking for trouble. 😁 Think of WebApiModule as a higher-level API over the whole request / response stuff: instead of receive / deserialize / compute / serialize / send, you get to do the fun part (compute) and it takes care of everything else.