Client re-using broken connection #2853
Comments
rnza0u
changed the title
rnza0u
changed the title
|
I have not seen this behavior before. Please reproduce this behavior with the environment variables |
|
Yes. Here is how i tested it:
I am now providing logs for both Node and Deno tests. Here are the logs in Node: 1st successful request (server up): Then i'm closing the server, i get these logs in real time: Second request (failed because server is offline): Then, i am restarting the server, and performing a third request, which is successful like the first one. Now let's see what is happening with Deno. Here is the first successful request while server is running: Then i shut down the server, but there are no real time logs indicating that the connection has ended. Then, for the second request: I restart the server, and then the final request still fails with the |
|
The gRPC client primarily relies on two behaviors in Node to detect that a connection has closed:
According to the log, Deno is not doing either of these things, so the gRPC client never knows that the connection has become unusable. This is a Deno bug; it needs to notify HTTP2 session users that connections have closed. As a side note, the logs show that you are creating a new client for every request. We recommend the opposite: that you use a single client to make all of your requests to the same service. |
|
Thanks for the analysis and for your piece of advice... I already made a tick to the Deno team here denoland/deno#26907. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem description
When i shut down my GRPC server, it invalidates all TCP connections bound to it, which is normal behavior.
However, it seems like the GRPC client still wants to re-use the dead connections.
After restarting the server and trying to send calls from the client, i get this error:
Received RST_STREAM with code 2 triggered by internal client error: connection closed because of a broken pipeI tried setting
grpc.enable_retriesto1, that did not work.I also tried to play with the keep alive settings so maybe the client would invalidate the connections and create new ones for further calls :
But then i randomly get:
Received RST_STREAM with code 2 triggered by internal client error: The pending stream has been canceled.It seems like the client is always trying to use dead connections and is not creating new ones.
What are the steps in order to overcome this ? Am i missing something ?
Thanks !
Environment
@grpc/grpc-jsin version1.12.2The text was updated successfully, but these errors were encountered: