Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Since version 1.10.9 grpc-node doesnt comply with grpc spec in case of non 200 response #2822

Open
RemiBou opened this issue Sep 10, 2024 · 7 comments

Comments

@RemiBou
Copy link

RemiBou commented Sep 10, 2024

Problem description

When server returns HTTP 404 with "404 not found" in body grpc-node throws a RESOURCE_EXHAUSTED

Reproduction steps

Init your grpc client with a non grpc server, it should throw "UNIMPLEMENTED" error, instead it might throw "RESOURCE_EXHAUSTED"

Environment

  • grpcnode >= 1.10.9

Additional context

I think it's same issue as #2809
I want to add that this should be fixed in grpc-node as it doesn't comply with the grpc spec anymore. Reference
https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md

Implementations should expect broken deployments to send non-200 HTTP status codes in responses as well as a variety of non-GRPC content-types and to omit Status & Status-Message. Implementations must synthesize a Status & Status-Message to propagate to the application layer when this occurs.

When "404" is return by server then grpc status 12 and error UNIMPLEMENTED should be send to the client application.

Java and GO grpc library work that way and grpc-node used to work that way until this commit 674f4e3

@murgatroid99
Copy link
Member

OK, I see the problem: the RESOURCE_EXHAUSTED error from attempting to parse the response body is superseding the UNIMPLEMENTED error from the HTTP status.

@RemiBou
Copy link
Author

RemiBou commented Sep 18, 2024

@murgatroid99 do you think it should be fixed ?

@murgatroid99
Copy link
Member

Nothing has happened with this yet.

@erlendnils1
Copy link

We are experiencing a similar issue. A HTTP 502 error from the load balancer used to throw an UNAVAILABLE error, but now throws RESOURCE_EXHAUSTED "Received message larger than max" based on misinterpreting the response as a gRPC response.

@tysseng
Copy link

tysseng commented Oct 14, 2024

For others experiencing the same, the reported message body sizes in the case mentioned by @erlendnils1 are:

1633951815 = 0x61 0x64 0x20 0x47 or 'ad g', meaning the client received 'bad gateway'
1013478509 = 0x3C 0x68 0x74 0x6D or '<htm', when the client receives an error page from the GKE (Kubernetes) load balancer, which returns a message saying 'The server encountered a temporary error and could not complete your request. Please try again in 30 seconds.'

@anotheri
Copy link

anotheri commented Dec 8, 2024

@tysseng and what about 1752460652?

UPD: After some investigation, it looks like this message is related to not available resource (e.g. b/c of firewall).

@jwulf
Copy link

jwulf commented Dec 13, 2024

I am seeing this as well. When the gateway returns 504 timeout, it is misinterpreted and an error is thrown: Grpc Stream Error: 8 RESOURCE_EXHAUSTED: Received message larger than max (1752460652 vs 4194304).

This is with version 1.12.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants