Grpc Stream Error: 8 RESOURCE_EXHAUSTED: Received message larger than max (1752460652 vs 4194304)

[DylanAhTeck-ONE]

SDK Component

Zeebe

Expected Behavior

In the previous community project, are workers were able to poll for work

Current Behavior

Using the new sdk and the same worker/ZB client configs, our workers are getting Grpc Stream Error: 8 RESOURCE_EXHAUSTED: Received message larger than max (1752460652 vs 4194304) errors

This is happening immediately after worker creation - before any process instances etc are created - for all of our workers. The received message size (1752460652) is the same for all the worker errors.

Possible Solution

Steps to Reproduce

Context (Environment)

[jwulf]

I'm not aware of a code change that will do that. The Zeebe gRPC client code is basically ported across.

Please see this issue: camunda/camunda#12104

If this doesn't allow you to fix it, then what we'll need for this is a Minimal Reproducer. That would be a Docker compose to start a broker setup, and a small project that triggers this error condition. With that in hand we can fix it.

[jwulf]

This seems to be caused by grpc/grpc-node#2822

It will happen if there are no jobs available for 60 seconds on SaaS. It is a misinterpretation of the 504 timeout response.

The workaround is to set longPoll: 50000 in the worker constructor. This causes the poll to timeout on the client before the gateway returns the 504 timeout response.