Error message on queue:retry related to "batching"

[ezmiller]

I recently tried to run the sls queue:failed:retry command, and received an error:

Batch requests cannot be longer than 262144 bytes. You have sent 577828 bytes.

What gives me a pause about this error is that we are using a batch size of 1, which I believe is the default for the queue construct. This error makes me think that this command is trying to batch more messages into one batch than just one.

If so, that would be a problem because our code is written with the assumption that there's a batch size of 1. We could chang that of course, but I would expect the retry logic to transfer items back into the queue with the same batch size.

I could be wrong that the retry is using a batch size > 1, but then I would have expected to get an error message like this during processing the first time. (And I know that that wasn't the error that put the items in the DLQ because I deliberately threw an error to test this error flow.)

Can anyone help me understand what may be going on here?

[mnapoli]

Hi, do you have the full stack trace to pinpoint where the error is thrown?

I wonder if this might not be here actually: https://github.com/getlift/lift/blob/master/src/constructs/aws/queue/sqs.ts#L93-L111

We send messages in batches of 10 to SQS (when retrying them), maybe it goes over the limit if the 10 messages are too big?

After all, the error message says batch requests cannot be longer than 256k, and a single SQS message can be up to 256k. So maybe we should check the size of the messages before sending them in batch, and adjust the batch size based on the message size?

[ezmiller]

I can try to send, but I think that your idea of the origin of the error is correct.

My question , however, is why/if it makes sense for lift to batch process the DLQs if the default setting for SQS queues thesmelves is to have batching disabled. Shouldn't the policies of the two queues match?