Performance issues related to pulling 1 message at a time and returnImmediately is deprecated

[bernardwiesner]

Thanks for working on this package. I was testing it though and noticed there is a performance issue with it. Pull and ack together can take anywhere from 100ms to up to 1 second. This is not related to networking but by design on pubsub infrastructure. Pubsub is not intended to pull and ack 1 message at a time, the pull feature is to be used with batches of messages, so for example pull 500 and acknowledge 500 messages.

This package can be used if you dont have heavy working tasks, but whenever you need to process thousands of jobs in a minute your bottleneck for sure will be the time it takes to pull and ack to pubsub.

Additionally returnImmediately is also deprecated as is pointed out here: https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/pull#request-body

Warning: setting this field to true is discouraged because it adversely impacts the performance of subscriptions.pull operations. We recommend that users do not set this field.

This thread is also useful to read in regard to pulling and acknowledging 1 message at a time: googleapis/google-cloud-php#939 (comment)

I would recommend against using pubsub in general with PHP if you care about scaling. Another reason is regarding publishing messages. The first time you publish to pubsub it takes anywhere from 0.5-1s. Since php always starts a new request, your web servers will always have go through first time publish, hanging your app for up to 1s just to publish a message, this is unacceptable. Using redis and laravel queue dispatch, it takes max a few milliseconds to set or get a message.

Using other languages such as go, java, nodejs, etc they have a server running in memory that starts up once and is reused for every further request. Even if first publish takes time, all further publish events will be very fast.

Pubsub seems to have some connection issues on first requests, I guess its related to authentication.