Gatekeeper prevents streaming output #645
Comments
|
Had a similar, related issue with "streaming" requests. Use caseUsing MJPEG-streamer to expose a webcam. The way this "stream" is implemented (http handler here), is by sending a This particular software exposes such a stream on Expected resultLouketo is able to proxy this request as an unbuffered, infinite response body. Actual resultWhile buffering is not an obvious issue here, the stream is closed after 10 seconds. A workaround for this would be to set ReproducingRequirements
Start up this compose file with # docker-compose.yml
version: '3.7'
services:
oidc-gate:
image: quay.io/louketo/louketo-proxy
command: >-
--server-write-timeout=0s
--upstream-url=http://webcam:80
--listen=:3000
--enable-default-deny=true
--discovery-url=https://example-keycloak/auth/realms/local-testing
--client-id=local-app
--client-secret=12345678-1234-1234-1234-123456789012
--encryption-key=AgXa7xRcoClDEU0ZDSH4X0XhL5Qy2Z2j
ports:
- 3000:3000
webcam:
image: sixsq/mjpg-streamer
devices:
# Streams from a V4L2 camera. Like a laptop/usb webcam.
- /dev/video0
ports:
# Runs on :80 internally,
# we expose 8080 for testing without auth.
- 8080:80
Additional InformationOther reverse proxy setups, do not require changing such a timeout in the first place.
This is closer to the problem @pahrens is observing, because Caddy will disable buffering of the response this way. Which makes a lot of sense, because no buffers means no latency, no congestion (for the proxy), no memory hogging... |
|
I've poked a little bit at the provided PHP example. Using a similar repro as I shared for my use-case: # docker-compose.yml
version: '3.7'
services:
oidc-gate:
image: quay.io/louketo/louketo-proxy
command: >-
--server-write-timeout=0s
--upstream-url=http://php:80
--listen=:3000
--enable-default-deny=true
--discovery-url=https://example-keycloak/auth/realms/local-testing
--client-id=local-app
--client-secret=12345678-1234-1234-1234-123456789012
--encryption-key=AgXa7xRcoClDEU0ZDSH4X0XhL5Qy2Z2j
ports:
- 3000:3000
php:
image: php:apache
volumes:
- ./stream-example.php:/var/www/html/index.php
ports:
- 8080:80Used techThe way PHP streams the response here is using Additionally, my PHP response included ObservationsThe proxied response also reports Using a Now,
This suggests, removing the chunks and buffering, is perfectly within spec. |
|
Digging into this more, I found the issue. Cause, default flushing behaviourThe proxy dependency goproxy does not control how flushing is done. So it defaults to what the go standard library implements for both the upstream and downstream clients. https://github.com/elazarl/goproxy/blob/0581fc3aee2d07555835bed1a876aca196a4a511/proxy.go#L180 Flushing manuallyTo flush sooner, the That would flush whatever we have in the buffer so far, which may result in different sizes of chunks than PHP originally gave us. (Though that's acceptable in HTTP spec). Adopting
|
TODO: - SIGSEGV spotted when using 2s default and `make test` - Unsure about Apache 2 & BSD-3 license compatibility as used
TODO: - Unsure about Apache 2 & BSD-3 license compatibility as used - Test for streaming, streaming + gzip
|
I got some of the way there with my WIP, feel free to check that out and use it. |
Gatekeeper prevents streaming output
Summary
I have a PHP based website that has a feature that sends already partly output to the requester. Below you can find a simple php example for this logic.
When using Gatekeeper before this website with this feature, only the complete output is displayed at the requester and not already the parts of the output.
Environment
Expected Results
I would like to have a way to configure gatekeeper to return already parts of the output of a website instead of the complete output.
Actual Results
see summary
Steps to reproduce
Additional Information
During my research, I have found out that there is a similar issue related to Nginx, were the internal buffer from Nginx is preventing this kind of logic to work. Maybe it's a similar reason here as well.
The text was updated successfully, but these errors were encountered: