Is it possible to use caddy's modules inside L4 routes?

[stronny]

I'm prepared to try my hand at implementing it, but I want to ask first. Basically, I need caddy's reverse_proxy inside L4's route as a handler. This is because I want to pass mTLS fields upwards, and I use tls handler to terminate TLS inside L4, so caddy's http server doesn't see the certificate anymore and I can't think of an easy way to pass them along akin to PROXY protocol. This would also eliminate the need to listen on a separate port to loop a request back from L4 to http, but that's not the main point.

Any thoughts?

[mholt]

There's WIP (stalled currently though) that will allow you to pipe your L4 connection directly to your HTTP server without opening another socket: caddyserver/caddy#5040

[stronny]

This is cool, but will it address the main point? I need to add headers to an HTTPS request that should contain mTLS fields, like subject, will the piping allow for that?

[mholt]

I don't see the original post mentioned anything about adding HTTP headers; so you actually want to transfer data from the mTLS handshake to the HTTP headers for an HTTP proxy to send to an upstream?

[stronny]

Headers would be the way to "pass mTLS fields upwards". Yes, I want to:

• terminate TLS inside an L4 route
• add headers to the HTTP request inside the TLS stream
• reverse proxy it onwards as HTTP

[mholt]

That should be possible. When the TLS matcher is used, it sets placeholders. We may need to add the fields you want to the replacer though:

caddy-l4/modules/l4tls/matcher.go

Lines 102 to 105 in 2e6e133

	// also add values to the replacer
	repl := cx.Context.Value(layer4.ReplacerCtxKey).(*caddy.Replacer)
	repl.Set("l4.tls.server_name", chi.ClientHelloInfo.ServerName)
	repl.Set("l4.tls.version", chi.Version)

[stronny]

What do I do with the placeholders though? L4 doesn't have a concept of HTTP as far as I'm aware, right?

[mholt]

L4 has the "concept" of it in that it can match HTTP and get info from it, but it doesn't implement HTTP semantics like an HTTP server does.

If you want to manipulate the HTTP request you'll use the HTTP server. The HTTP server provides its own placeholders.

[stronny]

Yes, but at that point I don't have the SSL information anymore. Once gain, here's the overview of the caddy config:

client --(TLS)--> L4 app --(PROXY protocol)--> HTTP app --(HTTP)--> server

I want the upstream server to receive the information about the client's SSL certificate.

[mholt]

I guess it begs the question as to why you need to terminate TLS in the layer4 app then. The HTTP app also supports PROXY protocol... if you terminate TLS there then you should be able to do what you want.

[stronny]

The PROXY protocol is just a vehicle to pass the client's address upstream, it doesn't leave Caddy. I do a lot of things in L4, so TLS has to stay there.

Besides, it separates the stack layers cleanly, unlike doing everything inside http. It is both aesthetically pleasing, and very useful, the only problem with this separation being that there is no obvious mechanism to pass values around.

[mholt]

Can you just not terminate the requests that need HTTP requests manipulated?

I don't have a way to carry data between the apps yet, for a given connection... it's also weird because multiple requests can come in on a single connection, so what happens to the second request? It would already be handed off to the HTTP app at that point.

There would be some questions left to answer before it's clearly feasible.

[stronny]

The whole connection is using the same mTLS cert, just as it does have the same IP address.

I suppose I can just invent my own protocol, but it seems like an unfortunate solution.