Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Injected proxy variables into endpoint elasticsearch does not include NO_PROXY variable #6209

Open
janniten opened this issue Dec 4, 2024 · 8 comments
Labels
bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Comments

@janniten
Copy link
Contributor

janniten commented Dec 4, 2024

Hello,
We did an upgrade from 8.14.2 to 8.16.1.
In our environment the agent has the HTTP_PROXY, HTTPS-PROXY and NO_PROXY configured.
NO_PROXY variable is not being injected into endpoint .
As a result, the elastic endpoint is trying to connect the ES cluster Via proxy, which is not correct.

Thank you

@janniten janniten added the bug Something isn't working label Dec 4, 2024
@cmacknz cmacknz added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Dec 4, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@michel-laterman
Copy link
Contributor

HTTP_PROXY/HTTPS_PROXY injection was added with: #5044
We'll probably need to alter the injectProxyURL func to add NO_PROXY.

@intxgo, would the expected value in the config be no_proxy: string, or no_proxy: []string?

@intxgo
Copy link
Contributor

intxgo commented Dec 6, 2024

Since the env vars are injected as a regular output proxy settings, there's no place for NO_PROXY (endpoint won't recognize it, ignore it). However since the output URL is known, I think the NO_PROXY can be matched against the output URL at Agent side before deciding to inject HTTP_PROXY/HTTPS_PROXY variable to the output config.

@intxgo
Copy link
Contributor

intxgo commented Dec 24, 2024

Alternative solution would be to pass all three HTTP_PROXY, HTTPS_PROXY, NO_PROXY to Endpoint as Agent env variables, a new structure in the config message instead of injecting them to the output configuration. We will need to handle this on Endpoint side too, but in general Endpoint proxy table already has place for storing Agent env variables.

@ManojPraveeen
Copy link

Hi Team,

The issue is related to case 01791476. From what I understand from the @intxgo statement, you’re suggesting including these proxy configurations directly in the agent's environment settings, right?

It would look something like the configuration below:

proxy_settings:
  http_proxy: 
  https_proxy:
  no_proxy: 

Adding the above parameters to the configuration should fix the issue, correct me if I’m wrong.

@intxgo
Copy link
Contributor

intxgo commented Jan 15, 2025

@ManojPraveeen it's more complicated. Since quite some time we're not forwarding the config between components anymore, instead we have a GRPC base communication, see demo client https://github.com/elastic/elastic-agent-client

The GRPC is roughly equivalent to the config, however currently the protobuf definition doesn't have a place to forward the Agent env proxy settings, so instead Agent's env variables (only the first 2) are evaluated at Agent side and injected as regular proxy setting as if it came from Fleet. If we follow the path to revert the injection, introducing a proper means to forward all the variables via protobuf, we will have to update Agent and Endpoint to support the change. BTW, since these are env variables, they won't be ever stored in Endpoint vault as they are volatile.

The other option, a quicker patch to existing solution, was to evaluate all three env variables at Agent side to see if NO_PROXY nullifies HTTP_PROXY setting for the Elasticsearch address.

@ManojPraveeen
Copy link

ManojPraveeen commented Jan 15, 2025

@intxgo
From what I understand the Elastic Agent is currently only uses the HTTP_PROXY and HTTPS_PROXY settings. You want to implement a logic from agent end, So when the agent tries to connect directly to Elasticsearch using NO_PROXY, it should bypass the proxy settings. Is that correct?

If that's what you are saying, could you explain how we can achieve this workaround?

@intxgo
Copy link
Contributor

intxgo commented Jan 15, 2025

@ManojPraveeen there is already a workaround in place #5044 which unfortunately didn't account for NO_PROXY so either it has to be further improved or reverted to be replaced with a new mechanism

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

No branches or pull requests

6 participants