Huber on Linux does not work behind a proxy

[sarg3nt]

When running Huber from behind a proxy it is unable to talk to Github so huber search and any other command that need to talk to Github do not work.

I think you just need to configure reqwest to use the HTTP_PROXY, HTTPS_PROXY and NO_PROXY vars if they exist.
See: https://docs.rs/reqwest/latest/reqwest/struct.Proxy.html

NOTE: reqwest docs say:

NOTE: System proxies are enabled by default.

So something must be interfering with this or turning it off.
Please let me know if need someone to test changes.

[AWDDude]

+1
Not supporting proxies, prevents this from being used in many enterprises.

[sarg3nt]

Hi, this is super critical to us as we currently can't use huber in our automation platform to auto update docker images that use huber to install apps.
I added this bug on April 19 and it looks like it's not getting any attention.
Should we give up and abandon huber?
Not trying to be a jerk but we need this to function to use huber.

Note: Currently we are getting around this by using WSL on Windows as it uses the Windows proxy and to Linux in WSL it just works. However we can't use this on a real Linux box so we can't use huber in Jenkins, which is our automation platform and really really need to. Currently I'm manually rebuilding all the Docker images that use huber but that is not sustainable.

Thank you.

[DanceMore]

I have some ideas about how to set up a test environment for this. Not everyone has a proxy server or enterprise networking restrictions to test behind; and it's potentially non-trivial for many home-labbers to set up.

No promise on a timeline, I'm just a random SRE that really likes huber and I want to be helpful.

I think you might have nerd-sniped me by asking an interesting question that should be totally tractable. especially "NOTE: System proxies are enabled by default."... I've consumed HTTP Libraries that had proxy support out of the box, what's going on if just setting ENV doesn't work... 🤔

[DanceMore]

I went digging and I found that most of the HTTP work is actually inside hubcaps-ex library. It's really building a Github API Client complete with caching, and rarely doing direct HTTP Client style GETs.

I also had a sudden "hunch" to try explicitly specifying HTTPS_PROXY instead of HTTP_PROXY ....

squid  | 1701806670.918    137 172.22.0.1 TCP_TUNNEL/200 7320 CONNECT raw.githubusercontent.com:443 - HIER_DIRECT/185.199.108.133 -

is it perhaps that the library is actually differentiating between HTTP and HTTPS, and because all the Github API Commands operate over HTTPS, the correct environment variable to set is only HTTPS_PROXY ...?

another research note, I think it was the curl manpage pointed out that historically, both upper-case and lower-case ENV was acceptable and lowercase http_proxy is handled before HTTP_PROXY. so you may want to export https_proxy=proto://host:port instead.

I also observed that it does seem to expect a protocol, it's not going to assume one for you.

[sarg3nt]

@DanceMore Thanks for looking into this. We really appreciate it.

We set all six possible variables, i.e.

HTTP_PROXY
HTTPS_PROXY
http_proxy
https_proxy
NO_PROXY
no_proxy

[DanceMore]

and what kind of errors / output do you get when it fails ?

[DanceMore]

uses -l not ENV LOG_LEVEL

% HTTPS_PROXY="http://localhost:3128" huber search -l trace
...
⢼ Updating managed repos
⣸ Updating managed repos
 2023-12-06T01:20:26.527Z TRACE simpledi_rs::di          > getting immutable "huber::service::repo::RepoService" from DI system
 2023-12-06T01:20:26.527Z TRACE simpledi_rs::di          > getting immutable "huber_common::model::config::Config" from DI system
⠁
 2023-12-06T01:20:26.527Z TRACE simpledi_rs::di          > getting immutable "huber_common::model::config::Config" from DI system
 2023-12-06T01:20:26.527Z INFO  huber::service::repo     > Saving https://raw.githubusercontent.com/DanceMore/huber-dancemore/master/huber.yaml to "/home/neoice/.huber/repos/remote-repo/huber.yaml"
 2023-12-06T01:20:26.532Z DEBUG reqwest::connect         > starting new connection: https://raw.githubusercontent.com/
 2023-12-06T01:20:26.532Z DEBUG reqwest::connect         > proxy(http://localhost:3128) intercepts 'https://raw.githubusercontent.com/'
 2023-12-06T01:20:26.533Z TRACE mio::poll                > registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
 2023-12-06T01:20:26.533Z TRACE reqwest::connect         > tunneling HTTPS over proxy
 2023-12-06T01:20:26.548Z TRACE want                     > signal: Want
 2023-12-06T01:20:26.548Z TRACE want                     > signal found waiting giver, notifying
 2023-12-06T01:20:26.548Z TRACE want                     > poll_want: taker wants!
 2023-12-06T01:20:26.597Z DEBUG reqwest::async_impl::client > response '200 OK' for https://raw.githubusercontent.com/DanceMore/huber-dancemore/master/huber.yaml
 ...

I will see matching logs on Squid for raw.githubusercontent.com too.

[AWDDude]

Here is the trace output when I get when trying to search huber:

> HTTPS_PROXY="http://corporate.proxy.local:8080" huber search -l trace
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding Config { log_level: "TRACE", output_format: Console, home_dir: "/home/awddude/.huber", github_token: None, github_key: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber_common::model::config::Config" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding PackageService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::package::PackageService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding ReleaseService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::release::ReleaseService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding CacheService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::cache::CacheService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding HuberUpdateService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::update::HuberUpdateService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding RepoService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::repo::RepoService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > adding ConfigService { container: None } to DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting mutable "huber::service::config::ConfigService" from DI system
 2024-02-28T18:34:02.306Z DEBUG simpledi_rs::di > initializing DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::package::PackageService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::release::ReleaseService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::cache::CacheService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::update::HuberUpdateService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::repo::RepoService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::config::ConfigService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber_common::model::config::Config" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::cmd::search::SearchCmd" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::package::PackageService" from DI system
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di > getting immutable "huber::service::cache::CacheService" from DI system
 2024-02-28T18:34:02.306Z INFO  huber::service::cache > Updating repos
 2024-02-28T18:34:02.306Z TRACE simpledi_rs::di       > getting immutable "huber_common::model::config::Config" from DI system
 2024-02-28T18:34:02.306Z INFO  huber::service::cache > Updating "/home/awddude/.huber/repos/huber"
⢹ Updating managed repos                                                         2024-02-28T18:34:02.350Z INFO  huber::component::github > Cloning huber github repo
 2024-02-28T18:34:02.350Z DEBUG huber::component::github > Merging huber repo update
⡧ Updating managed repos                                                                                                                                                                                                   2024-02-28T18:36:12.931Z DEBUG huber::component::github > Failed to fetch huber github repo, failed to connect to github.com: Connection timed out; class=Os (2)
 2024-02-28T18:36:12.940Z INFO  huber::component::github > Cloning huber repo via https
error: failed to connect to github.com: Connection timed out; class=Os (2)awddude@awddudelxpc:~/source

As you can see it fails to connect to github.

For comparison here is the output I get when cloning the huber repo using git on the same workstation:

>  HTTPS_PROXY="http://corporate.proxy.local:8080" git clone https://github.com/innobead/huber.git
Cloning into 'huber'...
remote: Enumerating objects: 6489, done.
remote: Counting objects: 100% (992/992), done.
remote: Compressing objects: 100% (467/467), done.
remote: Total 6489 (delta 583), reused 880 (delta 523), pack-reused 5497
Receiving objects: 100% (6489/6489), 1.07 MiB | 7.72 MiB/s, done.
Resolving deltas: 100% (4160/4160), done.

Git works just fine.

Thanks again for looking into this for us.

[sarg3nt]

Gave up.