Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: optional flag to skip a host if it has a private IP address. #1408

Merged
merged 6 commits into from
Nov 2, 2023

Conversation

CodyCline
Copy link
Contributor

Implements feature request outlined in #1407.

Copy link
Member

@dogancanbakir dogancanbakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

$ go run . -u localhost,scanme.sh -eph -debug

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

                projectdiscovery.io

[INF] Current httpx version v1.3.6 (latest)
[DBG] Skipping private host localhost
[DBG] Failed 'localhost': target has a private ip and will only connect within same local network
[INF] Dumped HTTP request for https://scanme.sh

GET / HTTP/1.1
Host: scanme.sh
User-Agent: Mozilla/5.0 (compatible; YandexNews/4.0; +http://yandex.com/bots)
Accept-Charset: utf-8
Accept-Encoding: gzip

[INF] Dumped HTTP response for https://scanme.sh

HTTP/1.1 200 OK
Connection: close
Content-Length: 2
Content-Type: text/plain; charset=utf-8
Date: Thu, 19 Oct 2023 11:19:45 GMT

ok
https://scanme.sh

@ehsandeep ehsandeep linked an issue Oct 30, 2023 that may be closed by this pull request
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR @CodyCline, it works fine without port input, but it still passthrough when a port is specified in input or via port option.

./httpx -u localhost -sc -title -p 8080 -eph

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

		projectdiscovery.io

[INF] Current httpx version v1.3.5 (latest)
http://localhost:8080 [200] [Burp Suite Professional]
~/Github/httpx $ ./httpx -u localhost -sc -title -p 8080 -eph

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

		projectdiscovery.io

[INF] Current httpx version v1.3.5 (latest)
http://localhost:8080 [200] [Burp Suite Professional]

@CodyCline
Copy link
Contributor Author

CodyCline commented Oct 30, 2023

@ehsandeep nice catch! This happens because I was passing URL.Host instead of URL.Hostname() into the skipPrivateHosts function. The latter option strips out the port number from the host before calling the DNS lookup. Fix committed now. The output should look similar to this:

go run cmd/httpx/httpx.go -u localhost,scanme.sh -sc -title -p 8080 -eph -debug

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

                projectdiscovery.io

[INF] Current httpx version v1.3.5 (outdated)
[DBG] Skipping private host localhost:8080
[DBG] Failed 'localhost:8080': target has a private ip and will only connect within same local network
[INF] Dumped HTTP request for https://scanme.sh:8080

GET / HTTP/1.1
Host: scanme.sh:8080
User-Agent: HTC-ST7377/1.59.502.3 (67150) Opera/9.50 (Windows NT 5.1; U; en) UP.Link/6.3.1.17.0
Accept-Charset: utf-8
Accept-Encoding: gzip

[INF] Dumped HTTP request for http://scanme.sh:8080

GET / HTTP/1.1
Host: scanme.sh:8080
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36
Accept-Charset: utf-8
Accept-Encoding: gzip

[DBG] Failed 'http://scanme.sh:8080': GET http://scanme.sh:8080 giving up after 1 attempts: Get "http://scanme.sh:8080": [:RUNTIME] ztls fallback failed <- dial tcp [2400:6180:0:d0::91:1001]:8080: connect: network is unreachable

Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build fails is known and unrelated to this PR.

@ehsandeep ehsandeep merged commit 7dee998 into projectdiscovery:dev Nov 2, 2023
10 of 11 checks passed
@CodyCline
Copy link
Contributor Author

Build fails is known and unrelated to this PR.

Thanks for clarifying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optional flag to skip host with a private IP address
3 participants