Feature request - resolve IPs to DNS names

[williamdes]

There is two things I did find frustrating when reading reports today:

• When I click on the IP link all the information is useless to me as I would have needed
  • It's reverse DNS name
  • What MX records exist for this reversed DNS name
• reverse DNS support when displaying an IP, since an IP is not very informative when browsing my clients reports
  I need to have a visual way to find out what service is behind the IP in the report without having to open a ton of tabs

So what about reverse DNS support ?

[liuch]

When I click on the IP link all the information is useless to me as I would have needed

FYI: You can set up another service to display this information.

I need to think how and where to accurately display this information. BTW: there is already a suitable pull request: #5

[KlaasBonnema]

The current link to whois from an IP address link does not give much usefull information. Mostly it gives the owner information of a complete ip block. What you want to know is if there is a proper mail server behind the ip-address. If it resolves to something funny or does not resolve at all then it is time to start digging (with another tool maybe.)

My suggestion is to create a link that will attempt to resolve a domain name from the ip-address and display it as a popup. Opening up a new page is too distracting for a quick check.
Alternative is to resolve all ip-addresses and add as a title attribute when you create the HTML page. This may however incur a lot of dns overhead and delays.

Note that an ip-address to domain lookup may not come up with the actual domain because there may be hundreds of domains associated with the ip-address. However since the ip-reverse check is very common in determining valid mail servers you may expect that proper mail handlers will set this up in a decent way.

[KlaasBonnema]

I made some changes that will do a hostname lookup on server IP and then validate that a reverse lookup for ip-address using the hostname returns the original ip-address. A failing lookup of ip-address match will mark the ip-address - hostname entry as red while a match will mark the entry as green. This is similar to the ip-rev check that mail receivers will perform to spot improper mail senders.
The actual lookup is being performed by a PHP script. It is called in parallel by an asynchronous fetch so that the dns lookups are executed also in parallel and will not adversly impact page load times.

common.js

	static makeIpElement(ip) {
		let url = null;
		let type = ip.includes(":") && 6 || 4;
		switch (type) {
			case 4:
				url = Common.ipv4_url;
				break;
			case 6:
				url = Common.ipv6_url;
				break;
		}
		let tn = document.createTextNode(ip);
		if (url) {
			url = url.replace("{$ip}", ip).replace("{$eip}", encodeURIComponent(ip));
			let el = document.createElement("a");
			el.setAttribute("href", url);
			el.setAttribute("target", "_blank");
			el.setAttribute("title", "IP address - hostname");
			el.appendChild(tn);
			this.resolveIp(ip, type)
				.then((response) => {
					el.innerHTML = el.innerHTML + " - " + response.hostname;
					if (response.status == 'valid')
						el.setAttribute("class", 'report-result-pass');
					else
						el.setAttribute("class", 'report-result-fail');
				})
				.catch(console.log);
			return el;
		}
		else {
			this.resolveIp(ip, type)
				.then((response) => {
					tn.textContent = tn.textContent + " - " + response.hostname;
					if (response.status == 'valid')
						tn.parentNode.setAttribute("class", 'report-result-pass');
					else
						tn.parentNode.setAttribute("class", 'report-result-fail');
				})
				.catch(console.log);
		}
		return tn;
	}

	static resolveIp = async (ip, type) => {
		const response = await fetch("resolve_ip.php?ip=" + ip + "&type=" + type);
		if (response.ok) {
			return response.json();
		}
	};

resolve_ip.php

<?php
/**
 * Resolve ip-address to hostname and perform reverse lookup to match resolved hostname with ip-address.
 * Mail servers should be set up in dns with a dedicated static hostname. 
 * Flag invalid and non-matching ip-address - hostname pairs.  
 * @var string $ip
 */
$response = array();
$ip = $_GET['ip'];
$type = $_GET['type'];

// empty $ip parameter is invalid
if (empty($ip))
{
    $response['status'] = "invalid";
    $response['hostname'] = "hostname lookup invalid";
    echo json_encode($response);
    exit();
}

// get the hostname by ip
$hostname = gethostbyaddr($ip);
if ($hostname === false)
{
    $response['status'] = "invalid";
    $response['hostname'] = "hostname lookup failed";
    echo json_encode($response);
    exit();
}

// lookup dns to get an array of ip_addresses by hostname
switch ($type)
{
    case "6":
        {
            // lookup ipv6 
            $ip_res = dns_get_record($hostname, DNS_AAAA);
            break;
        }
    case "4":
    default:
        {
            // lookup ipv4
            $ip_res = dns_get_record($hostname, DNS_A);
            break;
        }
}

// invalid if result is not an array
if (! is_array($ip_res))
{
    $response['status'] = "invalid";
    $response['hostname'] = $hostname;
    echo json_encode($response);
    exit();
}

// match our input ip with the result ip's. A match returns a valid status
foreach ($ip_res as $entry)
{
    if (($type == "6" && isset($entry['ipv6']) && inet_ntop(inet_pton($entry['ipv6'])) == $ip) || (isset($entry['ip']) && $entry['ip'] == $ip))
    {
        $response['status'] = "valid";
        $response['hostname'] = $hostname;
        echo json_encode($response);
        exit();
    }
}

// no match found, return invalid status
$response['status'] = "invalid";
$response['hostname'] = $hostname;
echo json_encode($response);
exit();

[liuch]

I will try to implement something like this. At the very least, your php code lacks input parameter validation and access level validation. There are questions to the js code as well. But the main thing is your idea. Thank you.

[williamdes]

https://github.com/domainaware/parsedmarc/blob/8.17.0/parsedmarc/resources/maps/base_reverse_dns_map.csv

To map domains to ISP names

[liuch]

What MX records exist for this reversed DNS name

Probably not one. The hostname will most likely contain the domain name, but does not have to be identical to the domain. If configured correctly, it must not match. For example, gmail email hosts don't even always have the domain part matching.

In a recent email with a sender on the gmail server, from the Received header I took this IP address: 209.85.220.41
Checking:

$ host 209.85.220.41
41.220.85.209.in-addr.arpa domain name pointer mail-sor-f41.google.com.

$ dig +short mx mail-sor-f41.google.com
$

Nothing. Perhaps I misunderstood something and you meant a different check. Could you please share your thoughts?

[KlaasBonnema]

I am running with my suggested code for several month's now. It will do a dynamic reverse lookup to get the dns name and then a dns name to ip lookup to verify it matches the original ip-address. Essentially this is the ip-rev check that mail servers perform.
The code displays the dns name together with the ip-address in summary reports and incomming record reports. If the ip-rev check fails the name will be in red.
This gives me a quick view of the mail server that processed the request.

In practice you will see many names matching your own mail provider's servers. They will probably look something like mo4-p00-ob.smtp.provider.com where the "provider.com" part is identical. The vast majority of servernames will be from a very small number of providers.

A better way to do this is to perform the ip-rev check when processing the incomming reports. Before the ip-rev check, lookup the ip-address in a database table of previously found adresses where you store the full dns name. This will further avoid the overhead of doing the ip-rev checks.
Then allow the user to configure an alias name for addresses matching a common "provider.com" part.

In the summary report you can create an option to group all servers matching a common "provider.com" together. Servernames that were not yet assigned to a common provider name will stick out, which is what you want to spot exceptions.
In incomming records you would display ip-address, full dns name and the common provider name alias.

A further enhancement could be to collect mx records. Question then is how you would detect a change in mx configurations.

Using a big provider list cannot be maintained. The list provided by Williamdes does not contain any of the providers that I commonly see.

[liuch]

@KlaasBonnema, thank you for your reply. Unfortunately, the method you described does not work for every configuration. The host that sends mail and the host that receives mail may be in different domains. IP addresses in the result of gethostbyaddr -> dns_get_record may not match.

A little later I will add the hosts table to the database, but now I'm trying to figure out what host information will be most useful in the Web UI report view dialog.

[KlaasBonnema]

"The host that sends mail and the host that receives mail may be in different domains. IP addresses in the result of gethostbyaddr -> dns_get_record may not match."

Do you mean that you may not be able to resolve the smtp server ip_address reported by dmarc because it is not accessable by the server that runs dmarc-srg (behind a firewall)? Dmarc-srg unable to do dns resolve requests would indeed block any informational lookup.

The smtp server itself behind a firewall is unlikely because it would not be able to do its job.

In recent reports I have two ip_addresses that did not pass the ip-rev test. One has a japanese domain name but no matching ip-address. The other does not even resolve to a domain name. Both requests were dmarc rejected. It is easy to identify these two attempts as illegal use of our domain for sending e-mail using a faked ip_address.
The other hundreds of smtp server ip-addresses can be grouped in no more than a dozen provider domains.

[liuch]

I apologize, I seem to be confused myself in my experiments with rDNS. I guess I need to write tests and check my conclusions on more reports.

[liuch]

I have currently implemented this functionality as follows:

1. IP address
2. rDNS name for this IP
3. Reverse IP (match / not match)
4. Total reports for this IP address (all domains available to the user)
5. Total messages for this IP address (all domains available to the user)
6. Last report containing this IP address excluding the current report, if possible (all domains...)

A request to the server is sent only when the corresponding button is pressed. This is implemented so as not to overload the server with numerous requests if there are many records in the report. Perhaps later requests to the server will be sent as the report page scrolls or something.

rDNS and Reverse IP are cached for 24 hours for the current browser tab. Caching on the server is not implemented yet. Ideally, these requests should be cached by DNS resolver installed on the server.

[liuch]

As for the summary report - I haven't figured out how to do it properly and safely yet.

[williamdes]

This looks really nice !

rDNS and Reverse IP are cached for 24 hours for the current browser tab. Caching on the server is not implemented yet. Ideally, these requests should be cached by DNS resolver installed on the server.

Can you make the code in order to allow a community plugin to be added for the resolving process ?
I guess such a plugin could enrich the data shown in the popup. But also do the same work but with server side caching or fetch from another source than DNS

A class that can be extended could work, I am not sure. Do what is best for you :)

PS: For example adding the ISP name: https://github.com/domainaware/parsedmarc/blob/8.17.0/parsedmarc/resources/maps/base_reverse_dns_map.csv

[liuch]

@williamdes I thank you for your appreciation of this functionality. As for the plugin manager: I like the idea, but the problem is that I have no idea yet how it should be implemented. The ones I googled in a few minutes I didn't like. I am going to familiarize myself with this subject more deeply in the near future.

[williamdes]

@williamdes I thank you for your appreciation of this functionality. As for the plugin manager: I like the idea, but the problem is that I have no idea yet how it should be implemented. The ones I googled in a few minutes I didn't like. I am going to familiarize myself with this subject more deeply in the near future.

Sure!

You could provide a php interface file and check that the plug-in implements it.

foreach files in plug-in folder
if instance of
then wire into the code