Do not silently use _spf.DOMAIN.EXT

[meineerde]

Since f838afb, the gem prioritizes a query to _spf.DOMAIN.EXT instead of using a query to the requested domain itself. This produces invalid results in case a domain publishes SPF records on both _spf.DOMAIN.EXT as well as DOMAIN.EXT.

In general, the only valid SPF record for a domain is in the TXT record of the domain itself. Nowhere in the spec (as far as I'm aware) does it specify that the _spf.DOMAIN.EXT record should be used in preference (or is even valid without explicit rules) rather than the record of the domain itself. Per the RFC, if _spf.DOMAIN.EXT is to be used, it has to be referenced on the domain itself via e.g. an include or redirect rule.

Thus, the automatic check for _spf.#{domain} in SPF::Query.query should preferably be removed completely there since it produced incorrect results for some domain. It even contradicts the examples from the README.md for e.g. facebook.com.

If it is not possible to remove this check for any reason, it should at least be de-prioritized from the check of the passed domain itself and controlled via a flag.

I'd be happy to provide a pull request to remove or deprioritize this check for _spf.DOMAIN.EXT.

[postmodern]

RFC7208 does in fact mention _spf and it is commonly used. It used to be that certain domains would only provide a _spf.DOMAIN SPF record. Now, domains such as google.com provide a top-level SPF record which includes _spf.google.com. I think this can be fixed by querying DOMAIN first, then falling back to _spf.google.com if no TXT record containing spf=v1 is found.

[meineerde]

Could you point me to the exact location in RFC 7208 where they state to implicitly fallback to (or even to prefer) _spf.DOMAIN for queries?

From my understanding, the authors of RFC 7208 just use the (indeed common) _spf label throughout in various examples to show how you may include or redirect certain parts of the spec from central locations or how to use macros to build lookups. Section 4.8 specifically talks about macro expansion and the ability to generate legal hostnames containing the _spf label.

Instead, RFC 7208 specifically states that the SPF record has to be published at the exact owner name it belongs to.

Specifically in section 3:

Each SPF record is placed in the DNS tree at the owner name it
pertains to, not in a subdomain under the owner name. This is
similar to how SRV records [RFC2782] are done.

and later more explicitly in section 4.1:

<domain> - the domain that provides the sought-after authorization
information; initially, the domain portion of the
"MAIL FROM" or "HELO" identity.

and section 4.4:

In accordance with how the records are published (see Section 3
above), a DNS query needs to be made for the <domain> name, querying
for type TXT only.

Nowhere (according to my understanding) does RFC 7208 or any of its predecessors such as RFC 4408) specify to implicitly add the _spf label to a domain query. As such, I still believe that it is entirely wrong to consider a fallback to _spf.DOMAIN, regardless of the fallback's priority.

With that being said, it is indeed quite common to publish some parts of a final SPF policy on a hostname like _spf.example.com. This (partial) policy must however be used in an redirect or include rule for the actual SPF record (except for the rare case that someone might actually want to send email from an address such as jane@_spf.example.com). Thus, for this policy to be used, you would usually publish an SPF policy such as

v=spf1 include:_spf.example.com ~all

as a TXT record on the example.com domain.

Again, if I'm wrong, I'm eager to revise my understanding of the RFCs here. I just have not found any mention of such an implicit fallback.

In the case of Planio (the organisation sponsoring my initial pull request in #11), we publish our own primary SPF policy at TXT plan.io. This policy is a superset of _spf.plan.io where we publish some hosts which are allowed to send mail for our main service. Our customers may setup rules to allow us to send mail on their behalfby setting up an include:_spf.plan.io rule to their domains. On our main domain however, we allow additional services such as newsletter services which send mails on behalf of the plan.io domain. These newsletter services however are not relevant to our main service and will never send mail on behalf of our customer domains (hence are not included in _spf.plan.io).

With the current lookup rules in the spf-query gem, you would wrongly check TXT _spf.plan.io instead of TXT plan.io and thus would deny newsletter mails from the plan.io domain from being received.

[rfgil]

I have the same experience @meineerde is describing with the domain pwc.com. Parsing _spf.pwc.com first misses the
includes on pwc.com, thus missing the important _spf.google.com.