Update documentation to reflect prioritization and democratic flow of processing requests

[dnsguru]

Looking for some help in defining an order of processing from @weppos @sleevi to document the factors that impact prioritization and pace of processing requests within our volunteer pool.

The queue and order of PR received for processing is something that I want to try to further define in the documentation.

I'd like to update to the documentation that shows factors impacting processing and potentially identifies the order of processing of updates to the PSL in a manner that should hopefully attract submissions to flow through using the main funnel over other options.

I have reviewed the spectrum of PR and tried to capture the majority of the scenarios that come in to us. The following list is a starting draft to start the discussion of the order of processing (including a category for where it will not be processed)

Glossary:

• 1st party = validated first party by match to admin contact
• 3rd party = non-validated party or community submission
• DNS Validated = the PR number validation record present for DNS _psl TXT validation

The following is the list of submission types we receive

• Any Submissions for entries outside ICP-3 defined through vetted IETF,IANA,ICANN process (ex:.ONION Add onion #386) by 1st or 3rd party
• Automated updates from ICANN v2 json performing additions/removals of gTLDs from ICANN Section of PSL
• ICANN Section add submitted by 1st party ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL that are DNS validated
• ICANN Section update submitted by 1st party gTLD, ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL that are DNS validated
• ICANN Section add submitted by 3st party ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL that are DNS validated
• ICANN Section update submitted by 3st party gTLD, ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL that are DNS validated
• PRIVATE Section add submitted by 1st party that are DNS validated
• PRIVATE Section update submitted by 1st party that are DNS validated
• PRIVATE Section add submitted by 3rd party that are DNS validated
• PRIVATE Section update submitted by 3rd party that are DNS validated
• ICANN Section add submitted by 1st party ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL NOT DNS validated
• ICANN Section update submitted by 1st party gTLD, ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL NOT DNS validated
• ICANN Section add submitted by 3st party ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL NOT DNS validated
• ICANN Section update submitted by 3st party gTLD, ccTLD or IDNccTLD present in IANA root TLD zone file but not in ICANN Section of PSL NOT DNS validated
• PRIVATE Section add submitted by 1st party that are DNS validated
• PRIVATE Section update submitted by 1st party that are DNS validated
• PRIVATE Section add submitted by 3rd party that are DNS validated
• PRIVATE Section update submitted by 3rd party that are DNS validated
• PRIVATE Section deletion submitted by 1st party NOT DNS validated
• PRIVATE Section deletion submitted by 1st party that are DNS validated
• PRIVATE Section deletion submitted by 3rd party NOT DNS validated
• PRIVATE Section deletion submitted by 3rd party that are DNS validated

Worth noting that these things are unlikely to be addressed

• Any Submissions for TLDs in alternative roots (ie not ICP-3 compliant) by 1st or 3rd party
• Any Submissions for IPv4 or IPv6 addresses or ARPA.INT
• Entries whose sole purpose is to circumvent Let's Encrypt rate limits

[dnsguru]

Documentation for assistance in roadmap #671

[sleevi]

This is a very complex ontology, and I'm not sure it's correct. For example, I'm unaware of any examples of 3rd party submissions that are DNS validated. Do you have examples?

[dnsguru]

My .fj pr submission was third party with validated DNS I was thinking maybe if it was a matrix it would be more clear visually, I just dont have even a yellow belt in markdown to make a table for it.

…

On Tue, Feb 25, 2020, 5:13 PM sleevi ***@***.***> wrote: This is a very complex ontology, and I'm not sure it's correct. For example, I'm unaware of any examples of 3rd party submissions that are DNS validated. Do you have examples? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#982?email_source=notifications&email_token=AACQTJP3KD3ZDXHQCENF2Y3REW63JA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6L5EY#issuecomment-591183507>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQTJI6H6VRUFLYO6GBPBLREW63JANCNFSM4K3YSM4A> .

[sleevi]

You’re saying someone other than the registry modified the DNS records? 🤨 Thanks, I’ll look more into it

[dnsguru]

You’re saying someone other than the registry modified the DNS records?

No no no no no... I am saying that I, as a third party to the registry, submitted the PR and the DNS was updated accordingly by the registry.

[dnsguru]

Thinking of it as 4 dimensions - First or Third party, Validated DNS with _PSL (or not), Add/Delete/Modify, and what Section (ICANN or PRIVATE)

• First Party, Validated DNS, Add, ICANN Section
• First Party, Validated DNS, Add, PRIVATE Section
• First Party, Validated DNS, Modify, ICANN Section
• First Party, Validated DNS, Modify, PRIVATE Section
• First Party, Validated DNS, Delete, ICANN Section
• First Party, Validated DNS, Delete, PRIVATE Section
• Third Party, Validated DNS, Add, ICANN Section
• Third Party, Validated DNS, Add, PRIVATE Section
• Third Party, Validated DNS, Modify, ICANN Section (Update fj entry #964)
• Third Party, Validated DNS, Modify, PRIVATE Section
• Third Party, Validated DNS, Delete, ICANN Section
• Third Party, Validated DNS, Delete, PRIVATE Section
• First Party, No DNS Validation, Add, ICANN Section
• First Party, No DNS Validation, Add, PRIVATE Section
• First Party, No DNS Validation, Modify, ICANN Section
• First Party, No DNS Validation, Modify, PRIVATE Section
• First Party, No DNS Validation, Delete, ICANN Section (Removal of education.tas.edu.au #977)
• First Party, No DNS Validation, Delete, PRIVATE Section
• Third Party, No DNS Validation, Add, ICANN Section
• Third Party, No DNS Validation, Add, PRIVATE Section
• Third Party, No DNS Validation, Modify, ICANN Section
• Third Party, No DNS Validation, Modify, PRIVATE Section
• Third Party, No DNS Validation, Delete, ICANN Section
• Third Party, No DNS Validation, Delete, PRIVATE Section

[sleevi]

No no no no no... I am saying that I, as a third party to the registry, *submitted* the PR and the DNS was updated accordingly by the registry. Oh, why would that distinction of who created the PR matter?

All that matters for prioritization is the method of validation, and that should be: - 1P automatic (Including parent domains) - 3P by way of publicly documented registry policies - 3P by way of registry contacts The section shouldn’t make much of a difference, nor who created the PR.

The type definitely matters; additions carry lower risk than removals, and

understanding why additions also matters.

[dnsguru]

On Tue, Feb 25, 2020, 7:42 PM sleevi ***@***.***> wrote: > No no no no no... I am saying that I, as a third party to the registry, > *submitted* the PR and the DNS was updated accordingly by the registry. > > Oh, why would that distinction of who created the PR matter?

I suggest this matters in the instance of those not validated via DNS

All that matters for prioritization is the method of validation, and that should be: - 1P automatic (Including parent domains) - 3P by way of publicly documented registry policies - 3P by way of registry contacts

I suggest that 1P w/o dns _psl fits in there somewhere, but deprioritized amidst the requests that have been able to validate

The section shouldn’t make much of a difference, nor who created the PR.

I can see that

The type definitely matters; additions carry lower risk than removals, and understanding why additions also matters.

I agree that additions seem less prone to risk than deletions Modifications can carry high impact as well, but I reckon we could put del/mod in the same bucket

[weppos]

Personally, I apply the following priority:

1. Requests related to ICANN/IANA suffixes are processed first
2. Anything in PRIVATE afterwards

Within 2. I tend to give a slightly higher priority to issues that modify or remove existing entries. The reasoning behind that is because those requests are generally easier to handle as the contributor has past experience, hence I can expect a lower LOE.

I'm open to hear alternative approaches, but I'd suggest we don't come with a process that introduces more overhead than the one we try to solve. I like to keep things simple. 😉

[dnsguru]

OK, so I think the answer is that there are some dimensions to how we process these. Tweak or comment:

LoE, impacts the pace of processing, validation or merge of a contribution

This project has a number of contributors, most all of whom are volunteers. To best process things in the most efficient manner, the following are some factors and dimensions that can impact the order and pace of processing by the volunteers.

What type of change is being requested?

• Additions typically require lower LoE
• Modifications/Deletions are higher LoE

Contributor frequency (new vs repeat) submitting party might indicate experience or familiarity that may make the reduce the LoE, or may be posting frequent changes indicating circumstances of higher LoE from closer review.

Validation of submission(s): ALWAYS

This has a range of LoE. These are not mutually exclusive validation techniques, and the bias is the DNS validation as it is difficult or impossible to forge. The other methods create a burden of research for the volunteers and severely impact the pace of processing or prioritization.

• On the lower side of LoE, Zone validation via DNS and _PSL TXT RFC8553-style record matching the PR# will help prove that there is a connection between the authoritative admin or domain owner and the submitted PR.
• Documented, publicly available policies or enumeration / explanation of the submission from an authoritative source
• Was this submitted directly from the registry or domain owner?

Request size and complexity can also impact the LoE and pace of processing:

• Is this a modification that moves entries in whole or in part from one section to another section?
• Is this a request that has a lot of domains to validate?
• Are there a number of clarifying questions that the contributor or the volunteers must pose?

[sleevi]

Relative to additions, additions are not all equal.

• Does this have the template filled out in a way that answers the common questions
  • Why is it being requested
  • What are example domains (this is particularly important with respect to wildcards and doing the right thing)
• How many domains are being requested
• Are the domains being requested for internal purposes (e.g. organization Foo wants their dev/staging servers, which only they use/access) or for general utility
  • Basically, does every user shipping the PSL benefit from the added size/processing?
• What's the registration period for the domain (i.e. the risk of churning)

I totally admit, I'm not thrilled with having to ask these questions. They're an unfortunate and necessary evil involved with a static list, which is a result of cookies being scoped to "domains" (boundaries) and not merely hosts.

[dnsguru]

Good material @sleevi Covers some of what I doc'd and then some. I will integrate it after a night's rest.

…

On Thu, Feb 27, 2020, 3:57 PM sleevi ***@***.***> wrote: Relative to additions, additions are not all equal. - Does this have the template filled out in a way that answers the common questions - Why is it being requested - What are example domains (this is particularly important with respect to wildcards and doing the right thing) - How many domains are being requested - Are the domains being requested for internal purposes (e.g. organization Foo wants their dev/staging servers, which only they use/access) or for general utility - Basically, does every user shipping the PSL benefit from the added size/processing? - What's the registration period for the domain (i.e. the risk of churning) I totally admit, I'm not thrilled with having to ask these questions. They're an unfortunate and necessary evil involved with a static list, which is a result of cookies being scoped to "domains" (boundaries) and not merely hosts. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#982?email_source=notifications&email_token=AACQTJPAVC6GYRISOZZJYMDRFBHNZA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENGNMPI#issuecomment-592238141>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQTJJEMKE6XPBOJSYWEP3RFBHNZANCNFSM4K3YSM4A> .

[dnsguru]

I appreciated the inputs @weppos and @sleevi. I want to keep it simple, but we have to also recognize that many of the people submitting issues or PR are not familiar with some of the factors that can reduce or increase the success and time requirements of them.

Here is the latest, incorporating more inspired from the thread and our empirical experience, in the hopes we're helping users better understand how they can best compose their PR by providing us important information, and have a sense for factors that can impact the time or success.

New Version, Below

LoE, impacts the pace of processing, validation or merge of a contribution

This project has a number of contributors, most all of whom are volunteers. To best process things in the most efficient manner, the following are some factors and dimensions that can impact the order and pace of processing by the volunteers.

What type of change is being requested?

• Additions typically require lower LoE than Modifications/Deletions; However...
• Modifications/Deletions are higher LoE

All additions secnarios are not equal, and there are factors that can cause things to take longer or not happen (see "Information furnished in the PR template")

Contributor frequency (new vs repeat) submitting party might indicate experience or familiarity that may make the reduce the LoE, or may be posting frequent changes indicating circumstances of higher LoE from closer review.

Information furnished in the PR template (THOROUGH)

Questions and clarifications, research or review may or may not be needed, based upon the completeness, clarity, and accuracy of the submission.

• Does the PR have the template filled out completely and clearly in a way that answers the common questions
  • Why is the PR being requested? Please be detailed, but not so detailed as to lose the reviewer(s) in the weeds.
  • What are example domains' use scenarios? Describe the desired outcome. Enumerate through example subdomains or the name iteself and how these are desired to work as a result of the PR being merged. This is particularly important with respect to wildcards (*) and bangs (!) and ensuring the result will align with the request.
• How many domains are being requested within the PR?
• Are the domains being requested for internal purposes (e.g. organization Foo wants their dev/staging servers, which only they use/access) or is this for general utility
  • Basically, does every user shipping the PSL benefit from the added size/processing?
• What's the registration period for the domain (if in the PRIVATE section)? (i.e. the risk of churning)

Validation of submission(s): (ALWAYS)

This has a range of LoE. These are not mutually exclusive validation techniques, and the bias is the DNS validation as it is difficult or impossible to forge. The other methods create a burden of research for the volunteers and severely impact the pace of processing or prioritization.

• On the lower side of LoE, Zone validation via DNS and _PSL TXT RFC8553-style record matching the PR# will help prove that there is a connection between the authoritative admin or domain owner and the submitted PR.
• Documented, publicly available policies or enumeration / explanation of the submission from an authoritative source
• Was this submitted directly from the registry or domain owner? (if Registry, does this match IANA database in some verifiable manner)

The first one is the most crucial. The other two are helpful. Having all three makes things go VERY quickly in the ICANN section.

Request size and complexity can also impact the LoE and pace of processing:

• Is this a modification that moves entries in whole or in part from one section to another section?
• Is this a request that has a lot of domains to validate?
• Are there a number of clarifying questions that the contributor or the volunteers must pose?

[dnsguru]

Updated the Guidelines based upon this issue. @sleevi @weppos take a look please.

[weppos]

Additions typically require lower LoE than Modifications/Deletions
Modifications/Deletions are higher LoE

Can you clarify? It's actually exactly the opposite for me. It's faster to deal with a modification than a brand new request. In most cases, same apply to deletion.

[dnsguru]

Could you propose some alternative text? This was me attempting to reword the following from @sleevi

The type definitely matters; additions carry lower risk than removals, and understanding

why additions also matters. Perhaps I could word my incorporation of that differently, as I had mashed deletes and modifications together, and may also be confusing "risk" with "loe" I think mods from the same party that had previously submitted PR for same string might carry a simpler validation of the requestor. So I see your point, and in sleevi's, he wants to ensure that the request itself has been considered. In the end, I want the documentation to be accurate but don't want to overthink it.

…

On Mon, Mar 2, 2020, 6:25 AM Simone Carletti ***@***.***> wrote: Additions typically require lower LoE than Modifications/Deletions Modifications/Deletions are higher LoE Can you clarify? It's actually exactly the opposite for me. It's faster to deal with a modification than a brand new request. In most cases, same apply to deletion. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#982?email_source=notifications&email_token=AACQTJP4CL4QK3KR23CFMBLRFO6U3A5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPPXSQ#issuecomment-593427402>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQTJKQJQDEBMRTNPRXF43RFO6U3ANCNFSM4K3YSM4A> .

[weppos]

Perhaps I need more context from @sleevi . From my experience:

1. An addition is higher LOE because often comes with an education effort. I spend extra time looking into potential side effects the submitter did not consider, sometimes teaching the submitter about incorrect submissions or unexpected behaviors.
2. Modifications (in terms of editing an existing entry or adding rules to an existing block) have lower LOE to me because I generally can skip the educational aspect as the submitter is in most cases already knowledgeable of the process.

I guess at this point I'm not sure whether stating the LOE in the guidelines has any valid interpretation from the eyes of the submitter. In other words, how is this information relevant?

[sleevi]

On Tue, Mar 3, 2020 at 7:44 AM Simone Carletti ***@***.***> wrote: Perhaps I need more context from @sleevi <https://github.com/sleevi> . From my experience: 1. An addition is higher LOE because often comes with an education effort. I spend extra time looking into potential side effects the submitter did not consider, sometimes teaching the submitter about incorrect submissions or unexpected behaviors. 2. Modifications (in terms of editing an existing entry or adding rules to an existing block) have lower LOE to me because I generally can skip the educational aspect as the submitter is in most cases already knowledgeable of the process. I saw those as additions. Modifications were modifications to actual

entries - e.g. adding or removing wildcards. I agree additions carry much less risk (and any risk is, well, tough for the domain holder), while removals and modifications carry more risk to end users, which I very much care about. Loss of privacy/security is, to me, worse than denial of service.

I guess at this point I'm not sure whether stating the LOE in the guidelines has any valid interpretation from the eyes of the submitter. In other words, how is this information relevant?

agreed. I didn’t realize this was being proposed for the Guidelines, vs informal documentation. I’ve been hesitant to say something, as I worry it would seem like I’m adding incessant stop-energy, but I’m not a fan 😅😞

[dnsguru]

To me, I think the goal had been to encourage complete entries where all the steps were performed and have thorough information included with each PR, and to help submitters appreciate factors that might contribute to how fast or slow things happen - in the amount of time it takes to process their requests. If there is a better way to state something, or there is a disagreement with what was written, I do not have to be the only editor on these :)

…

On Tue, Mar 3, 2020, 4:59 AM sleevi ***@***.***> wrote: On Tue, Mar 3, 2020 at 7:44 AM Simone Carletti ***@***.***> wrote: > Perhaps I need more context from @sleevi <https://github.com/sleevi> . > From my experience: > > 1. An addition is higher LOE because often comes with an education > effort. I spend extra time looking into potential side effects the > submitter did not consider, sometimes teaching the submitter about > incorrect submissions or unexpected behaviors. > 2. Modifications (in terms of editing an existing entry or adding > rules to an existing block) have lower LOE to me because I generally can > skip the educational aspect as the submitter is in most cases already > knowledgeable of the process. > > I saw those as additions. Modifications were modifications to actual entries - e.g. adding or removing wildcards. I agree additions carry much less risk (and any risk is, well, tough for the domain holder), while removals and modifications carry more risk to end users, which I very much care about. Loss of privacy/security is, to me, worse than denial of service. > I guess at this point I'm not sure whether stating the LOE in the > guidelines has any valid interpretation from the eyes of the submitter. In > other words, how is this information relevant? > agreed. I didn’t realize this was being proposed for the Guidelines, vs informal documentation. I’ve been hesitant to say something, as I worry it would seem like I’m adding incessant stop-energy, but I’m not a fan 😅😞 — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#982?email_source=notifications&email_token=AACQTJKQTTZQ2YQ6YOAFVKDRFT5LRA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENTMLGI#issuecomment-593937817>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQTJPZFCLQGR3NXVD67VLRFT5LRANCNFSM4K3YSM4A> .

[dnsguru]

Gonna close this one for now. Seems reasonably well addressed in the update I posted.