Run only one order at a time #74
Comments
|
I’ll need to investigate this more. The desired behavior is how I thought I designed it (it was originally done due to wildcards and it was working). I’ll need to look back at it since the cname alias option was added later. |
|
This actually was working as intended. However, it seemed like there was a minor (but consistent) "hangover" when reusing the same domain for a challenge with a CNAME. I'm not sure why it didn't impact wild card certificates when not using a CNAME, which was the original reason I had to build it like this. I couldn't find any elegant solution other than to add a delay when re-using the same domain / validation resource. This consistently fixed the issue in testing. |
|
Building the new release now. Please let me know if you encounter this issue again. |
|
Sorry for the lag, but I can confirm this doesn't resolve the issue for now. I've tried with another certificate that have the same problematic :
But after one retry, it worked successfully: |
|
Interesting. I’m not certain that is the same exact issue. All of those domains have CNAME records to the same singular domain? Did you create any CNAME records just before running the attempt to generate a certificate (in particular the smtp subdomain)? Who is your DNS provider? I might need to test more specifically with the same provider to see what might be going on. |
|
Yes, all ours domains point to the same CNAME for acme-challenge : No, all these CNAME records are pretty olds (few years I would say), I'm migrating from a standalone script that use lego cli. And it was working fine with this script. We use the dns01goacme provider, which update our BIND9 DNS server using RFC2136. The TSIG key can only update |
|
I'm really not totally sure what's going on in your case. The fact it didn't ever propagate in your most recent log but then worked fine on retry is really throwing me off. (edit: I guess your prior logs show that too which is sort of strange.) I've got a couple ideas for additional checks to implement to try and properly fix this. You may also want to try to switch to the |
|
Can you provide the full log from the failed job (from start of job through the job being complete)? For the future, please edit the config.yaml to set Please also send a screenshot of how your aliases are configured:
|
|
Hi, using a single CNAME as delegation for an _acme-challenge on multiple domains is normally not practical. You are relying on sequential response so you don't clobber previous challenges, so you need to wait at least a minute after each update then submit your challenge response then move on to the next challenge response. You may have gotten this to work with some other ACME tool in the past bu it definitely doesn't scale as you add more domains. What you can do is CNAME each domain to it's own record, e.g. if you had an example subdomain You can also use tools like acme-dns or compatible services to handle your delegated challenge response CNAMEs |
|
I agree the situation isn't ideal and it might not be something I can fully fix. I do think this is worth at least a little more exploring though as I do think the challenge handling could be updated a little bit for better results for everyone (and perhaps some error messages for specific issues). I'm already considering 1) add a CNAME record check when aliases are in use (so an error can be returned to users if their DNS records aren't correct) and 2) add a pre-check to verify the expected challenge dns name doesn't already exist prior to provisioning and to either sleep to see if the issue resolves or to abort and immediately return an error (I'm leaning toward the former and returning the latter after the issue isn't resolved after a sane timeout). |
|
It depends a little on your DNS providers as to whether you have fine grained control over how they work, in an ideal world you would allow a TXT record to have 2 values and discard the oldest/first value whenever you need to add a new one, that way wildcard + apex domain validation works first time without a cached validation and optionally in parallel. For all DNS providers, adding the TXT value takes time for the values to reach all the nameservers for that domain (the CA will check some or all of them for the same answer), likewise removing it. If you had a particularly slow DNS vendor (most are synced within 1 minute, some are up to 5 minutes) and you want to add the TXT, then wait for nameserver propagation, then get the challenge validated, then clean up the record before trying the new one then you are up to 10 minutes per domain. You can cut that by making your cleanup of the old value be a replacement instead of a removal. As a worst case scenario if you had 100 subdomain domains it could take 8 hrs (best case 1hr 40mins) to renew that cert using sequential DNS updates to the same zone (or if working sequentially across many zones within the same cert order). Note that in my own app that does this the approach has grown over time and can be different depending on the DNS provider being used, so I can't talk about best practises :) in an ideal world we would have one DNS API and multi-value TXT records would be easier and consistently handled, then you could validate most certs in one shot. |
In my case, I use a CNAME record for acme challenge entries, by doing this, I should only run on order at a time.
As of today, if many order runs at the same time, only one will be successful.
Could you add something such as a way to reduce the number of workers?
Thank you by advance :)
Also seems like that certificates with multiple domains (and again same CNAME for all) are stuck while doing challenge.
The text was updated successfully, but these errors were encountered: