Permanently Deleted
Don't use google's and cloudflare's nameservers at the same time. Use Google (
8.8.8.8
and8.8.4.4
), cloudflare (1.1.1.1
and1.0.0.1
), or quad9 (9.9.9.9
), but don't mix them.Sometimes Cloudflare is down, sometimes Quad9 is down, etc. When it is, switch to someone else temporarily.
uname -a
curl -v letsencrypt.org
traceroute letsencrypt.org
sudo certbot renew --dry-run
share output from these and recent log entries from
/var/log/letsencrypt/
.spoiler
$ uname -a Linux [redacted] 6.7.1-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 21 Jan 2024 22:14:10 +0000 x86_64 GNU/Linux
$ curl -v letsencrypt.org * Host letsencrypt.org:80 was resolved. * IPv6: 2a05:d014:275:cb02::c8, 2a05:d014:275:cb00::c8 * IPv4: 3.72.140.173, 3.70.101.28 * Trying 3.72.140.173:80... * Connected to letsencrypt.org (3.72.140.173) port 80 > GET / HTTP/1.1 > Host: letsencrypt.org > User-Agent: curl/8.5.0 > Accept: */* > < HTTP/1.1 301 Moved Permanently < Content-Type: text/plain; charset=utf-8 < Date: Sun, 03 Mar 2024 14:25:33 GMT < Location: https://letsencrypt.org/ < Server: Netlify < X-Nf-Request-Id: 01HR2B9B0D8VKSFR5K5ES23ZKA < Content-Length: 39 < * Connection #0 to host letsencrypt.org left intact Redirecting to https://letsencrypt.org/
$ traceroute letsencrypt.org traceroute to letsencrypt.org (3.70.101.28), 30 hops max, 60 byte packets 1 * * * 2 * * * 3 ae0.984-2.edge01.egh.as49581.net (80.91.223.29) 0.650 ms 0.614 ms 0.578 ms 4 ae0.1176-2.edge01.egh.as49581.net (80.91.223.27) 1.064 ms 0.936 ms 0.888 ms 5 92.223.127.44 (92.223.127.44) 3.430 ms * 3.374 ms 6 be3458.ccr42.ams03.atlas.cogentco.com (154.54.39.185) 4.126 ms ae-8.a01.amstnl07.nl.bb.gin.ntt.net (157.238.227.144) 3.699 ms * 7 amsix02-ams1.amazon.com (80.249.210.217) 3.831 ms ae-3.r21.amstnl07.nl.bb.gin.ntt.net (129.250.7.88) 4.405 ms ae-3.r20.amstnl07.nl.bb.gin.ntt.net (129.250.7.86) 4.925 ms 8 ae-1.a00.amstnl07.nl.bb.gin.ntt.net (129.250.7.71) 4.192 ms 54.239.114.46 (54.239.114.46) 4.851 ms 52.93.112.12 (52.93.112.12) 4.443 ms 9 ae-0.amazon.amstnl07.nl.bb.gin.ntt.net (129.250.207.114) 4.707 ms 54.239.114.101 (54.239.114.101) 24.337 ms ae-0.amazon.amstnl07.nl.bb.gin.ntt.net (129.250.207.114) 4.386 ms 10 52.93.112.161 (52.93.112.161) 5.052 ms 52.93.112.153 (52.93.112.153) 4.747 ms * 11 54.239.114.105 (54.239.114.105) 3.949 ms * 54.239.114.31 (54.239.114.31) 7.732 ms 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 * * * 25 * * * 26 * * * 27 * * * 28 * * * 29 * * * 30 * * *
$ sudo certbot renew --dry-run Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/tankietanuki.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Account registered. Simulating renewal of an existing certificate for tankietanuki.com and 12 more domains Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems: Domain: files.tankietanuki.com Type: dns Detail: DNS problem: NXDOMAIN looking up A for files.tankietanuki.com - check that a DNS record exists for this domain; DNS problem: NXDOMAIN looking up AAAA for files.tankietanuki.com - check that a DNS record exists for this domain Domain: matrix.tankietanuki.com Type: dns Detail: DNS problem: NXDOMAIN looking up A for matrix.tankietanuki.com - check that a DNS record exists for this domain; DNS problem: NXDOMAIN looking up AAAA for matrix.tankietanuki.com - check that a DNS record exists for this domain Domain: toot.tankietanuki.com Type: dns Detail: DNS problem: NXDOMAIN looking up A for toot.tankietanuki.com - check that a DNS record exists for this domain; DNS problem: NXDOMAIN looking up AAAA for toot.tankietanuki.com - check that a DNS record exists for this domain Domain: ttk.sh Type: unauthorized Detail: 2606:4700:3035::ac43:85e2: Invalid response from http://ttk.sh/.well-known/acme-challenge/f8CpDtPROuez3hIMxEa4moZYGGVERGrASN2tdMFRO-E: 409 Domain: www.ttk.sh Type: unauthorized Detail: 2606:4700:3035::ac43:85e2: Invalid response from http://www.ttk.sh/.well-known/acme-challenge/2ScBqEvqE6r_FJcHEtJ0WSZZmeFOV3-iTidV1inRL9I: 409 Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet. Failed to renew certificate tankietanuki.com with error: Some challenges have failed. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - All simulated renewals failed. The following certificates could not be renewed: /etc/letsencrypt/live/tankietanuki.com/fullchain.pem (failure) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 renew failure(s), 0 parse failure(s) Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
When I transfered the DNS nameserver to cloudfare, I chose to enable a couple https-only settings that it recommended but I didn't fully understand.
I'll edit this to add the logs soon.
Two options:
- Disable the cloudflare shit entirely - you can let it host your DNSes but that's it (what I'd suggest). It'll be an instant fix.
- Update all your vhosts to make them match the fact that cloudflare is now between your visitors and your webserver. This also means you'll have to deploy Cloudlare's certs, not let's encrypt ones.
I'd also remove this entire post, by the way; I haven't checked but it likely gives too much info about you. I'll happily help you with either of those two options in DM, or matrix, if you want.
So, you can resolve letsencrypt.org, since the ping shows the resolved IP. But you can't get ICMP pings back from it. not all servers/firewalls allow ICMP so that isnt necessarily a problem
changing the nameservers that serve as the canonical source for your own domains records doesnt mess with how your server resolves other people's domains
as for what broke the site thats harder to say, especially now that its fixed, and without more details than "it broke"
I thought I could ping letsencrypt.org on my desktop but I may have been mistaken.
Along with the nameserver switch it prompted me for a few settings which I enabled.
One of them was brotli compression and I'm trying to find the other two. Something related to https-only.The settings were brotli compression, Always Use HTTPS, and Automatic HTTPS Rewrites.According to one peertube user, longer videos are not working. I'm trying to get more details from the logs.
Edit: From the nginx error log:
2024/02/25 15:27:57 [error] 423#423: *11436 open() "/var/www/peertube/storage/streaming-playlists/hls/f9f7ec9a-15a6-4c5a-a250-78dbbb8905d8/d5f36d96-bd3b-4561-a553-4d1278507489-720-fragmented.mp4" failed (2: No such file or directory), client: [redacted], server: tankie.tube, request: "GET /static/streaming-playlists/hls/f9f7ec9a-15a6-4c5a-a250-78dbbb8905d8/d5f36d96-bd3b-4561-a553-4d1278507489-720-fragmented.mp4 HTTP/1.1", host: "tankie.tube" 2024/02/29 15:01:11 [error] 423#423: *87174 open() "/var/www/peertube/storage/web-videos/0facf2a0-ccad-4fcd-b0b6-d4319c5d422e-360.mp4" failed (2: No such file or directory), client: [redacted], server: tankie.tube, request: "GET /static/web-videos/0facf2a0-ccad-4fcd-b0b6-d4319c5d422e-360.mp4 HTTP/2.0", host: "tankie.tube", referrer: "https://tankie.tube/w/gtvBq8FFzeBWBaBW3mAjYA"
Looks like peertube is looking for files that don't exist (anymore)?