Infrastructure

I think at some point we should give up the DNS aliases and just move to the ISPs canonical name, then this ticket is also handled alongside

The majority of our domains is now connected to KeyDNS at RRPproxy, including DNSSEC signing
The main/production domains are not changed yet
The existing zones at Hetzner are not deleted yet
Glue records are updated, but (obviously) not deleted yet

For the DE domains, the DNSSEC keys have to be manually pushed, the automated push does not work
For some domains, pushing DNSSEC keys doesn't seem to work at all, chasing this one already

Guilhem and me to coordinate on the next steps

Domains where pushing DNSSEC does not work are the following. Let's try again tomorrow, if that fails, we need to reach out to support:

documentfoundation.fi
Pushing keys failed. Please try again later or contact support. Invalid attribute value syntax

libreoffice.cz
Pushing keys failed. Please try again later or contact support. Command failed; Command failed

libreoffice.fi
Pushing keys failed. Please try again later or contact support. Missing required attribute; missing tech contact

libreoffice.it
Pushing keys failed. Please try again later or contact support. Invalid attribute value [extepp:wrongvalueextepp:reasoncode => 10011: DNSSEC: dsData to add is already associated with the domain]

Current status:

Domains where pushing DNSSEC does not work are the following. Let's try again tomorrow, if that fails, we need to reach out to support:

documentfoundation.fi
Pushing keys failed. Please try again later or contact support. Invalid attribute value syntax

libreoffice.cz
Pushing keys failed. Please try again later or contact support. Command failed; Command failed

libreoffice.fi
Pushing keys failed. Please try again later or contact support. Invalid attribute value syntax

Will mail support about that

Florian Effenberger wrote:

Domains where pushing DNSSEC does not work are the following.

documentfoundation.fi
Pushing keys failed. Please try again later or contact support. Invalid attribute value syntax

libreoffice.fi
Pushing keys failed. Please try again later or contact support. Invalid attribute value syntax

As you hinted at during the last 1:1 call .fi doesn't accept SHA-1 digest (type 2) in in DS records. (Perhaps a policy of the TLD, although I couldn't find sources for that.) Manually entering the DS record (SHA-256 only) and DNSKEY record fixes this. But that doesn't make the “Push KSK Keys to domain” button disappear and I'm not sure what that entails; can't make the button disappear for .at either, we'll see during the next key rotation/rollover if that causes any issue.

libreoffice.cz
Pushing keys failed. Please try again later or contact support. Command failed; Command failed

AFAICT the problem here was that the name servers were set to an external 3rd-party. This also caused resolution to fail with SERVFAIL, like for odfauthors.org and odftoolkit.org.

Guilhem Moulin wrote:

• A record lookups at the root do succeed (NOERROR) for all 129 domains. A, AAAA, MX, TXT, SRV and CNAME records match what we have had at hetzner.

That's no longer the case, I've now marked domains as null MX (IN MX 0 . and IN TXT "v=spf1 -all", cf. RFC 7505) whenever that made sense: NL domains, tdf.io, etc.

• create zones (if required Guilhem to file a ticket)
• set low TTL (e.g. 5-10 minutes)
• import existing two (!) Hetzner zones -> today
• start signing process
• leave DNSSEC off for the moment
• keep zones at Hetzner for the moment
• check leftover Hetzner zones - WHOIS on domain, check if Hetzner or nsX.documentfoundation.org is used -> Florian to ping the owners then

• all of us review the DNS entires, share comments/thoughts/additions/deletions
• merge all this
• then agree on a day for switchover -> plan is Friday afternoon

• remove zones at Hetzner
• raise TTL
• enable DNSSEC
• remove glue records

Florian Effenberger wrote:

AI Guilhem

• create zones (if required Guilhem to file a ticket)
• set low TTL (e.g. 5-10 minutes)
• start signing process

Done for documentfoundation.org, for libreoffice.org the folks at RRProxy need to reassign the zone first.

• import existing two (!) Hetzner zones -> today

Blocking on the zone reassignation. Will proceed with the two domains at once.

• leave DNSSEC off for the moment
• keep zones at Hetzner for the moment
• check leftover Hetzner zones - WHOIS on domain, check if Hetzner or nsX.documentfoundation.org is used -> Florian to ping the owners then

• documentfoundation.com.br
• documentfoundation.net.br
• documentfoundation.org.br
• documentfoundation.srv.br
• libreoffice.com.br
• libreoffice.net.br
• libreoffice.org.br
• libreoffice.srv.br
• thedocumentfoundation.org.br

• documentfoundation.ca
• documentfoundation.dk
• libre-office.ca
• libreoffice.ca
• libreoffice.dk
• thedocumentfoundation.ca

• document-foundation.ca
• documentfoundation.ru
• libreoffice.ir

Guilhem Moulin wrote:

Florian Effenberger wrote:

AI Guilhem

• create zones (if required Guilhem to file a ticket)
• set low TTL (e.g. 5-10 minutes)
• start signing process

Done for documentfoundation.org, for libreoffice.org the folks at RRProxy need to reassign the zone first.

• import existing two (!) Hetzner zones -> today

Blocking on the zone reassignation. Will proceed with the two domains at once.

Done. Some observations:

• Non-default TTLs need to be added manually, RRProxy appears to have a 8h default which is unaffected by the SOA's MINTTL (which per RFC 2308 is fine);
• RRPproxy chokes on SRV records with a trailing period, so these records can't be imported straight from a BIND-formatted zone file (once the trailing period is removed the DNS reply looks good though); and
• RRPproxy chokes on RRs with implicit second-level domain such as foo IN MX mail, one needs to explicitly append a second-level and a trailing dot, namely foo IN MX mail.documentfoundation.org., so these records can't be imported straight from a BIND-formatted zone file (once the second-level domain and trailing period are added the DNS reply looks good though).

Guilhem Moulin wrote:

• RRPproxy chokes on SRV records with a trailing period, so these records can't be imported straight from a BIND-formatted zone file

Interestingly “null” SRV records such as _foo._tcp SRV 0 0 0 . (RFC 6186 §3) can be defined/imported, so at least we don't miss on expressivity here.

• […]

• […]

Done, it's only the 6+2 zones left at Hetzner now (all active).

Original comment text:

Thanks a lot for your work on this!
You can DELETE the following zones at Hetzner:

documentfoundation.com.br
    documentfoundation.net.br
    documentfoundation.org.br
    documentfoundation.srv.br
    libreoffice.com.br
    libreoffice.net.br
    libreoffice.org.br
    libreoffice.srv.br
    thedocumentfoundation.org.br

I will check in parallel with Olivier about the domains, hostmaster@ is in cc - should we re-register, we will use KeyDNS at RRPproxy.
For the moment, please KEEP the following zones at Hetzner:

documentfoundation.ca
    documentfoundation.dk
    libre-office.ca
    libreoffice.ca
    libreoffice.dk
    thedocumentfoundation.ca

I reached out to the domain name holders, asking how to proceed - either we transfer to TDF, or we ask the holders to change the DNS
You can also DELETE the following zones at Hetzner:

document-foundation.ca
    documentfoundation.ru
    libreoffice.ir

For documentfoundation.org

Note: I did NOT check if all entries from Hetzner have been properly converted. If in doubt, better double-check this, please. I ONLY checked for wrong or superflous records in the imported zone.

• @ 600 IN CAA 0 issuewild ";"
  Is that desired? For the other domains I always set "letsencrypt.org" for "issuewild" as well. Fair enough if we don't use wildcard certificates, of course. ;-)

• What we might need to have in mind is for split-DNS situations (external IP = 1.2.3.4, internal IP = 5.6.7.8 eg in VPN) to look if DNSSEC can be enabled, or if that breaks the trust anchor. I remember there is something like a negative trust anchor (https://www.freedesktop.org/software/systemd/man/dnssec-trust-anchors.d.html) but didn't look closer.

• Does it make sense to keep multiple old DKIM entries? DKIM keys are only checked when the mail is received, so three year old, legacy records could be deleted if not used

• There are some hostnames that could be old/outdated. Not sure, might be worth checking. Some that could be old are, maybe double-check with Cloph on these
  • floyd
  • engine
  • gucky
  • imageboard
  • owncloud
  • translations-old

• Do we still run a Torrent tracker (tracker.tdf CNAME vm185)?

For libreoffice.org

Note: I did NOT check if all entries from Hetzner have been properly converted. If in doubt, better double-check this, please. I ONLY checked for wrong or superflous records in the imported zone.

• @ 600 IN CAA 0 issuewild ";"
  Is that desired? For the other domains I always set "letsencrypt.org" for "issuewild" as well. Fair enough if we don't use wildcard certificates, of course. ;-)

• What we might need to have in mind is for split-DNS situations (external IP = 1.2.3.4, internal IP = 5.6.7.8 eg in VPN) to look if DNSSEC can be enabled, or if that breaks the trust anchor. I remember there is something like a negative trust anchor (https://www.freedesktop.org/software/systemd/man/dnssec-trust-anchors.d.html) but didn't look closer.

• There are some hostnames that could be old/outdated. Not sure, might be worth checking. Some that could be old are, maybe double-check with Cloph on these
  • ask-staging (yields "internal server error")
  • blog.vi (points to vi.blog, and then "this site is no longer available")
  • bugassistant (I think the assistant doesn't exist anymore, so we might want to get rid of the hostname; Cloph or Xisco should have details)
  • gerrit-test2
  • manual-test
  • roadshow (shows empty blog)

• after September 30 (shutdown of Hosted.IM) we can also delete all XMPP/Jabber-related SRV entries

Florian Effenberger wrote:

For documentfoundation.org

Note: I did NOT check if all entries from Hetzner have been properly converted. If in doubt, better double-check this, please. I ONLY checked for wrong or superflous records in the imported zone.

Like for other zones I ran a mechanic check on A+AAA+MX+TXT+SRV+CNAME RRs.

• @ 600 IN CAA 0 issuewild ";"
  Is that desired? For the other domains I always set "letsencrypt.org" for "issuewild" as well. Fair enough if we don't use wildcard certificates, of course. ;-)

That's a defense in depth principle: lock down what we don't need/use; should we need wildcard certs later it's trivial to update later. Did a bulk change of other zones last weeks, cf. #1496#note-16.

• What we might need to have in mind is for split-DNS situations (external IP = 1.2.3.4, internal IP = 5.6.7.8 eg in VPN) to look if DNSSEC can be enabled, or if that breaks the trust anchor. I remember there is something like a negative trust anchor (https://www.freedesktop.org/software/systemd/man/dnssec-trust-anchors.d.html) but didn't look closer.

I use that at home because my local resolver is spoofing various records, which breaks the trust anchor indeed. I don't think split DNS applies in our case because the Name Server on gustl is a passthrough except for the (invalid) tdf TLD.

• Does it make sense to keep multiple old DKIM entries? DKIM keys are only checked when the mail is received, so three year old, legacy records could be deleted if not used

Those could be deleted indeed.

• There are some hostnames that could be old/outdated. Not sure, might be worth checking. Some that could be old are, maybe double-check with Cloph on these
  • floyd

Well that's the salt master :-P

• engine
• gucky
• imageboard
• owncloud
• translations-old

imageboard is used, https://imageboard.documentfoundation.org/ is a 303 redirect to the design blog since the mascot fiasco. Would also keep owncloud as removing it would break very old federation lists (not sure either how the desktop client deals with 301 redirects). The rest can be deleted indeed.

• Do we still run a Torrent tracker (tracker.tdf CNAME vm185)?

curl http://tracker.documentfoundation.org:6969/stat says yes, and people do complain when it's down :-)

Florian Effenberger wrote:

For libreoffice.org

Note: I did NOT check if all entries from Hetzner have been properly converted. If in doubt, better double-check this, please. I ONLY checked for wrong or superflous records in the imported zone.

• @ 600 IN CAA 0 issuewild ";"
  Is that desired? For the other domains I always set "letsencrypt.org" for "issuewild" as well. Fair enough if we don't use wildcard certificates, of course. ;-)

• What we might need to have in mind is for split-DNS situations (external IP = 1.2.3.4, internal IP = 5.6.7.8 eg in VPN) to look if DNSSEC can be enabled, or if that breaks the trust anchor. I remember there is something like a negative trust anchor (https://www.freedesktop.org/software/systemd/man/dnssec-trust-anchors.d.html) but didn't look closer.

See #1496#note-26

• There are some hostnames that could be old/outdated. Not sure, might be worth checking. Some that could be old are, maybe double-check with Cloph on these
  • ask-staging (yields "internal server error")

Used by Evgeny for development AFAICT; could probably use a local resolver instead, but IMHO we can keep this record until ask.libo is used in production.

• bugassistant (I think the assistant doesn't exist anymore, so we might want to get rid of the hostname; Cloph or Xisco should have details)

Right that's a redirect to bugs.tdf

• gerrit-test2

Unused, can be deleted.

• manual-test

See #2668.

• blog.vi (points to vi.blog, and then "this site is no longer available")
• roadshow (shows empty blog)

Italo removed these a while ago, records can be deleted I guess.

• after September 30 (shutdown of Hosted.IM) we can also delete all XMPP/Jabber-related SRV entries

Ack.

Like for other zones I ran a mechanic check on A+AAA+MX+TXT+SRV+CNAME RRs.

Perfect!

That's a defense in depth principle: lock down what we don't need/use;
should we need wildcard certs later it's trivial to update later. Did a
bulk change of other zones last weeks, cf. #1496#note-16
<https://redmine.documentfoundation.org/issues/1496#note-16&gt;.

Fine for me!

• Does it make sense to keep multiple old DKIM entries? DKIM keys
  are only checked when the mail is received, so three year old,
  legacy records could be deleted if not used

Those could be deleted indeed.

Then let's remove them. :)

Well that's the salt master :-P

Ooops :)

|imageboard| is used, https://imageboard.documentfoundation.org/ is a
303 redirect to the design blog since the mascot fiasco. Would also keep |owncloud| as removing it would break very old federation lists (not
sure either how the desktop client deals with 301 redirects). The rest
can be deleted indeed.

Ok, then let's make it so.

Used by Evgeny for development AFAICT; could probably use a local resolver instead, but IMHO we can keep this record until ask.libo is used in production.

Let's keep it then indeed.

bugassistant (I think the assistant doesn't exist anymore, so we might want to get rid of the hostname; Cloph or Xisco should have details)

Right that's a redirect to bugs.tdf

If Cloph has no objections, let's delete it.

gerrit-test2

Unused, can be deleted.

Let's do that. :)

blog.vi (points to vi.blog, and then "this site is no longer available")
roadshow (shows empty blog)

Italo removed these a while ago, records can be deleted I guess.

Same here, let's remove it then.

We can't transfer the CA domains due to local presence requirements, but
the domain holder can update the WHOIS.

Can you create DNS records at RRPproxy for

documentfoundation.ca
libre-office.ca
libreoffice.ca
thedocumentfoundation.ca
document-foundation.ca

and let me know when done?

Cleaned up accordingly (only on the KeyDNS side). Kept bugassistant though since there are likely left over links around.

Note to self: we can also remove ns[1-3].documentfoundation.org once the migration is complete.

Florian Effenberger wrote:

Can you create DNS records at RRPproxy for

documentfoundation.ca
libre-office.ca
libreoffice.ca
thedocumentfoundation.ca
document-foundation.ca

and let me know when done?

Done. All 4 zones are unsigned, null MX, and with A+AAAA records for @ and www pointing to the main site.

Perfect! Let's chat this afternoon for the switchover for the main
domains :)

For the DK domain names, we have these indeed already, but directly with Hostmaster DK
I was creating the default set of DNS entries at RRPproxy (please double check, IIRC you modified e.g. the CAA record), and started DNSSEC signing
I've changed the nameservers (without DNSSEC yet) at Hostmaster DK
Will in parallel find out with RRPproxy if we can transfer the domain names directly there

Florian Effenberger wrote:

I was creating the default set of DNS entries at RRPproxy (please double check, IIRC you modified e.g. the CAA record)

I think record sets are useless as they provide no advantage over the API and the API gives more flexibility, but I did bring the set in line with the zones last week (updated the CAA record and set null MX config), so these two new zones are consistent with the other ones :-)

• No problems reported so far, but let's keep libreoffice.org and
  documentfoundation.org as zones at Hetzner, for a fallback/backup, for
  some more days at least.

• Can you check again if you can push the keys for libreoffice.cz? Last
  times I got an error message, ticket with RRPproxy is open.

• You can delete the .DK and .CA domains from Hetzner. WHOIS shows
  RRPproxy DNS, dig yields a result, so transition to RRPproxy seems done.

• For the .DK domains, if you let me know the DNSSEC data, I can update
  the domains at the registrar to use a signed zone.

Florian Effenberger wrote:

• Can you check again if you can push the keys for libreoffice.cz? Last times I got an error message, ticket with RRPproxy is open.

Answered this in #1496#note-15, per #1496#note-16 A record lookups at the root is fully validated.

• You can delete the .DK and .CA domains from Hetzner.

Done, that leaves only documentfoundation.org and libreoffice.org. ns[1-3].documentfoundation.org RRs are no longer used so I removed these from KeyDNS.

• For the .DK domains, if you let me know the DNSSEC data, I can update the domains at the registrar to use a signed zone

documentfoundation.dk. IN DNSKEY 257 3 7 AwEAAeukhjXrV5LX3fn+u28zq2P/lA6r+bAnupetWAiV6skMw7ZUZgMy5m7BEuuNXNHXi8v4a/osvu6op6VnEiqPyyeAHJpWSIAwGjLzVUKB8EKJshShf73KIDoVK426EHdr/rO4WW1Vq7iEwUvNwXqrcpjG3p4yOYW1vXOWrNl2/v8mhXzw/SFeQmN3zxabKYyO9Lg9jzxbmXJu93aAtK43cB4YaWUwIZxdc1MEXBsY/nJIuyZcHDlmOcqhYpfQdAozpdeTr8A5Y+gud05xz6/fJx9nU48R6wMzigQqN0LqpLPu+veLA9zznX1mRJVVxn1H6ZcefHrbcNtpaEKSORQeeuM=
documentfoundation.dk. IN DS 32871 7 1 262F5CFC1D433252285D98721BDEF944F4DBCAE4
documentfoundation.dk. IN DS 32871 7 2 7D9722AEFF45BBC43393D6BA3EC168550C5FF4FFF1DFB4E6F9DC899C36983C11
libreoffice.dk. IN DNSKEY 257 3 7 AwEAAci1m1nyqO83lW7Bbr+YwDdk9bGYB02Ag3XOldLYkuNUriv6jVDCHQBiYIIZRieZQdPiUH/cB2si/0uNVoMjVQkCLHQvXBdXMorbXSQQyxCqtc020dSbu9rGraez6+TCpCFN4u7Iv6qEdBvcLZk9B6ciM0HP7IKjoXd73RPzAp+uiOCmy9yAy/KWuBPr/2xbKJU3WZu/qcHQp4HyTLYQ0Ixf7IV/ciTsNEyjfhr3+QKvN6+ukfIEOz9I6NlA8JHAJRugku477s4FEOZXO/NR8aT5u+dQI93b5wKL05Hrgf9hZCCjq16xxw/Ew76Mk01N+I2u+e7y8+LiZNK+Dhzlu+E=
libreoffice.dk. IN DS 13773 7 1 E827A5A34892D98DE44E72B330C0D2920BE1F96B
libreoffice.dk. IN DS 13773 7 2 85BBB617D397AF09C7C76F7D76990C471B659C4C249714657BC2AB23E9AB7D38 

(Fetched using the API as DNS isn't authenticated yet.)

documentfoundation.dk. IN DNSKEY 257 3 7 AwEAAeukhjXrV5LX3fn+u28zq2P/lA6r+bAnupetWAiV6skMw7ZUZgMy5m7BEuuNXNHXi8v4a/osvu6op6VnEiqPyyeAHJpWSIAwGjLzVUKB8EKJshShf73KIDoVK426EHdr/rO4WW1Vq7iEwUvNwXqrcpjG3p4yOYW1vXOWrNl2/v8mhXzw/SFeQmN3zxabKYyO9Lg9jzxbmXJu93aAtK43cB4YaWUwIZxdc1MEXBsY/nJIuyZcHDlmOcqhYpfQdAozpdeTr8A5Y+gud05xz6/fJx9nU48R6wMzigQqN0LqpLPu+veLA9zznX1mRJVVxn1H6ZcefHrbcNtpaEKSORQeeuM=
documentfoundation.dk. IN DS 32871 7 1 262F5CFC1D433252285D98721BDEF944F4DBCAE4
documentfoundation.dk. IN DS 32871 7 2 7D9722AEFF45BBC43393D6BA3EC168550C5FF4FFF1DFB4E6F9DC899C36983C11
libreoffice.dk. IN DNSKEY 257 3 7 AwEAAci1m1nyqO83lW7Bbr+YwDdk9bGYB02Ag3XOldLYkuNUriv6jVDCHQBiYIIZRieZQdPiUH/cB2si/0uNVoMjVQkCLHQvXBdXMorbXSQQyxCqtc020dSbu9rGraez6+TCpCFN4u7Iv6qEdBvcLZk9B6ciM0HP7IKjoXd73RPzAp+uiOCmy9yAy/KWuBPr/2xbKJU3WZu/qcHQp4HyTLYQ0Ixf7IV/ciTsNEyjfhr3+QKvN6+ukfIEOz9I6NlA8JHAJRugku477s4FEOZXO/NR8aT5u+dQI93b5wKL05Hrgf9hZCCjq16xxw/Ew76Mk01N+I2u+e7y8+LiZNK+Dhzlu+E=
libreoffice.dk. IN DNS 13773 7 1 E827A5A34892D98DE44E72B330C0D2920BE1F96B
libreoffice.dk. IN DNS 13773 7 2 85BBB617D397AF09C7C76F7D76990C471B659C4C249714657BC2AB23E9AB7D38

DK Hostmaster lets me add "DNSSEC key". Is that DNSKEY or DS?

Done, that leaves only documentfoundation.org and libreoffice.org.
ns[1-3].documentfoundation.org DS RRs are no longer used so I removed
these from KeyDNS.

Then I think we can also remove the glue records, correct? Can you do that?

Florian Effenberger wrote:

DK Hostmaster lets me add "DNSSEC key". Is that DNSKEY or DS?

The (public) key material is in the DNSKEY record. DS RRs are its digest values using various alorigthms (SHA-1 and -256 in the above).

Then I think we can also remove the glue records, correct? Can you do that?

Right, done.

Fields to add are Keytag, Algorithm, Digest type and Digest, so then
I'll try with the latter :)

The Document Foundation Redmine wrote:

The (public) key material is in the DNSKEY record. DS RRs are its digest
values using various alorigthms (SHA-1 and -256 in the above).

Florian Effenberger wrote:

Fields to add are Keytag, Algorithm, Digest type and Digest

Just split the record by word?

documentfoundation.dk. IN DS 32871 7 1 262F5CFC1D433252285D98721BDEF944F4DBCAE4

Means

• Key tag: 32871
• Digest algorithm: 7 (ie RSASHA1-NSEC3-SHA1: https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml)
• Digest type: 1 (ie SHA-1: https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml#ds-rr-types-1)
• Digest value: 262F5CFC1D433252285D98721BDEF944F4DBCAE4

Cf. RFC 4034 sec. 5.1 :-)

Guilhem Moulin wrote:

Just split the record by word?

(But the DNSKEY record should be enough.)

Couldn't add that due to missing algorithms, so was using the DS, seems
to work :)

Florian Effenberger wrote:

Couldn't add that due to missing algorithms, so was using the DS, seems to work :)

Confirmed

Lowering severity now, the only thing left is to raise the TTL and

ACK

It'll be one week tomorrow that we updated the name servers and AFAICT
the transition was smooth. I propose to update the SOA and RRs TTLs

I agree, no problems whatsoever. :-)

tomorrow (ie before deploying DNSSEC), any objection? AFAICT the change
requires to delete and recreate each record, but using the API this can
be done in a single call which appears to be atomic.

If you can take care of it and are available in case of emergencies,
fine for me - doing it during the weekend is definitely better than
during the week, so fine for me!

Florian Effenberger wrote:

If you can take care of it and are available in case of emergencies, fine for me

Done. I wasn't able able to it atomically though: the records were updated (exporting the zones using the API showed the new TTL) but the changes were not reflected in DNS queries to the authoritative server. But AFAICT the DNS has a 5min cache and picks up changes at fixed intervals, so in practice consecutive changes won't yield bogus replies.

Guilhem Moulin wrote:

Kept bugassistant though since there are likely left over links around.

Seems it was deleted anyway… this name is also used as SAN in the bugs.tdf CSR, so blindly removing it would have failed the next ACME challenge thus cert renewal for bugs.tdf! Added back for now.

One more "cleanup" task:

For all domains, in the past I've set the "Whois banner", "RSP tag", "URL" fields. These can be removed, as we can configure them centrally. Nothing urgent, but can be done as some cleanup tasks. A API call on all domains should be able to remove the fields/empty the strings.

Another thing we can consider, likewise "cleanup" and not urgent:

We've used the non-whitelabel DNS for all domains because of the "Inconsistent set of NS RRs" error message for .DE domains.
The solution to that is to, for .DE domains only, add an explicit NS entry in the zone, pointing to the whitelabel DNS.
So we can, if we want, move all domains to the whitelabel variant.

Turns out that even if you set whitelabel DNS, the NS entry in the zone is on the branded one, so if we change that for .DE, we should change it for all zones actually

FYI the ZSK rotation seems stable, one can see the RRSIGs updating and of the 138 registered zones 128 still validate, the remaining ones being the 5 zones under the ca TLD, .{hu,li,lt} for which there is no DNSSEC support, and {libreoffice,documentfoundation}.org. As far as I'm concerned it's all good to flip the switch on the latter two also.

Fine for me, during the weekend, whenever you are around to react in
case of issues. :-)

Checked

https://dnssec-analyzer.verisignlabs.com/libreoffice.org

and

https://dnssec-analyzer.verisignlabs.com/documentfoundation.org

and all looks good :)