An Singular Anecdote about the DNS Root Zone Signing Ceremony

22/08/2024

By Carlos Martinez Cagnazzo, LACNIC CTO

I am honored to be among the global security experts participating in the now-famous ‘key signing ceremony,’ a critical operational event coordinated by the Internet Corporation for Assigned Names and Numbers (ICANN) that is fundamental to how we secure the Domain Name System (DNS). Each year, four ceremonies take place in El Segundo, California (USA) and Culpeper, Virginia (USA), where cryptographic keys —one public and one secret—are used in a coordinated effort to secure the DNS root zone. The primary purpose of these ceremonies is to provide a secure environment where the root zone Key Signing Key (KSK) can be used to sign zone keys. This process is applied to generate three months of cryptographic signatures to be used for the daily signing of the root zone.

A few weeks ago, I had the chance to participate once again in the West Coast ceremony in El Segundo, just minutes away from Los Angeles International Airport. This time, it was a different and particular experience. It’s worth noting that these ceremonies are not only unique but also quite a theatrical experience, as they involve a physical process to certify the health and security of the Internet environment.By ‘physical,’ I mean that the cryptographic process involves specific hardware, such as the hardware security module (HSM). An HSM is a physical computing device designed specifically for working with sensitive cryptographic material. The key generated within it never leaves the device.

This ceremony was exceptional in that it was much longer than usual. Why? Because the HSMs that are currently in production will no longer be guaranteed after 1^st January 2025, as their manufacturer has decided to discontinue them. The new HSMs are produced by a different manufacturer, so in this ceremony four new HSMs —two pairs— were formatted (the reason being that each pair provides backup to the other).

The singular aspect of this process is that the initialization of HSMs is highly proprietary, which means that there are few standards, and those that do exist are applied inconsistently across the different devices.Specifically, when different manufacturers are involved, there is no adequate way to retrieve the private key from the old HSM and migrate it to the new one. Even if there was only one manufacturer involved, the key would have to be transferred between two identical HSMs, which is how the backup HSM is configured. In this situation, the key is retrieved using a proprietary mechanism (which adheres to certain security parameters) and copied to the backup HSM.

For this particular ceremony and due to this unique requirement, a new KSK was generated. How was this accomplished? Specifically, a backup of the new KSK that had already been generated during the East Coast ceremony at the end of March was imported, initialized as if it were new, and the other backup HSMs were also initialized.

I want to highlight that this exceptional process was quite lengthy: we were there for a total of eight hours. It’s also important to stress that the formatting of the new HSMs took place within the framework of a ceremony where everything is meticulously controlled, scripted, and audited, and where the roles of each participant and the steps they must follow are carefully defined. Various organizations that have an impact on Internet governance play central roles in the process: ICANN; the Internet Assigned Numbers Authority (IANA), which is responsible for the secure management of the DNS root zone on behalf of ICANN; Verisign, the company currently responsible for maintaining and operating the DNS root zone, and other actors who guarantee the transparency of the entire process, including the ceremony administrators, other crypto officers, internal witnesses, auditors, as well as credentials and hardware safe controllers.

Another aspect I would like to highlight is that the HSM cannot function without the involvement of the operators, the community representatives who provide transparency to the processes carried out on the root.Within the security room, there are two safes, one for the HSM and another for the smart cards that activate it. As crypto officers, we hold the keys that unlock the safe containing the smart cards that allow operating the HSM.

Finally, as we’re talking about DNS security, this is a good time to mention an important milestone that will take place in the coming months. Just as it is considered good practice for Internet users to regularly change their passwords, ICANN is planning to rollover the ‘top’ pair of cryptographic keys used in the KSK in 2025.

This rollover involves generating a new pair of cryptographic keys and distributing the new public component to all DNSSEC validating resolvers worldwide. Why is this a significant change? Because every Internet query using DNSSEC relies on the root zone’s KSK to validate the destination. Therefore, once the new keys are generated, network operators performing DNSSEC validation will need to update their systems with the new key to ensure that when a user tries to visit a website, their query can be validated against the new KSK.