I’ve written previously about a key concept in PKI systems, M of N. M of N is how you carve up ownership of necessary information to use a cryptographic hardware security module (HSM) to ensure a process that is visible to the community.
In that post, I talked about how the key could be split up and given to various key holders. Then, a set number of those keys (and presumably their key holders if they haven’t been passing them around) can come together to use their key fragments.
Today we’re going to look at how they come together in a ‘key ceremony’ and why this process needs to be as visible as possible. Key ceremonies ensure the community trusts what these key holders are doing. Trust is why you use something like M of N in the first place.
When you hear the term ‘ceremony’, it brings up images of rituals and strict processes that must be observed. This is actually an appropriate term, as in this post I’ll explore what those ‘rituals’ look like in the context of a key signing ceremony.
Much of what’s written here is based on what ICANN themselves document for their processes and motivations.
It’s all about trust
Using private keys in an HSM is not actually all that difficult. Participants go through all these steps not because it’s complicated, but because mistakes will undermine trust in the activity of signing.
What they’re actually doing in the ceremony is making an assertion that can be tested; a cryptographic proof of something.
But why are they doing it? What stems from this signing?
The trust in all the world’s domain names stems from the trust in the cryptography anchored in that HSM.
So, if that HSM looked like it had been tampered with, there’s a problem; a big one.
Imagine for a moment that the entire family gathers around their shared safety deposit box in the bank. They all see that the lid has been scratched, and the envelope that has the key inside has already been opened.
You’re lucky if they don’t start fighting on the spot. How do you make sure that scenario doesn’t happen?
When you put it in the bank in the first place, you gather everyone around to watch, and have trusted family members participate in locking it away safely. It may not be perfect, but it should work as long as trust is maintained and nothing suspicious happens while you’re doing it.
As ICANN themselves say in their slide pack on their key ceremonies: “The purpose is to ensure trust in the process. DNSSEC only provides security if the community is confident the HSMs have not been compromised.”
Ceremonies are about two things: codified acts, and witnesses to see them done properly. Trust comes from both the performance and the audience.
Getting underway
An HSM is a device designed to secure private keys, and apply private keys without revealing their value, so nobody else can use them. This means they are designed to help inform you if anyone else has been accessing the system, and if any other use of the keys has been recorded. They do this in different ways, with physical tamper-proof checks, but also with audits on key usage, serial numbers, logs and records on what has been done.
So, the role of the ceremony is very likely to include pre-checks on the HSM itself. There are some key questions that need to be answered here:
- Has it been stored in a secure location? Can we show that it hasn’t left this location? (Logs of access to its location; logs of things brought into, and removed from, this location).
- Has it been altered in any way? Has anyone attempted to alter it? (Checks of tamper-proof indicators, systems monitoring).
- Has it been used since the last time we recorded it was used? (Logs of access to the HSM as a system. Logs of key usage, serial numbers of events).
ICANN refers to these steps as ensuring “the chain of custody of all the elements is verified throughout the process.”
How does ICANN resolve these questions?
- Opening tamper-proof bags, protecting Pin Entry Devices (PED) or cards, or like ‘tokens’ of identity. The tamper-proof bags probably have serial numbers, which were recorded in the last ceremony, so you can show the bags are both integral, and the same ones recorded when you last performed this function.
- Checking that all necessary witnesses and M of N participants are present (or their alternates).
- Checking that recording devices, and other tools to ensure integrity of the ceremony are functional.
If anything goes wrong, then STOP
On the assumption all is well, a ceremony would proceed to the next steps of actually invoking the ceremony to use the systems (in a wider sense, we’ve already started the ceremony by performing these pre-checks) as scripted.
It is important to note that if ANY of the questions listed above can’t be answered, the right thing to do at this point, is to STOP.
You can’t proceed if you have any reason to doubt that the HSM and associated systems are not in the state you expected. So, a common design pattern in the key ceremony is that each step is a simple, positive-proof process, and you can only proceed if you can ‘pass’ the test.
Once the ceremony has begun, it’s actually quite simple
Considering all these checks and balances, actually using the keys is itself really boring. Signing consists of presenting data to the HSM, specifying the key to use, invoking the M of N process to authorize use of the key, and then the HSM itself performs complex mathematics in public-private cryptography, to make the signature. The outcome is ‘more data’, as well as serial numbers, audit logs, and records of what happened.
In the ceremonial sense, these audits and logs are more important than the signed data (well, not really. The signed data is why we’re using an HSM. But the way the data is handled is now just as important as the data itself).
At this stage, you perform the ‘inverse’ of the checks you did, before beginning the ceremony:
- Confer with witnesses that the ceremony had no ‘unusual’ qualities which demanded it be considered failed.
- Write down the time, and serial data, inside the tamper proof record keeping, which itself is now recorded as being used, so that next ceremony you can check there has been no misuse in-between.
- Place a PED, cards, tokens into tamper proof bags, seal all the tamper proof packaging, restore it to the secured area.
- Export and publish the recordings, so that people can see what was done, and confirm what we say was seen to be done by the participants.
Luckily, this process only usually takes three to four hours and only has to be done a few times a year. ICANN operates with a community of 14 ‘cryptographic officers’ and seven ‘recovery key shareholders’ (with eight backups) all of whom are considered Trusted Community Representatives (TCRs).
What happens if you need to change the ceremony?
Ideally, you would never vary a ceremony.
However, COVID-19 has driven a truck through some of our assumptions about what it is to be a participant in a ceremony. Just as we have had to get used to moving weddings and funerals to having remote participants, the ICANN key signing ceremony had to be adjusted, to take into account these circumstances. You can read about the changes they made here. The important thing is, they documented the variances. That is substantively what a key ceremonial process change demands: if you vary it, document it, and show it.
At the end of the day, that’s what it comes down to: Does the ceremony provide trust? Is it recorded, and does it have witnesses? If the answer to all these is “yes” then you have a decent ceremony. If not, then you may have a gap between how you operate, and how people understand it and can trust it.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.