APNIC registry services availability during Q1 2025

By on 12 May 2025

Category: Tech matters

Tags: , , , , ,

1 Comment

Blog home

From this year, APNIC’s quarterly registry service availability figures will be reported on the APNIC Blog, along with lessons learned from any incidents experienced during the quarter. These reports will continue to be published in the quarterly APNIC Executive Council (EC) reports and compiled in APNIC Annual Reports.

APNIC’s core registry services are whois, Registration Data Access Protocol (RDAP), Resource Public Key Infrastructure (RPKI), Internet Routing Registry (IRR), and Reverse Domain Name System (rDNS). The availability of these services for Q1 2025 is shown in Table 1.

ServiceAvailability
Whois and IRR99.991%
RDAP99.998%
RPKI99.999%
rDNS99.998%
Table 1 — APNIC core service availability for Q1 2025.

To measure availability, APNIC uses a third-party monitoring service that sends queries to these services every minute and records downtime when it detects a problem with the service.

Notable issues during Q1

Whois and IRR

The reported 99.991% availability for the whois service is slightly lower than usual due to recent changes in our third-party monitoring setup. These changes increased the system’s sensitivity, leading it to detect brief, low-impact events as downtime. This included routing changes between probe locations and the whois servers. We’ve since updated the monitoring configuration to verify potential issues from multiple locations, ensuring that only genuine service disruptions are counted.

rDNS

During Q1, an issue encountered with the rDNS service revealed some gaps in APNIC’s monitoring systems, which resulted in a lower availability score. APNIC’s rDNS servers are deployed using anycast, with servers around the world answering queries sent to the same IP address. This relies on the network routing these queries to the server closest to the client. The network uses Border Gateway Protocol (BGP) to send these queries from a client to the APNIC server and back, selecting the shortest path as the best one to use.

This setup works well when the shortest path goes to the nearest server. However, it is possible that the shortest path, as seen by the routing layer, goes to a server on the other side of the world instead of the one in the same region. In early March, during a regular transit link failover test, we found that not all the rDNS anycast prefixes were being announced to one of the transit providers in Brisbane, where some of APNIC’s rDNS servers are located.

To correct this, the anycast prefixes were announced to this transit provider. After this change, there were increases in DNS queries to the rDNS servers in Brisbane, but not enough to trigger monitoring alerts. Two days later, an email was received notifying that when querying the reverse DNS server, timeouts were occurring and that DNSMON, the RIPE NCC DNS monitoring tool, was seeing unanswered User Datagram Protocol (UDP) queries. DNSMON was checked, and it was observed that four of the 50 probes monitoring this anycast service had started seeing increased failure rates around the time the transit provider had started announcing the anycast prefixes. The transit provider was contacted, and they withdrew the routes. Shortly afterwards, DNSMON showed that the four probes were no longer seeing any unanswered queries. We’re working with this transit provider to only announce these prefixes locally to Australia.

Incidents like this are always opportunities to learn and improve. As a result, RIPE ATLAS measurements will be used in future as another way of monitoring the rDNS servers. This will be combined with atlas-exporter, Prometheus, and Grafana to alert on probes seeing increased failure rates and to improve dashboards.

On track for 2025

The APNIC registry’s target is to maintain availability of at least 99.99% for APNIC core registry services. Regardless of the lower-than-expected whois and IRR and rDNS availability for Q1, the results are on track to meet the 2025 target.

Rate this article

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Top