APNIC’s Resource Explorer (REx) provides a friendly version of the latest delegated statistics file on its Delegations page. When REx began providing global Internet data, statistics, and measurements at the beginning of August, the table on that page was also upgraded to show the most recent transfer date for resources where possible.
This post will discuss why APNIC added this information, where the data comes from, and how challenges that surfaced along the way were addressed.
Why are transfer dates important?
Delegation dates alone do not tell the full story about resource assignment. The first dial-up Internet connection in Pakistan was established in 1994. However, when checking delegations for Pakistan in REx, sorted by oldest delegation date, it’s apparent that some of those resources were delegated before 1994. Besides this example from Pakistan, another common reason for delegation dates to look strange is when an organization holds resources that were originally delegated before they started operating.
These scenarios occur in data from almost all economies and suggested to APNIC that having transfer information in REx would bring some clarity to these delegations. After all, the core purpose of REx’s delegations table is to make it easier to see the who, where, and when of currently assigned resources.
Where does transfer data come from?
The Regional Internet Registry (RIR) statistics exchange format used by delegated stats files (which the table in REx is based on) does not contain transfer dates. However, there is a special case for the delegation date when a resource transfer occurs, described in section 4.3:
Where the allocation or assignment has been transferred from another registry, this date represents the date of first assignment or allocation as received in from the original RIR.
RIR statistics exchange format, section 4.3 Record format
This means that if a resource was originally delegated by ARIN to an organization in the US in 1990, and then transferred to APNIC for a holder in Pakistan in 2020, the 1990 delegation date will still be listed in the stats file.
The RIRs publish transfer logs, which contain accurate transfer information for resources. More information about APNIC’s transfer logs can be found online. The biggest problem with these logs is that they do not cover the complete history of all resources that are currently in use today. APNIC’s transfer logs go back to late 2009, with the 2018 being the latest start time among the logs from all RIRs. Meanwhile, approximately one-third of today’s assigned and allocated resources were delegated prior to 2009.
As part of the upgrades to serve global data, the quality of delegation statistics was also improved, which had REx parsing every historical delegated stats file from each of the five RIRs to give accurate aggregations for resource allocation over time. It was a logical next step to try detecting resource transfers by watching for changes between the historical files over time, potentially being able to detect certain types of resource transfers all the way back to late 2001.
Changes to RIR, ISO3166 Economy code (CC), delegation status, or the size of a resource block between files could all be potential indicators of a transfer taking place. However, there are many situations that could lead to some of these attributes changing other than a resource transfer. Only a change of RIR can be definitively taken as a resource transfer.
Transfer periods
Even with a transfer condition to look for in the delegated stats files clearly identified, accurately tracking it without introducing too much complexity to APNIC’s existing analysis remained a challenge because of a plethora of edge cases.
Firstly, older delegated stats files vary in frequency, unlike today’s much more reliable daily availability of freshly generated files. For example, APNIC produced these files monthly from 2001 until mid-2003, while LACNIC generated files on the first and fifteenth day of each month during 2003. This means that when a transfer is detected during a period like this, there is uncertainty about when the actual transfer took place between files.
Even when daily data is available from all five RIRs, a resource transfer might not occur seamlessly between them in the delegated statistics files. Ideally, a resource block would be present in the file from one RIR on the first date, then it would disappear from there and show up with a different RIR on the following date. On some occasions the resource may not show up within any RIR during the transfer period, and it is also possible that the source and recipient RIRs both appear to have delegation authority during the transfer period before eventually resolving to the recipient.
This project covered almost every edge case this set of problems introduced by keeping track of the last date any given resource block was seen in a delegated stats file, where a resource block is tracked by its delegation date and start value (first IP address or first Autonomous System number (ASN) of the block).
As files from all RIRs are parsed each day, if there is any conflict then REx will ignore the changed resource, leaving the last-seen date untouched. Eventually, the conflict between files resolves, at which point the last-seen date may be taken as the start of the transfer period, with the resolution date becoming the end of that period. The resource can then be updated, and ready for further changes.
After parsing all historical delegated stats files, REx has a record of all the transfers it detected for each resource and displays the most recent of these in the table on the Delegations page. When the best available data for a transfer time is a range of dates, the most recent date from that range is displayed in REx. Making all these transfer details accessible for users to examine is then a possibility for future upgrades and improvements to REx.
Since there is no clear disadvantage to doing so, REx continues to infer transfers from delegated stats files, even in today’s data! Since it does not distinguish older transfers from the ones it picks up from the detailed transfer logs provided by the RIRs, inferred transfers are upgraded with more specific timestamps where possible. As a result, almost all transfers between RIRs that were inferred since transfer logs began have been upgraded with that additional information, and there are occasionally transfers identified that are not present in the full logs.
Go and discover!
APNIC hopes that the additional context provided by these transfer dates is valuable to the community and invites users to explore the global Internet data that is now available in REx.
Feedback is always welcome. To help improve APNIC products and services that will benefit the whole community, sign up or get more information via the User Feedback Group or the APNIC User Feedback Group on Orbit.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.