Did you know that more than 75% of the prefixes in Bangladesh have valid RPKI ROAs? Or, perhaps more impressively, that less than 2% are invalid?
This is a massive improvement from only five months ago when only 29% of prefixes were valid, 69% were unknown and 2% were invalid. This has much to do with the National Data Centre’s (NDC) extensive campaign to increase awareness and use of RPKI validation among the economy’s 800+ ASNs.
Why we started validation
One of the major motivations for NDC to start RPKI validation and dropping invalids was related to its commitment to its users and stakeholders to implement best current security practices (as well as its commitment to contribute to global routing infrastructure security).
Over the years, we’ve observed many domestic routing incidents in Bangladesh that have directly and indirectly impacted our services and operations, including seeing our prefixes announced by others, making our service partially unavailable for significant amounts of time. We’ve also observed prefix hijacking and route leaking as well as announcements of non-routable prefixes by local operators in the global BGP table.
Since the majority of NDC’s users are within Bangladesh, we had to think about first reducing the impact of RPKI validation locally. We started by identifying the ASNs with invalid and unknown prefixes and reached out to each of them to help them fix their ROAs.
We reached out to more than 600 ASNs to help them create new ROAs for more than 3,500 unknown BGP prefixes and fix about 100 invalids. We started reaching out to them during early October 2019 and saw significant improvement in the number of valid ROAs before the validation started on 1 December 2019.
Developing awareness in the community
During the soft implementation period, we carried out an extensive awareness campaign to make sure that everyone understood the impact of NDC’s RPKI validation and how it affects them. This included:
- Publishing a blog post in the local language (Bengali) explaining the basic information and benefits of RPKI and ROA, why NDC is going to do the validation, who and how it will affect the ISPs and users in Bangladesh and how ROAs can be created and verified.
- Sending emails to the bdNOG Mailing List mentioning the key dates of NDC’s RPKI deployment plan.
- Posting similar articles on different social media, including bdNOG’s Facebook page and BGD e-GOV CIRT’s Facebook and Twitter pages.
- Providing detailed steps on creating ROAs and several ways to verify them. We also shared our contacts so that anyone can reach out to us if they faced any issues.
- Creating a list of all ASNs in Bangladesh that include the number of IPv4 and IPv6 prefixes of each ASN, and the number of valid, invalid and not-found ROAs. I contacted each of the ASN contacts via email, phone, SMS and online messages and informed them of the ROA status of their prefixes.
- Helping law enforcement agencies, government organizations, ISPs, IXPs, banks and financial organizations, transit providers, data centres, universities and R&E networks to create ROAs for their prefixes.
- Referring some cases to the APNIC Helpdesk due to the issue requiring assistance with getting access to the MyAPNIC portal.
Helping local ISPs in BD to fix/create their ROAs. BD has 2% invalid and 69% unknown prefixes which needs to be reduced. Hope to see some improvements on @RoutingMANRS Observatory pretty soon.— Md Abdul Awal 🚲 (@0xAwal) October 23, 2019
I'm so happy seeing this sharp uprise of #RPKI #ROA in #Bangladesh. That well paid off my efforts for last couple of weeks. Appreciate the champions who agreed to sign their prefixes in a very short time. @RoutingMANRS @apnic @routinator3000 @ripencc @ripelabs pic.twitter.com/n7i1ZdcFcm— Md Abdul Awal 🚲 (@0xAwal) November 19, 2019
Dropping RPKI invalids since 1 December 2019
Finally, on 1 December 2019, NDC deployed RPKI validation and started dropping invalids.
About 51 IPv4 and 20 IPv6 invalid prefixes of local ASNs were initially dropped due to validation and users on those IPs couldn’t access the content hosted at NDC. Several of them have contacted us since and we have explained the issue to them and helped them fix their invalid ROAs.
Over the last six months, I’ve helped more than 600 ASNs resolve their RPKI ROA issues. I’ve guided them through online remote sessions and meeting in-person to create and/or fix their ROA issues.
Most of the network admins have been very cooperative but not everyone wants to fix their ROAs. I’ve come across people who said that they don’t need to fix them, or they don’t want my help.
It must be noted that this is not the only reason for invalids. I’ve found that the wrong max length value has created most of the invalids in Bangladesh and continues to, which is the reason for the number of invalids not dropping below 2%. An example of this is an ISP having valid ROAs for all their prefixes changing its BGP announcements with smaller subnets, which would introduce new invalids.
Interestingly, most of the new invalids are seen for IPv6 — the ROAs have been created with a /32 max length for IPv6 prefixes but later the BGP announcements are made with smaller prefixes that might introduce new INVALIDs.
76% IPv4 and 41% IPv6 prefixes of Bangladesh are #ROA signed. The number of routing incidents in BD significantly reduced since late 2019. This well paid off my efforts on MANRS awareness + RPKI validation in National Data Center. @apnic @RoutingMANRS @AbuseIp @routingtablepod pic.twitter.com/uhh6myE4Dn— Md Abdul Awal 🚲 (@0xAwal) February 5, 2020
About 79% of IPv4 prefixes and 45% of IPv6 prefixes marked #RPKI VALIDs in Bangladesh. Thanks to the network operators who responded and created ROAs. We need everyone's support to make global routing infrastructure secure.@apnic @RoutingMANRS @Opsudaysis @AbuseIp pic.twitter.com/SO8A5P3Zby— Md Abdul Awal 🚲 (@0xAwal) March 21, 2020
Communities need to take a lead to help their economies
The main reason behind missing ROAs seems to have been a lack of awareness.
Despite lots of discussions globally about RPKI deployment, almost no effort had been made to reach out to the individual ASNs in Bangladesh. While many of them were aware of RPKI and were able to create ROAs using MyAPNIC, they just simply didn’t feel it necessary to enable it. Some admins didn’t know the procedures of creating ROAs from the MyAPNIC portal and some even didn’t know about RPKI ROA itself.
I think there is a significant knowledge gap and a lack of awareness about RPKI. While the discussion is happening globally, we need to discuss more about RPKI in local NOGs and help each other within our community to be successful in a wider deployment of RPKI.
Md Abdul Awal is a passionate network engineer and a Mozilla Open Internet Engineering Fellow. He manages the operations of the National Data Centre in Bangladesh.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.