Recently I decided to set up a dedicated test environment to improve my knowledge around Resource Public Key Infrastructure (RPKI).
Now that the testbed is finally up and running, I would like to share my journey with the network operator community. I welcome feedback on this testbed, which I’ll use to help tune it.
Network setup
Using eve-ng, I linked a few routers to simulate a BGP IP hijacking attempt; aiming to neutralize this threat by implementing RPKI. For the sake of simplicity, I deployed only three routers, each one of them belonging to a different BGP Autonomous System (AS).
The router belonging to ASN 30 (as shown in Figure 1) started announcing an IP address that overlaps with an IP prefix range legitimately advertised by ASN 20. From now on, whenever ASN 10 tries to reach out to 170.0.0.1/32 it will be forwarded to the rogue router belonging to ASN 30.
The BGP protocol will prefer the more specific prefix (170.0.0.1/32) over the less specific one (170.0.0.1/24). The traffic will flow towards the compromised ASN 30 via next-hop 10.0.0.6 (instead of 10.0.0.2).
RP/0/0/CPU0:router-asn10#sh bgp sessions Mon Jul 26 14:28:00.527 CET Neighbor VRF Spk AS InQ OutQ NBRState NSRState 10.0.0.2 default 0 20 0 0 Established None 10.0.0.6 default 0 30 0 0 Established None RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast neighbors 10.0.0.2 routes | b ^Status Mon Jul 26 14:28:13.456 CET Status codes: s suppressed, d damped, h history, * valid, > best i - internal, r RIB-failure, S stale, N Nexthop- discard Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 170.0.0.0/24 10.0.0.2 0 0 20 i Processed 1 prefixes, 1 paths RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast neighbors 10.0.0.6 routes | b ^Status Mon Jul 26 14:28:23.945 CET Status codes: s suppressed, d damped, h history, * valid, > best i - internal, r RIB-failure, S stale, N Nexthop- discard Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 170.0.0.1/32 10.0.0.6 0 0 30 i Processed 1 prefixes, 1 paths RP/0/0/CPU0:router-asn10#traceroute 170.0.0.1 Mon Jul 26 14:28:48.354 CET Type escape sequence to abort. Tracing the route to 170.0.0.1 1 10.0.0.6 9 msec * 0 msec RP/0/0/CPU0:router-asn10#
RPKI IT infrastructure setup
Whenever it comes to RPKI, NLnetLabs is one of my main references. They’ve written clear documentation and developed the key software I used within (My)Lab infrastructure, including:
- Krill, which can be classified as Certificate Authority (CA) software. In my setup, it is mainly responsible for forging and publishing the Route Origin Authorizations (ROAs) to the parent CA.
- RTRTR, which can be classified as RPKI-to-router (RTR) protocol server software (RFC 6810-v0 and RFC 8210-v1). Its main job is to dispatch the validated ROAs to the BGP enabled routers.
Other than NLnetLabs, OpenBSD’s rpki-client is the core component of the whole solution. It’s responsible for:
- Synchronizing ROAs (X.509 Certificates), via RSYNC or RRDP, from a given Trust Anchor (TA). The TA is usually a Regional/Local Internet Registry (RIR/LIR) organization.
- Validating the chain of trust for the associated ROAs (including checking relevant certificate revocation lists).
- Caching a list of the validated ROA payloads (VRPs) in various formats, for example, JSON.
Since I’m working within a test environment, I decided to collapse all services on a single Linux host. However, potentially nothing is preventing me from horizontally scaling each one of the involved services.
I recommend referring to the respective official documentation for the software installation and configuration — see the references and resources section below.
KRILL
Krill can be deployed using two different models:
- Hosted RPKI
- Delegated RPKI (I chose this)
After completing both the repository setup and the parent setup, I was ready to start publishing new ROAs.
I named the child CA rpki-alfanetti, and as you can see from the text snippet below its relationship with the parent CA (testbed offered by NLnetLabs) is in Status: success.
The parent CA certifies my entitlement over some specific resources, ASN: AS20 and v4: 170.0.0.0/24 are some of them. Third parties can, in a second stage, download the signed certificate (to be verified), which proves ownership over that specific resource.
root@rpki01:~# krillc parents statuses --token e1bb6e95c21740f83dba1adb1ff19ade --ca rpki-alfanetti Parent: testbed URI: https://testbed.rpki.nlnetlabs.nl/rfc8181/rpki-alfanetti Status: success Last contacted: 2021-07-27T08:50:00+00:00 Next contact on or before: 2021-07-27T09:00:00+00:00 Resource Entitlements: asn: AS10, AS20, v4: 170.0.0.0/24, 172.0.0.0/24, v6: resource class: 0 issuing cert uri: rsync://testbed.rpki.nlnetlabs.nl/repo/ta/0/1040FA57F99F4B3DE2626F6E E1C56664CB81D2C8.cer received certificate(s): published at: rsync://testbed.rpki.nlnetlabs.nl/repo/testbed/0/660E433B340BC5B12B20 4F1544C6EA18A5931DB4.cer resources: asn: AS10, AS20, v4: 170.0.0.0/24, 172.0.0.0/24, v6: cert PEM: -----BEGIN CERTIFICATE----- MIIFsTCCBJ ... MojHUKkp30dIbbpo49FocyZyI58lFI7DsDVmXn9Bz0sAeYRB SNviq7K+O4SS/RZezDuY/5MEXHvDWsZXldfX1r+RxsgXi0/5fuudLU4CYlWQGzs -----END CERTIFICATE-----
Read: How to: Run delegated RPKI in Krill
OpenBSD’s rpki-client
OpenBSD’s rpki-client is periodically syncing with the parent CA looking for new ROAs. To do so, it searches for the TA belonging to the parent CA — in particular, there’s a certificate used to validate received ROAs.
OpenBSD’s rpki-client is somehow hiding the complexity coming from the cryptography required to validate the received ROAs.
Once the validation process is completed, the information from the ROAs is extracted from the associated certificates and presented in clear text format (for example, JSON), ready to be ‘digested’ by the routers.
### root@rpki01:~# cat /var/lib/rpki-client/json
{
"metadata": {
"buildmachine": "rpki01",
"buildtime": "2021-07-26T08:04:49Z",
"elapsedtime": "133",
"usertime": "0",
"systemtime": "0",
"roas": 6,
"failedroas": 0,
"invalidroas": 0,
"certificates": 49,
"failcertificates": 0,
"invalidcertificates": 0,
"tals": 1,
"talfiles": "/etc/tals/testbed.tal",
"manifests": 49,
"failedmanifests": 43,
"stalemanifests": 0,
"crls": 6,
"repositories": 9,
"vrps": 6,
"uniquevrps": 6
},
"roas": [
{ "asn": "AS65101", "prefix": "10.0.0.0/23",
"maxLength": 24, "ta": "testbed" },
{ "asn": "AS20", "prefix": "170.0.0.0/24",
"maxLength": 24, "ta": "testbed" },
{ "asn": "AS10", "prefix": "172.0.0.0/24",
"maxLength": 24, "ta": "testbed" },
{ "asn": "AS65001", "prefix": "192.168.0.0/23",
"maxLength": 24, "ta": "testbed" },
{ "asn": "AS37708", "prefix": "196.1.0.0/24",
"maxLength": 24, "ta": "testbed" },
{ "asn": "AS65001", "prefix": "2001:db8::/32",
"maxLength": 64, "ta": "testbed" }
]
}
Read: How RRDP was implemented for OpenBSD rpki-client
HTTP server/RTRTR and BGP router configuration
A basic HTTP service (python3 -m http.server 8081) is allowing the RTR server to access the JSON file storing the ROA records. RTRTR is now listening for incoming connections from the BGP routers on the TCP socket <192.168.122.253:8282>.
The RTR server should be reachable by the involved routers. The below configuration should be present on each one of them.
router bgp <ASN> rpki server 192.168.122.253 transport tcp port 8282 refresh-time 30 !
From the routers, checking the connectivity status to the RTR server should display something similar to this:
RP/0/0/CPU0:router-asn10#sh bgp rpki server summary Tue Jul 27 14:29:11.497 CET Hostname/Address Transport State Time ROAs (IPv4/IPv6) 192.168.122.253 TCP:8282 ESTAB 04:41:46 5/1
Enabling BGP destinations validation
Now that all the legitimate ASNs are participating in the RPKI process, I should be able to quickly identify invalid BGP destinations. As expected, displaying the IPv4 unicast BGP database of the router belonging to ASN 10, reveals that the destination towards ASN 30 is not verified:
RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast origin-as validity | b ^Origin-AS Tue Jul 27 14:42:25.998 CET Origin-AS validation codes: V valid, I invalid, N not-found, D disabled Network Next Hop Metric LocPrf Weight Path V*> 170.0.0.0/24 10.0.0.2 0 0 20 i I*> 170.0.0.1/32 10.0.0.6 0 0 30 i *> 172.0.0.0/24 0.0.0.0 0 32768 i Processed 3 prefixes, 3 paths
However, by default on CISCO-XR routers, invalid destinations are not automatically discarded, so I need to manually enable the valid destinations selection with this command:
router bgp <ASN> bgp bestpath origin-as use validity
Immediately after the new configuration is committed, only the valid destinations are selected and any risk of IP hijacking is finally defeated:
RP/0/0/CPU0:router-asn10#traceroute 170.0.0.1 Tue Jul 27 14:55:23.282 CET Type escape sequence to abort. Tracing the route to 170.0.0.1 1 10.0.0.2 0 msec * 9 msec
See the list below for references and resources I used for this experiment, as well as a link to my GitHub repository where you can clone and test my lab for yourself.
Clone and test:
Documentation:
NET resources:
- router ASN 10 configuration
- router ASN 20 configuration
- router ASN 30 configuration
- net verification commands
Service resources:
Adapted from the original post which appeared on Medium.
Salvatore Cuzzilla is an ICT System Engineer at Swisscom.
To learn more about RPKI, check out the
APNIC Academy RPKI Deployment Webinar
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.