RPKI: (My)Lab environment

By on 11 Aug 2021

Category: Tech matters

Tags: , ,

Blog home

Recently I decided to set up a dedicated test environment to improve my knowledge around Resource Public Key Infrastructure (RPKI).

Now that the testbed is finally up and running, I would like to share my journey with the network operator community. I welcome feedback on this testbed, which I’ll use to help tune it.

Network setup

Using eve-ng, I linked a few routers to simulate a BGP IP hijacking attempt; aiming to neutralize this threat by implementing RPKI. For the sake of simplicity, I deployed only three routers, each one of them belonging to a different BGP Autonomous System (AS).

The router belonging to ASN 30 (as shown in Figure 1) started announcing an IP address that overlaps with an IP prefix range legitimately advertised by ASN 20. From now on, whenever ASN 10 tries to reach out to 170.0.0.1/32 it will be forwarded to the rogue router belonging to ASN 30.

Figure 1 — Network setup.

The BGP protocol will prefer the more specific prefix (170.0.0.1/32) over the less specific one (170.0.0.1/24). The traffic will flow towards the compromised ASN 30 via next-hop 10.0.0.6 (instead of 10.0.0.2).

 RP/0/0/CPU0:router-asn10#sh bgp sessions
 Mon Jul 26 14:28:00.527 CET

 Neighbor VRF Spk AS InQ OutQ NBRState NSRState
 10.0.0.2 default 0 20 0 0 Established None
 10.0.0.6 default 0 30 0 0 Established None

 RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast neighbors 10.0.0.2 
 routes | b ^Status
 Mon Jul 26 14:28:13.456 CET
 Status codes: s suppressed, d damped, h history, * valid, > best
               i - internal, r RIB-failure, S stale, N Nexthop-
 discard
 Origin codes: i - IGP, e - EGP, ? - incomplete
    Network Next Hop Metric LocPrf Weight Path
 *> 170.0.0.0/24 10.0.0.2 0 0 20 i

 Processed 1 prefixes, 1 paths

 RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast neighbors 10.0.0.6 
 routes | b ^Status
 Mon Jul 26 14:28:23.945 CET
 Status codes: s suppressed, d damped, h history, * valid, > best
               i - internal, r RIB-failure, S stale, N Nexthop-
 discard
 Origin codes: i - IGP, e - EGP, ? - incomplete
    Network Next Hop Metric LocPrf Weight Path
 *> 170.0.0.1/32 10.0.0.6 0 0 30 i

 Processed 1 prefixes, 1 paths

 RP/0/0/CPU0:router-asn10#traceroute 170.0.0.1
 Mon Jul 26 14:28:48.354 CET

 Type escape sequence to abort.
 Tracing the route to 170.0.0.1

  1 10.0.0.6 9 msec * 0 msec 
 RP/0/0/CPU0:router-asn10# 

RPKI IT infrastructure setup

Whenever it comes to RPKI, NLnetLabs is one of my main references. They’ve written clear documentation and developed the key software I used within (My)Lab infrastructure, including:

  • Krill, which can be classified as Certificate Authority (CA) software. In my setup, it is mainly responsible for forging and publishing the Route Origin Authorizations (ROAs) to the parent CA.
  • RTRTR, which can be classified as RPKI-to-router (RTR) protocol server software (RFC 6810-v0 and RFC 8210-v1). Its main job is to dispatch the validated ROAs to the BGP enabled routers.

Other than NLnetLabs, OpenBSD’s rpki-client is the core component of the whole solution. It’s responsible for:

  1. Synchronizing ROAs (X.509 Certificates), via RSYNC or RRDP, from a given Trust Anchor (TA). The TA is usually a Regional/Local Internet Registry (RIR/LIR) organization.
  2. Validating the chain of trust for the associated ROAs (including checking relevant certificate revocation lists).
  3. Caching a list of the validated ROA payloads (VRPs) in various formats, for example, JSON.
Figure 2 — Services setup.

Since I’m working within a test environment, I decided to collapse all services on a single Linux host. However, potentially nothing is preventing me from horizontally scaling each one of the involved services.

I recommend referring to the respective official documentation for the software installation and configuration — see the references and resources section below.

KRILL

Krill can be deployed using two different models: 

  1. Hosted RPKI 
  2. Delegated RPKI (I chose this)

After completing both the repository setup and the parent setup, I was ready to start publishing new ROAs.

I named the child CA rpki-alfanetti, and as you can see from the text snippet below its relationship with the parent CA (testbed offered by NLnetLabs) is in Status: success.

The parent CA certifies my entitlement over some specific resources, ASN: AS20 and v4: 170.0.0.0/24 are some of them. Third parties can, in a second stage, download the signed certificate (to be verified), which proves ownership over that specific resource.

 root@rpki01:~# krillc parents statuses --token 
 e1bb6e95c21740f83dba1adb1ff19ade --ca rpki-alfanetti
 Parent: testbed
 URI: https://testbed.rpki.nlnetlabs.nl/rfc8181/rpki-alfanetti
 Status: success
 Last contacted: 2021-07-27T08:50:00+00:00
 Next contact on or before: 2021-07-27T09:00:00+00:00
 Resource Entitlements: asn: AS10, AS20, v4: 170.0.0.0/24, 
 172.0.0.0/24, v6: 
   resource class: 0
   issuing cert uri: 
 rsync://testbed.rpki.nlnetlabs.nl/repo/ta/0/1040FA57F99F4B3DE2626F6E
 E1C56664CB81D2C8.cer
   received certificate(s):
     published at: 
 rsync://testbed.rpki.nlnetlabs.nl/repo/testbed/0/660E433B340BC5B12B20
 4F1544C6EA18A5931DB4.cer
     resources: asn: AS10, AS20, v4: 170.0.0.0/24, 172.0.0.0/24, 
 v6: 
     cert PEM:

 -----BEGIN CERTIFICATE-----
 MIIFsTCCBJ ... MojHUKkp30dIbbpo49FocyZyI58lFI7DsDVmXn9Bz0sAeYRB
 SNviq7K+O4SS/RZezDuY/5MEXHvDWsZXldfX1r+RxsgXi0/5fuudLU4CYlWQGzs
 -----END CERTIFICATE----- 

Read: How to: Run delegated RPKI in Krill

OpenBSD’s rpki-client

OpenBSD’s rpki-client is periodically syncing with the parent CA looking for new ROAs. To do so, it searches for the TA belonging to the parent CA — in particular, there’s a certificate used to validate received ROAs.

OpenBSD’s rpki-client is somehow hiding the complexity coming from the cryptography required to validate the received ROAs.

Once the validation process is completed, the information from the ROAs is extracted from the associated certificates and presented in clear text format (for example, JSON), ready to be ‘digested’ by the routers.

 ### root@rpki01:~# cat /var/lib/rpki-client/json
 {
         "metadata": {
                 "buildmachine": "rpki01",
                 "buildtime": "2021-07-26T08:04:49Z",
                 "elapsedtime": "133",
                 "usertime": "0",
                 "systemtime": "0",
                 "roas": 6,
                 "failedroas": 0,
                 "invalidroas": 0,
                 "certificates": 49,
                 "failcertificates": 0,
                 "invalidcertificates": 0,
                 "tals": 1,
                 "talfiles": "/etc/tals/testbed.tal",
                 "manifests": 49,
                 "failedmanifests": 43,
                 "stalemanifests": 0,
                 "crls": 6,
                 "repositories": 9,
                 "vrps": 6,
                 "uniquevrps": 6
         },
         "roas": [
                 { "asn": "AS65101", "prefix": "10.0.0.0/23", 
"maxLength": 24, "ta": "testbed" },
                 { "asn": "AS20", "prefix": "170.0.0.0/24", 
"maxLength": 24, "ta": "testbed" },
                 { "asn": "AS10", "prefix": "172.0.0.0/24", 
"maxLength": 24, "ta": "testbed" },
                 { "asn": "AS65001", "prefix": "192.168.0.0/23", 
"maxLength": 24, "ta": "testbed" },
                 { "asn": "AS37708", "prefix": "196.1.0.0/24", 
"maxLength": 24, "ta": "testbed" },
                 { "asn": "AS65001", "prefix": "2001:db8::/32", 
"maxLength": 64, "ta": "testbed" }
         ]
 } 

Read: How RRDP was implemented for OpenBSD rpki-client

HTTP server/RTRTR and BGP router configuration

A basic HTTP service (python3 -m http.server 8081) is allowing the RTR server to access the JSON file storing the ROA records. RTRTR is now listening for incoming connections from the BGP routers on the TCP socket <192.168.122.253:8282>.

The RTR server should be reachable by the involved routers. The below configuration should be present on each one of them.

 router bgp <ASN>
  rpki server 192.168.122.253
   transport tcp port 8282
   refresh-time 30
  ! 

From the routers, checking the connectivity status to the RTR server should display something similar to this:

 RP/0/0/CPU0:router-asn10#sh bgp rpki server summary      
 Tue Jul 27 14:29:11.497 CET

 Hostname/Address   Transport     State      Time        ROAs (IPv4/IPv6)
 192.168.122.253    TCP:8282      ESTAB      04:41:46    5/1 

Enabling BGP destinations validation

Now that all the legitimate ASNs are participating in the RPKI process, I should be able to quickly identify invalid BGP destinations. As expected, displaying the IPv4 unicast BGP database of the router belonging to ASN 10, reveals that the destination towards ASN 30 is not verified:

 RP/0/0/CPU0:router-asn10#show bgp ipv4 unicast origin-as validity | 
 b ^Origin-AS
 Tue Jul 27 14:42:25.998 CET
 Origin-AS validation codes: V valid, I invalid, N not-found, D 
 disabled
     Network        Next Hop      Metric     LocPrf Weight Path
 V*> 170.0.0.0/24   10.0.0.2        0            0 20 i
 I*> 170.0.0.1/32   10.0.0.6        0            0 30 i
  *> 172.0.0.0/24   0.0.0.0         0           32768 i

 Processed 3 prefixes, 3 paths 

However, by default on CISCO-XR routers, invalid destinations are not automatically discarded, so I need to manually enable the valid destinations selection with this command:

 router bgp <ASN>
  bgp bestpath origin-as use validity 

Immediately after the new configuration is committed, only the valid destinations are selected and any risk of IP hijacking is finally defeated:

 RP/0/0/CPU0:router-asn10#traceroute 170.0.0.1
 Tue Jul 27 14:55:23.282 CET

 Type escape sequence to abort.
 Tracing the route to 170.0.0.1

  1 10.0.0.2 0 msec * 9 msec 

See the list below for references and resources I used for this experiment, as well as a link to my GitHub repository where you can clone and test my lab for yourself.

Clone and test:

  1. RPKI (My)Lab @github

Documentation:

  1. NLnetLabs
  2. Krill
  3. OpenBSD rpki-client
  4. RTRTR
  5. EVE-NG

NET resources:

  1. router ASN 10 configuration
  2. router ASN 20 configuration
  3. router ASN 30 configuration
  4. net verification commands

Service resources:

  1. Krill configuration
  2. rpki-client systemd service configuration
  3. RTRTR configuration
  4. ROAs JSON format

Adapted from the original post which appeared on Medium.

Salvatore Cuzzilla is an ICT System Engineer at Swisscom.

To learn more about RPKI, check out the
APNIC Academy RPKI Deployment Webinar

Rate this article

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

Leave a Reply

Your email address will not be published.

Top