Resource Public Key Infrastructure (RPKI) is designed to secure traffic over the Border Gateway Protocol (BGP) and protect Internet infrastructure from BGP hijacking.
In this post, I will show how RPKI can be set up on a testbed with PMACCT to collect and validate RPKI/Route Origin Authorization (ROA) information.
Network preparation
Simulators are useful tools to set up a testbed and troubleshoot potential problems when building a new network, or migrating current infrastructure.
Both routers (RTR-01 and RTR-02) should be configured with NetFlow and the traffic exporting port address (port 9999 in the example) should be the same as that of data collector (pmbgpd/nfacctd).
RTR-01#flow record NetRecord
RTR-01#match ipv4 tos
RTR-01#match ipv4 protocol
RTR-01#match ipv4 source address
RTR-01#match ipv4 destination address
RTR-01#match transport source-port
RTR-01#match transport destination-port
RTR-01#match interface input
RTR-01#collect routing source as
RTR-01#collect routing destination as
RTR-01#collect interface output
RTR-01#collect counter bytes
RTR-01#collect counter packets
RTR-01#collect timestamp sys-uptime first
RTR-01#collect timestamp sys-uptime last
RTR-01#collect application name
RTR-01#flow exporter NetExport
RTR-01#destination X.X.X.X
RTR-01#source FastEthernet 0/0
RTR-01#transport udp 9999 // the same port address should be set for pmbgpd configuration port
RTR-01#template data timeout 60
RTR-01#option application-table timeout 60
RTR-01#option application-attributes timeout 300
RTR-01#flow monitor NetMonitor
RTR-01#exporter NetExport
RTR-01#cache timeout active 60
RTR-01#record NetRecord
Once you’ve completed the initial installation and integration of the testbed, you can start installing the PMACCT tool.
How to set up PMACCT on Centos 7
PMACCT is an IP traffic accounting tool licensed under the GNU General Public License (GPL). Before getting started with the PMACCT, we need to set up the underlying environment on which PMACCT will run.
Relying on MySQL for the underlying data collection is only recommended for test environments. For large sets of data collections, it doesn’t scale well and in that case, we need to make use of real-time data stores such as Apache Druid.
1. Install MySQL
yum install mysql-community-server yum install libtool yum install mysql-community-devel
2. Run MySQL
$ systemctl start mysqld
3. Create a new database
$ mysql –u root –p $ CREATE DATABASE pmacct;
Create a table for BGP data collection as follows:
$ mysql -u root -p < pmacct-create-db_v1.mysql $ mysql -u root -p < pmacct-grant-db.mysql create table acct_bgp ( id INT(4) UNSIGNED NOT NULL AUTO_INCREMENT, agent_id INT(4) UNSIGNED NOT NULL, as_src INT(4) UNSIGNED NOT NULL, as_dst INT(4) UNSIGNED NOT NULL, peer_as_src INT(4) UNSIGNED NOT NULL, peer_as_dst INT(4) UNSIGNED NOT NULL, peer_ip_src CHAR(15) NOT NULL, peer_ip_dst CHAR(15) NOT NULL, as_path CHAR(21) NOT NULL, local_pref INT(4) UNSIGNED NOT NULL, packets INT UNSIGNED NOT NULL, bytes BIGINT UNSIGNED NOT NULL, stamp_inserted DATETIME NOT NULL, stamp_updated DATETIME, PRIMARY KEY (id);
Create a table for netflow data collection as follows:
$ mysql –u root –p create table acct ( mac_src CHAR(17) NOT NULL, mac_dst CHAR(17) NOT NULL, ip_src CHAR(15) NOT NULL, ip_dst CHAR(15) NOT NULL, src_port INT(2) UNSIGNED NOT NULL, dst_port INT(2) UNSIGNED NOT NULL, ip_proto CHAR(6) NOT NULL, packets INT UNSIGNED NOT NULL, bytes BIGINT UNSIGNED NOT NULL, stamp_inserted DATETIME NOT NULL, stamp_updated DATETIME, PRIMARY KEY (mac_src, mac_dst, ip_src, ip_dst, src_port, dst_port, ip_proto, stamp_inserted) );
Then exit from the MySQL prompt.
4. Install PMACCT
The simplest way to get and configure the PMACCT package is to download it from the official webpage or clone it from GitHub and then follow the steps below to configure it:
$ git clone https://github.com/pmacct/pmacct $ cd pmacct $ ./autogen.sh $ ./configure - -enable-mysql $ make $ sudo make install
If you are using PostgreSQL or SQLite, follow the step below to configure it:
$ ./configure - -enable-pgsql $ ./configure - -enable-sqlite3
5. Create plugins and configuration files
Configuration files are used to define how the PMACCT tool can work for a different type of network traffic. There are various plugins available for the PMACCT tool to correlate network traffic with RPKI/ROA. Those that are useful for the networking testbed are:
pmacctd — This is a libpcap-based accounting daemon that captures network data packets from one or multiple interfaces. It can be used to collect data from the interfaces it configured and export this traffic data to the server system via NetFlow or sFlow protocols.
nfacctd — This is a NetFlow/IPFIX daemon that listens for NetFlow versions v5 or v9 and the IPFIX packet on all interfaces it configured for both IPv4 and IPv6. Like pmacctd daemon, it collects packets from the interfaces and can send or replicate this to third-party collectors.
pmbgpd — This is a standalone BGP collector daemon and can work as the passive iBGP and eBGP neighbor and can log real-time BGP traffic on regular intervals to the back-end systems (MySQL, Druid).
Configuration file for pmbgpd (BGP data collector)
The pmbgpd service supports 4-byte ASNs, IPv4 and IPv6. When you store the data in a customized MySQL table you can mix and match BGP primitives with other primitives.
pmbgpd is a standalone BGP collector daemon but it can be integrated with one of the above plugins. This thread solution is suitable for correlating BGP data with other data sources. The configuration file can be saved as a bgp.conf file and kept under the directory pmacct.
$ mkdir -p /etc/pmacct $ vim /etc/pmacct/bgp.conf daemonize: true nfacctd_port: 9999 nfacctd_time_new: true plugins: mysql sql_optimize_clauses: true sql_dont_try_update: true sql_multi_values: 1024000 sql_history_roundoff: m sql_history: 5m sql_refresh_time: 300 sql_table: acct_bgp bgp_daemon: true bgp_daemon_ip: X.X.X.X bgp_daemon_max_peers: 100 bgp_aspath_radius: 3 bgp_follow_default: 1 nfacctd_as: bgp bgp_peer_src_as_type: map bgp_peer_src_as_map: /etc/pmacct/peers.map plugin_buffer_size: 10240 plugin_pipe_size: 1024000 aggregate: tag, src_as, dst_as, peer_src_as, peer_dst_as, peer_src_ip, peer_dst_ip, local_pref, as_path pre_tag_map: /path/to/pretag.map maps_refresh: true maps_entries: 3840
Once the collector (pmbgpd) has been configured, the next step is to configure the router to export the traffic samples to the BGP collector (pmbgpd) on the Linux virtual machine. Cisco routers running iOS can be used in the following configuration:
router bgp xxx neighbor X.X.X.X remote-as xxx neighbor X.X.X.X update-source Loopbackxxx neighbor X.X.X.X version 4 neighbor X.X.X.X send-community neighbor X.X.X.X route-reflector-client neighbor X.X.X.X description nfacctd
6. Configuration file for nfacctd (NetFlow data collector)
$ mkdir -p /etc/pmacct $ vim /etc/pmacct/netflow.conf interface: eth0 sql_max_writers: 25 sql_optimize_clauses: true pmacctd_force_frag_handling: true debug: true pmacctd_stitching: true plugins: mysql[c] aggregate[c]: src_host, dst_host, proto sql_db[c]: pmacct sql_table[c]: acct sql_passwd[c]: xxxxx sql_user[c]: root sql_refresh_time[c]: 6 timestamps_secs: true plugin_buffer_size: 10240 plugin_pipe_size: 1024000 snaplen: 1500
7. Running PMACCT
You need to run the following command as a root user under the directory where you install pmacct:
nfacctd –f /etc/pmacct/bgp.conf //
If you configure either pmbgpd or nfacctd, you can use the same command nfacctd followed by –f and the file name. The configuration file defines the type of data to be collected from the router.
To stop running the service, use the following command:
kill –INT nfacctd
You can verify your results on MySQL table’s acct for Netflow and acct_bgp for BGP data.
8. Configuring RPKI/ROA information
This configuration can be done either using downloaded RPKI data (ROAs) or via rpki_rtr_cache. The first method requires all ROAs to be downloaded and the second method can be used for live ROA updates.
ROAs can be imported from the rpki_roas_file config directive from a file into the RIPE NCC Validator format, as below:
{
"roas": [
{"asn": "AS0", "maxLength": 11, "prefix": "80.128.0.0/11", "ta": "ripe"},
{"asn": "AS0", "maxLength": 128, "prefix": "2001:12f8::/48", "ta": "lacnic"},
{"asn": "AS0", "maxLength": 128, "prefix": "2001:43f8::/48", "ta": "afrinic"},
);
The data in the above format can be downloaded from: https://rpki.gin.ntt.net/api/export.json
Once the rpki_roas_file or rpki_rtr_cache are defined, the configured traffic daemons above (pmbgpd and nfacctd) can be used to correlate the traffic with BGP and RPKI. The configuration file is as follows:
nfacctd_as: longest nfacctd_net: longest ! < .. > ! ! rpki_roas_file: /path/to/roas.json // file can be downloaded and saved to file. ! rpki_rtr_cache: 127.0.0.1:9999 ! rpki_rtr_cache_version: 1 ! bgp_daemon: true bgp_src_roa_type: bgp ! < .. > ! plugins: print[foo] // Here the data will be printed to a file. ! aggregate[foo]: peer_src_ip, peer_dst_ip, src_host, dst_host, src_roa, dst_roa // user can add more attributes like src_as, dst_as, etc. ! < .. >
The ROA field will be populated with one of the following values:
u — Unknown
v — Valid
i — Invalid
V — Invalid with a covering ‘Valid’ prefix
U —Invalid with a covering ‘Unknown’ prefix
Best current practice recommends having multiple RPKI/ROA validators such as ROUTINATOR, and RIPE NCC VALIDATOR installed and configured in the testbed.
Shinoj Pittandavida is a Network and Systems Engineer at IE Domain Registry CLG.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.