Deploying a RIPE Atlas software probe via PPP

By on 28 Jun 2021

Category: Tech matters

Tags: , , ,

Blog home

In February 2020, RIPE NCC announced the availability of software probes to improve RIPE Atlas coverage. With software probes, instead of installing a physical network device (the probe) to your network, you install the software package for the probe on a machine connected to your network. RIPE provides packages to install the probe at CentOS and Debian distributions. There is even a Docker image developed by the community. I used the latter option to host another probe on an OrangePI.

One might reasonably wonder though: ‘Hey, do you trust that this device won’t intercept your local network traffic, or tamper with your other devices in any way?’ As stated in the FAQ, the probe executes measurement commands against the public Internet only. For the security cautious, it is advised that the probe should be installed outside the local network (such as on a DMZ network).

I had followed this recommendation with my hardware probe before, but how am I going to perform such isolation in Docker? To make matters worse, I am using the CPE provided by my ISP, which has very limited capabilities in terms of configuring new networks. One possible solution is to reject any traffic from my probe’s network to internal networks via iptables. However, this would require additional steps when redeploying the RIPE Atlas probe. Ideally, my goal is to deploy the software probe out-of-the-box using a Docker compose manifest, without any additional commands.

So, what if the network traffic of our software probe container can be proxied to another service, which in turn tunnels it to the public Internet? In this post we are going to describe how to achieve this, by spinning up a dedicated Point-to-Point Protocol (PPP) connection with our ISP in Docker and using it as a ‘gateway’ to our software probe container.

Design

Deploying a RIPE Atlas software probe via PPP is depicted in Figure 1.

Figure 1 — The overall design.
Figure 1 — The overall design.

Our goal is to initiate a new PPPoE connection from a container inside an isolated network, so our RIPE Atlas probe can only perform its measurements via a PPP connection, without interfering with adjacent containers on the same host, as depicted in Figure 1.

Retrieving your username and password

Some ISPs (like mine) supply the username and password on paper along with CPE equipment. If yours did not, you can retrieve credentials from a configuration backup of your CPE.

The required PPP configuration can be easily generated using the pppoeconf tool. It is an ncurses based tool that helps you populate /etc/ppp/peers/dsl-provider and /etc/ppp/chap-secrets with the correct values. The next step is to dockerize the PPP client. I created a small Docker image that simply invokes the PPP client. Providing our configuration is under a file named ‘dsl-provider’ and located under /etc/ppp/peers directory, we could invoke the container as follows:

$ docker run --name ppp -d --privileged \
     --device=/dev/ppp \
     - v/etc/ppp:/etc/ppp \
     tzermias/ppp call dsl-provider

The PPP client requires access to /dev/ppp device to work properly, so we need to expose it accordingly. Furthermore, you need to use the macvlan network driver for the ppp container to work properly. A dedicated macvlan network can be created using the command below.

$ docker network create -d macvlan \
     --subnet=172.16.86.0/24 \
     --gateway=172.16.86.1
     -o parent=eth0 ppp

The ppp container can be attached to the network by adding the –network ppp flag on the docker run command above.

If everything went well, logs of the ppp container would indicate a successful call is performed and a new IPv4 address is assigned to the ppp0 interface of our container.

$ docker logs ppp
Plugin rp-pppoe.so loaded.
PPP session is 394
Connected to d8:67:d9:48:XX:XX via interface eth0
Using interface ppp0
Connect: ppp0 <--> eth0
PAP authentication succeeded
peer from calling number D8:67:D9:48:XX:XX authorized
local  LL address fe80::dead:beef:dead:beed
remote LL address fe80::beef:beef:dead:dead
replacing old default route to eth0 [172.16.86.1]
local  IP address XX.XX.XX.XX
remote IP address YY.YY.YY.YY

The PPP client will also create a resolv.conf file under /etc/ppp, containing DNS nameservers obtained from our peer. For this, you need to have the usepeerdns option in your /etc/ppp/peers/dsl-provider file.

Container networking

Now comes the interesting part. How can we ensure that any network traffic from our ripe-atlas container will only pass through the ppp container we started earlier? The answer is to share the network stack of this container. Apart from the usual bridgehost, or none options that one can pass to -network setting, there’s a special option named container, which allows you to use the network stack of another container. According to docker run reference:

With the network set to container, a container will share the network stack of another container. The other container’s name must be provided in the format of –network container:< name|id>.



Example running a Redis container with Redis binding to localhost then running the redis-cli command and connecting to the Redis server over the localhost interface.

With the ppp container running, we can easily validate that our setup works, just by spinning up a test container as shown below and performing some tests (such as a traceroute). Don’t forget to mount the /etc/ppp/resolv.conf file in this container, to have proper DNS resolution.

$ docker run -it --network ppp \
      -v /etc/ppp/resolv.conf:/etc/resolv.conf:ro \
      busybox:latest traceroute google.com

RIPE Atlas in Docker

I used a ripe-atlas image by jamesits for this purpose. Probe installation and registration are straightforward and well documented. On the first run, the probe generates an SSH keypair and saves it under /var/atlas-probe/etc/probe-key.pub. Then, you simply create a new directory where any relevant ripe-atlas files will be stored and mount it to the container.

$ mkdir -p atlas-probe

$ docker run --network "container:ppp" \
        --detach --restart=always --log-opt max-size=10m \
        --cpus=1 --memory=64m --memory-reservation=64m \
        --cap-add=SYS_ADMIN --cap-add=NET_RAW --cap-add=CHOWN \
        --mount type=tmpfs,destination=/var/atlasdata,tmpfs-size=64M \
        -v ${PWD}/atlas-probe/etc:/var/atlas-probe/etc \
        -v ${PWD}/atlas-probe/status:/var/atlas-probe/status \
        -v /etc/ppp/resolv.conf:/etc/resolv.conf:ro \
        -e RXTXRPT=yes --name ripe-atlas \
        jamesits/ripe-atlas:latest

After uploading the public key to RIPE Atlas WebUI, the probe is ready to accept new measurements.

IPv6 support

For those who wish the probe run IPv6 measurements, you can enable IPv6 support by performing the following modifications:

Firstly, you need to enable IPv6 support in the Docker daemon. Ensure that your /etc/docker/daemon.json contains the following:

{
  "ipv6": true,
  "fixed-cidr-v6": "2001:db8:1::/64"
}

Use $ systemctl restart docker to ensure that the Docker daemon is aware of those settings.

The next step is to make sure that our ppp container has IPv6 support enabled. For this we need to start our container with net.ipv6.conf.all.disable_ipv6=0 set.

And the last step (eventually) is to enable IPv6 support when performing the PPP call. This can be achieved by adding the following two options in our /etc/ppp/peers/dsl-provider config.

+ipv6
ipv6cp-use-persistent

Putting it together

To sum up, one should execute the following commands to have the probe and the PPP client up and running:

# Create network
docker network create -d macvlan --subnet=172.16.86.0/24 \
      --gateway=172.16.86.1 -o parent=eth0  ppp

# Create ppp container
docker run --name ppp --network ppp --restart unless-stopped\
       -d --privileged --device=/dev/ppp \
       --sysctl net.ipv6.conf.all.disable_ipv6=0  \
       -v /etc/ppp:/etc/ppp \
       tzermias/ppp call dsl-provider

# Create directory for atlas-probe config
mkdir -p atlas-probe

# Create ripe-atlas container
docker run --network "container:ppp" \
        --detach --restart=always --log-opt max-size=10m \
        --cpus=1 --memory=64m --memory-reservation=64m \
        --cap-add=SYS_ADMIN --cap-add=NET_RAW --cap-add=CHOWN \
        --mount type=tmpfs,destination=/var/atlasdata,tmpfs-size=64M \
        -v ${PWD}/atlas-probe/etc:/var/atlas-probe/etc \
        -v ${PWD}/atlas-probe/status:/var/atlas-probe/status \
        -v /etc/ppp/resolv.conf:/etc/resolv.conf:ro \
        -e RXTXRPT=yes --name ripe-atlas \
        jamesits/ripe-atlas:latest

I also created the following docker-compose.yaml file to spin up the network and the containers to avoid typing those commands again and again.

For those who wish to use this probe for their measurements, just use probe #1001487.

Software probes can increase the reach of RIPE Atlas platforms on more networks and geographic locations. With 260 probes currently deployed in Greece, we can clearly observe a large portion of them being deployed in the Athens metro area. Wouldn’t it be nice to have Internet measurements on less densely populated areas, like islands?

Furthermore, the option to share one container’s network stack with another can be leveraged on other applications, like proxying traffic through a VPN tunnel. If you have a RaspberryPi sitting idle on your network, I suggest you give it a try!

Aris Tzermias is an experienced Site Reliability Engineer focusing on automation and deployment of services, with a systems administrations background.

This post was originally published at Aris Tzermias’ blog.

Rate this article

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Top