A network operator perspective on IPv6 performance

An often-overlooked source of data reflecting Internet performance and function is the data within its constituent networks. Often, this data is not collected or considered within a given network, let alone across and between networks. The TechArk Network Operator Measurement Activity (NOMA) aims to provide a neutral project platform to build a collaborative resource for Internet health metrics measurements. The first focus is on tracking the relative performance of IPv6 and IPv4 for HTTP traffic.

The plan

The approach of NOMA is to generalize and expand what some operators already do:  take (simple) measurements from within operational networks to determine state and track trends of particular aspects of network function and experience.

These measurements are taken from multiple collection points spread across the access network. This gives a picture of the users’ experience of the network. (See this case study for a detailed description of an architecture for performance measurement in broadband networks). For a first effort, it made sense to look at the question of IPv4 versus IPv6 network performance for users, as this is a timely topic.

So, what are “simple” measurements? Using “libcurl” and http as the closest approximation of end-users’ experience, each collection point can gather:

• IPv6 DNS lookup to each target website
• Time to connect to the target website over IPv6
• Total time for each target website over IPv6
• IPv4 DNS lookup to each target website
• Time to connect to the target website over IPv4
• Total time for each target website over IPv4

With the timing numbers in hand, it’s easy to calculate a ratio of IPv4/IPv6 times as a simple test of whether IPv4 or IPv6 is performing better:

• < 1 means IPv4 activity is faster
• > 1 means IPv6 activity is faster
• =1 means IPv4 & IPv6 are the same

While the ratio is useful as a quick gauge of the state of affairs, it’s also useful to calculate the difference between the time for IPv4 and IPv6 tasks, as the ratio can blur important distinctions in total times.

Making it happen — RIPE Atlas

The measurements and calculations are simple — if you have the network from which to measure them. Absent that, and in order to illustrate the kind of results that are possible, NOMA has used the RIPE Atlas infrastructure to simulate the approach and to get real data. In our case, two US-based broadband networks were selected, and the RIPE Atlas probes within those networks were used to do measurements as if the networks were participating in the overall NOMA project.

There are some important differences between doing measurements using the RIPE Atlas infrastructure and running them from within one’s own network: all the http queries had to be directed to RIPE Atlas Anchors (that is, not generic websites such as ‘http://www.facebook.com’). Each probe reports its geographic coordinates.

Lacking any knowledge of network topology, this project used the probes’ coordinates to identify clusters of probes around geographic “pins” — each probe is clustered to its nearest pin, geographically. For the tests reported here, the pins were the geographic coordinates of the 12 RIPE Atlas Anchors that were used.

Some results

The measurements used for the data shown here were carried out every 30 minutes over a period of approximately one hour — that is, two tests per probe[1]. Clearly, longer-term measurements collection and review of data over time would be interesting, but they were beyond the scope of this phase of the project.

Each table shows the average responses from all responding probes in a cluster.

Each cluster towards a particular target

Here’s a sample of the kind of data one sees when having all probes measure towards a particular target. Pin 6 (the RIPE Atlas anchor identified in San Diego) was selected as the target.

IPv4/IPv6 Ratio

In the table below, green indicates that the ratio of the times (milliseconds) for DNS resolution, connection to the http server, to first byte, and overall execution is greater than one. That is, it took longer to complete the task over IPv4 than IPv6.

Pin	Locality	DNS (v4/v6)	Connection (v4/v6)	Time to first byte	Execution (excl DNS) (v4/v6)
0	Atlanta	0.934	1.061	1.055	1.0555
1	Brookline	7.155	0.958	0.994	0.994
2	Chicago	0.843	0.981	0.993	0.991
3	Dallas	1.331	1.116	1.125	1.125
4	Denver	0.896	0.928	0.980	0.979
5	Los Angeles	1.819	1.197	1.103	1.110
6	San Diego	0.737	1.249	1.453	1.471
7	Coral Gables	1.288	1.205	1.204	1.205
8	Palo Alto	12.826	0.372	0.646	0.648
9	Phoenix	0	0	0	0
10	Ashburn	4.459	0.812	0.833	0.832
11	Seattle	0.841	1.009	0.977	0.973

Difference

In the table below, green indicates that the difference of the times (milliseconds) for DNS resolution, connection to the http server, to first byte, and overall execution is greater than zero.  That is, it took longer to complete the task over IPv4 than IPv6.

Pin	Locality	DNS (v4-v6)	Connection (v4-v6)	Time to first byte (v4-v6)	Execution (excl DNS) (v4-v6)
0	Atlanta	-2.307	4.103	7.504	7.602
1	Brookline	139.178	-4.178	-1.251	-1.163
2	Chicago	-3.631	-1.315	-1.003	-1.226
3	Dallas	13.031	5.860	12.867	12.894
4	Denver	-2.560	-3.732	-2.291	-2.370
5	Los Angeles	16.845	3.403	4.084	4.437
6	San Diego	-21.130	4.227	14.656	15.328
7	Coral Gables	8.028	16.240	32.794	33.066
8	Palo Alto	216.735	-48.686	-37.315	-37.289
9	Phoenix	0	0	0	0
10	Ashburn	75.019	-19.237	-34.819	-35.239
11	Seattle	-3.454	0.372	-2.031	-2.346

Each cluster towards its closest target

Here’s a sample of the kind of data one sees when having all probes measure towards only a target that is (geographically) closest to itself.  This is an approximation of “near” in network terms, although it’s impossible to be certain of accuracy without any knowledge of topology.

IPv4/IPv6 Ratio

In the table below, green indicates that the ratio of the times (milliseconds) for DNS resolution, connection to the http server, to first byte, and overall execution is greater than one. That is, it took longer to complete the task over IPv4 than IPv6.

Pin	Locality	DNS (v4/v6)	Connection (v4/v6)	Time to first byte (v4/v6)	Execution (excl DNS) (v4/v6)
0	Atlanta	1.371	1.401	1.465	1.468
1	Brookline	2.521	1.025	0.990	0.999
2	Chicago	1.131	0.891	0.941	0.939
3	Dallas	0.806	2.300	1.397	1.396
4	Denver	4.088	1.292	1.562	1.554
5	Los Angeles	0.970	1.178	1.197	1.198
6	San Diego	0.737	1.250	1.453	1.471
7	Coral Gables	2.377	2.668	2.592	2.588
8	Palo Alto	17.318	0.154	0.346	0.347
9	Phoenix	0	0	0	0
10	Ashburn	1.760	1.037	1.056	1.031
11	Seattle	5.008	1.090	1.065	1.064

Difference

In the table below, green indicates that the difference of the times (milliseconds) for DNS resolution, connection to the http server, to first byte, and overall execution is greater than zero.  That is, it took longer to complete the task over IPv4 than IPv6.

Pin	Locality	DNS (v4-v6)	Connection (v4-v6)	Time to first byte (v4-v6)	Execution (excl DNS) (v4-v6)
0	Atlanta	10.956	7.158	16.850	17.052
1	Brookline	32.148	0.602	-0.521	-0.072
2	Chicago	2.557	-2.329	-2.569	-2.714
3	Dallas	-11.495	63.726	39.447	39.554
4	Denver	86.375	4.061	17.540	17.606
5	Los Angeles	-1.815	2.555	6.180	6.325
6	San Diego	-21.130	4.227	14.656	15.328
7	Coral Gables	19.840	26.353	53.962	54.141
8	Palo Alto	330.7820	-100.416	-89.477	-90.013
9	Phoenix	0	0	0	0
10	Ashburn	70.914	0.917	3.043	1.729
11	Seattle	80.526	1.300	2.125	2.143

These tables share enough data to be tantalizing. Network operators would naturally ask what caused the numbers above (traceroutes, which would be helpful for answering those questions, were collected but not analyzed). Since this is, effectively, a single snapshot in time, other questions arise as to whether the errors were transient and if the performance numbers are stable.

Discussion

These results are pretty preliminary, but they do open the door to further study. For instance, it seems the performance of IPv6 is better when measuring to a “near” target. One hypothesis is that performance improvement is because transit networks are not as friendly to IPv6 traffic as access networks. Other hypotheses are also possible, and only testing will tell.

Also, as it stands, the RIPE Atlas probe coverage is quite low (a few hundred probes in one network, and a couple dozen in the other). They are unevenly distributed throughout the networks, as the probe network is grown organically (as opposed to being planted by the operator for measurement purposes).

Nevertheless, while these results are pretty preliminary, they do highlight the value of the in-network perspective on IPv4 and IPv6 performance, and motivate further study. For instance, it seems the performance of IPv6 is better when measuring to a “near” target. One hypothesis is that performance improvement is because transit networks are not as friendly to IPv6 traffic as access networks. Other hypotheses are also possible, and only testing will tell.

For more information about the NOMA project, stay tuned to http://www.techark.org/noma .

 [1] These particular measurements were run on July 16, 2017.

Leslie Daigle is Principal at ThinkingCat Enterprises.

Rate this article

---

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.