Open sourcing the network model

By on 15 Apr 2020

Category: Tech matters

Tags: , ,

Blog home

Wide Area Networks (WANs) play a large if often hidden part in the lives of billions of people that seek to not tie their information and devices to a single location.

However, as the reliance on these networks grows, it is becoming increasingly difficult to understand how to sustain their coverage and capacity. Traffic loads can become unpredictable and rapidly shift, circuit routes are not always available and, if they are available, they often take months to order and activate.

As these networks become denser, it becomes more difficult to understand where additional capacity is best deployed, especially if the network has an RSVP auto-bandwidth mesh. Planning a network under these conditions alone is a daunting task but add to this the need to plan for failure events, and the problem becomes exponentially more complex.

As more and more operators encounter this problem, it becomes increasingly clear that there is a need for a new way of approaching it. Planners must understand how a change in one part of the network will affect the network overall.

Network modelling plays an important role in this new approach. It allows planners to run simulations to understand how a change in one part of the network affects the network as a whole and where to most efficiently allocate capital for maximum value and minimum risk. For example:

  • How will traffic failover if a circuit goes down?
  • Does the network need more capacity to handle an expected increase in traffic?
  • Where should the new capacity be deployed to handle the increase in traffic?
  • Will all the RSVP LSPs be able to signal during any given failure?
  • When is the best time of day to conduct specific maintenance?

In a reasonably large, complex network, these questions are almost impossible to answer; or at least impossible to answer quickly, without a model.

Network modelling/simulations can be thought of in the same way as a flight simulator: both are very useful and give you repetition for scenarios you will see in real life. Understanding the simulated situations provides insight as to what will happen in real life. Getting these simulator repetitions makes you better at your job.

 

Network modelling as a solution

The network model, as a solution, has two main components: 

  • traffic matrix, which describes how much traffic is transiting the network to and from each Layer 3 node.
  • Network topology data, including things like Layer 3 nodes, circuits between the nodes, IGP metrics, RSVP label switched paths (LSPs), shared risk link groups (SRLGs), and even Layer 1 underlay.

modelling engine produces a simulation by applying the traffic matrix data to the topology data. It is responsible for routing the traffic the way an actual network would — routing RSVP LSPs, adjusting the reserved bandwidth on RSVP LSPs, routing traffic over an Interior Gateway Protocol (IGP) and/or LSPs, and so forth — and considers the IGP metrics, circuit capacity, and other factors to route traffic the way a Layer 3 network would.

The traffic matrix - slide from NANOG78 presentation.
Figure 1 — The traffic matrix describes how much traffic is transiting the network to and from each Layer 3 node.

The capability to model the network has been around since the early 2000s in the form of commercial modelling products such as Cariden MATE (now Cisco WAE), WANDL (now Juniper Northstar), and Aria Networks.

These commercial products typically provide a complete solution for network modelling: a method to create a traffic matrix, a modelling engine, a GUI that allows people to easily interact with and understand the model, and a set of APIs to allow programmatic interaction with the network model. Some operators have also created internal modelling capabilities that can perform some or all of the functions of the commercial modelling solutions.

The value of these commercial solutions is immense; although they can cost many tens to hundreds of thousands of dollars per year, the value they return is many times that amount. Having said this, we are at a point where some of these capabilities in commercial solutions can and should become open-source and modularized. This is thanks to:

  • The commercial modelling solutions being more mature.
  • Python’s ascendance in the networking community and growing maturity as a coding language.
  • Many more operators facing scaling and planning challenges in the WAN.
  • More established platforms for open-sourcing and sharing code.

pyNTM is one such open-source, python3-based WAN modelling engine. It allows a programmatic, API-based approach to network modelling.

Its APIs also allow users to describe the network topology and define a traffic matrix. This means that if an operator has the requisite knowledge (not money), they can now afford to unlock the value that network modelling provides.

FeatureCommercial optionspyNTM
Cost$$$$0
APIs for programmatic modellingYY
Includes capability to create traffic matrixYN
Sophisticated GUI for visualizationYN
Open-sourceNY
Dependent on vendorYN

pyNTM can help you now

If you have basic python coding skills, you can begin to use pyNTM right now. 

To start, take a look at the example scripts available in the GitHub repository. Use the example model data files to help construct a topology file for your own network. The sample scripts provide simple examples for how to load the topology, create a model, and interact with the model. 

The training modules are also good for this purpose. The basic information to describe your network topology is hopefully readily available. The traffic matrix does not need to be perfect to derive value: you can use a very general traffic matrix to test some basic assumptions about how different traffic will react during a failure or how the network will react to additional traffic. For example, you can model and track the paths of specific demands to determine if they will, in fact, failover to the expected paths. 

If your normal method of operation is to use general rules of thumb and assumptions about your network during a failure, simply test your assumptions and/or rule(s) of thumb. These basic tests do not demand a perfect traffic matrix. They also serve as important data points to help you start to understand your network and to help justify capital spending.

To start using pyNTM, you can download the repository from GitHub or download the python3 package from PyPi (pip3 install pyntm from the cli). The GitHub repository is very helpful because it has fully functional example scripts that can serve as examples to help you get started.

If there are some features you’d like to see and/or you’d like to contribute code, please do so on the issues page on the pyNTM GitHub repository.

To learn more about pyNTM, check out the free training modules, the examples in the Github repository, and read the docs website

You can also check out my NANOG 78 pyNTM presentation and view the video of the presentation below.

Tim Fiola is an automation and network modelling enthusiast.

Rate this article

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

Leave a Reply

Your email address will not be published. Required fields are marked *

Top