In two words, what is the best way to build a large-scale network?
Ask ten networking folks (engineers, designers, or whatever else), and you’re likely to get the same answer from at least nine: clean abstractions. They might not say the word abstraction, of course; instead, they might use words such as build things in modules, using summarization and aggregation to divide the modules up. Or they might say “make certain to reduce the failure domain to the smallest you possible can everywhere you can”, or “use hierarchical design”. These answers are, however, variants of the single word: abstraction.
This response came to mind when I was reading an article on clean code (it’s amazing how often software architecture overlaps with network architecture):
Once we learn how to create abstractions, it is tempting to get high on that ability, and pull abstractions out of thin air whenever we see repetitive code. After a few years of coding, we see repetition everywhere — and abstracting is our new superpower. If someone tells us that abstraction is a virtue, we’ll eat it. And we’ll start judging other people for not worshipping [sic] “cleanliness”.
Dan Abramov, Overreacted
I have been teaching and writing about network design for many, many years — I co-authored my first book on the subject, ‘Advanced IP Network Design‘, in 1998. The entire object that book was to teach hierarchical network design, which relies on modularization through aggregation and summarization to separate complexity from complexity (though I didn’t really use this wording until many years later) in order to break up failure domains.
It has been 22 years since Don, Alvaro, and I wrote that book and hierarchical network design is still as relevant today as it was then. But in this time I’ve still learned plenty more about network design.
Among the lessons I’ve picked up is this one: ‘if you haven’t found the trade-offs, you haven’t looked hard enough’. Or perhaps ‘there is no such thing as a free lunch’. Abstraction is a superpower, and it can make your network a lot cleaner, even when you’re using it correctly (not using it to paper over complexity). But building the perfectly clean network can mean reducing the agility of the design to the point of fragility. For instance, in the article linked above, Dan Abramov notes changing requirements made his “clean revision” of the code much more complex — a classic sign of fragility.
Perhaps an example would be helpful here. If you think of the Routing Information Protocol (RIP) as a link-state protocol with summarization (abstraction of topology) at every hop — given you understand how link-state and distance-vector protocols work — you can probably quickly grasp what you have gained by summarizing at every hop, and what you have lost.
You should still use abstraction to break up failure domains. You should still use abstraction to separate complexity from complexity. But you should not use abstraction like you would any other tool. Rather you should decide the best places and times to use abstraction after understanding the whole system.
For instance, a lot of people really insist on aggregating routing information in their data centre fabric, especially in the underlay control plane. Why?
The underlay is a constrained routing domain with known properties. Aggregation in this environment can cause routing black holes and unpredictable traffic flow behavior, both of which require added complexity to ‘work around’. If there is another solution available, it might be best to use it.
At the same time, I see a lot of people insisting BGP is the only option for data centre underlays, or that it is the simplest option because you can use a single protocol for the underlay and overlay. This, in my opinion, is wrong, as well, because it does not properly separate two different parts of the network, each with their own purpose, into separate failure domains.
Rather than looking at a network and saying, “we can abstract here, so we should abstract here,” you should look at a network and say, “what are the modules here, and what purposes do they serve?” Once you know that, you can start thinking about when and were abstraction makes sense.
To paraphrase Dan,
Don’t be a clean network design zealot. Clean network design is not a goal. It’s a good guide when you don’t understand the network. Such guides are often useful, but they are guides rather than rules.
Adapted from original post which appeared on Rule 11.
Russ White is a Network Architect at Juniper Networks.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.