Network Fabrics

Traditional layerd design

For decades, networks were built in a hyrearchical way using

• Access

• Distribution

• Core

These networks were built to support traffic going up and down the hierarchy, therefore being better suited for situation where North-South traffic is dominant.

Furthermore, when using switched networks, by introducing redundancy you also introduce the possibility to create loops. In order to avoid that we started to use Spanning Tree protocols that prevent loops from forming, but that also some of the links were purposely disabled so the capacity wasn't fully used.

Evolutions of this design eventually settled on use of vPC where you can create virtual PortChannels (MCLAG - Multi Chassis Link Aggregation) in order to get loop free multi line use.

To scale the application usage, you can scale-up the servers supporting the application and then your network infrastructure needs to scale-up with the capacity needed for the servers. Eventually you can reach the limits of what the core network can scale up to.

Network Fabrics

Another option is to scale-out. Instead of adding more capacity to a single server and having more North-South traffic, you can add multiple servers that can "cluster" or work together to provide a similar capacity. That means you suddenly get more East-West traffic.

The idea of scaling out ties well to the use of Network Fabrics. The typical example of this is a Clos Architecture where you can have multiple input and output stages connected by a middle stage

Modular devices can be easier to scale-up but they also add complexity to the network because they are running an internal fabric to connect all modules.

Now in order to provide connectivity between leafs you can use L2 (with most of its downsides). This is what Cisco's proprietary FabricPath technology did but it is now considered obsolete. Although FabricPath used IS-IS, it used it to advertise MAC (so L2) reachability.

A better approach is to use L3 connectivity between leafs. With routed networks, you can take advantage of ECMP (Equal Cost Multi Pathing) to utilize all existing links. The modern and standardized network fabrics are built as L3 networks.

Realities of todays networking also bring us the idea that we need to provide multiple isolated network services on the same physical network, so the idea of virtualizing the network functions to support multiple use cases or tenants has a lot of support. This can be created by splitting the network into an underlay network (basically connected all the leafs and the spines) and several overlay networks that run on top of the underlay network.

Down at packet level, overlay just means encapsulating user data so that it can be transmitted through the underlay network.

What kind of overlay services can we offer?

• Layer 2 overalys

• Layer 3 overlays

• Layer 2 + Layer 3 overlays

VXLAN

VXLAN is an encapsulation protocol (RFC 7384) which is used to encapsulate L2 frames (MAC) into UDP packets (and therefore IP packets). The advantages of VXLAN are:

• uses UDP so it ls transported over L3 networks which can provide a loop free, ECMP network

• uses UDP so it can reduce traffic polarization. By using variable UDP source ports, the underlay entropy (you can say variation in hashing) is increased thus making the load sharing more effective across multiple paths. The destination port remains fixed (UDP 4789) which make it easy to be identified. UDP doesn't provide reliability but this can be handled by the application whos traffic is encapsulated in the VXLAN UDP packets

• supports network segmentation at scale through the use of VXLAN VNIs (24 bits) resulting in 16 million segments instead of the traditional VLAN ID (12 bits - 4096 segments)