The data center has experienced rapid growth in technological innovation; from simple virtualization to a true software-defined data center. With this change, there has been an evolution of the network infrastructure that allows all of the pieces to communicate. We are seeing a change from the traditional three-tier architecture to a spine-and-leaf topology. Below I’ve done a quick comparison between the two architectures, how they’ve changed and the evolution of data center switching.
Traditional three-tier data center switching design historically consisted of core Layer 3 switches, aggregation Layer 3 switches (sometimes called distribution Layer 3 switches) and access switches. Spanning Tree Protocol was used between the aggregation layer and the access layer to build a loop-free topology for the Layer 2 part of the network. Spanning Tree Protocol had a lot of benefits including a relatively easy implementation, requiring little configuration, and being simple to understand. Spanning Tree Protocol cannot use parallel forwarding paths however, and it always blocks redundant paths in a VLAN. This impacted the ability to have a highly available active-active network, reduced the number of ports that were usable, and had high equipment costs.
The Fall of Spanning Tree Protocol
From this architecture, as virtualization started to grow, other protocols started to take the lead to allow for better utilization of equipment. Virtual-port-channel (vPC) technology eliminated Spanning Tree blocked ports, provided an active-active uplink from the access switches to the aggregation Layer 3 switches, and made use of the full available bandwidth. The architecture also started to change from the hardware standpoint by extending the Layer 2 segments across all of the pods. With this, the data center administrator can create a central, more flexible resource pool that can be allocated based on demand and needs. Some of the weaknesses of three-tier architecture began to show as virtualization continued to take over the industry and virtual machines needed to move freely between their hosts. This traffic requires efficiency with low and predictable latency. However, vPC can only provide two parallel uplinks which leads to bandwidth being the bottleneck of this design.
The Rise of Spine-and-Leaf Topology
Spine-and-leaf topology was created to overcome the bandwidth limitations of three-tier architecture. In this configuration, every lower-tier switch (leaf layer) is connected to each of the top-tier switches (spine layer) in a full-mesh topology. The leaf layer consists of access switches that connect to servers and other devices. The spine layer is the backbone of the network and is responsible for interconnecting all leaf switches. Every leaf switch is connected to every spine. There can be path optimization so traffic load is evenly distributed among the spine. If one spine switch were to completely fail, it would only slightly degrade performance throughout the data center. Every server is only a maximum number of hops from any other server in the mesh, greatly reducing latency and allowing for a smooth vMotion experience.
Spine-and-leaf topology can also be easily expanded. If you run into capacity limitations, expanding the network is as easy as adding an additional spine switch. Uplinks can be extended to every leaf switch, resulting in the addition of interlayer bandwidth and reduction of oversubscription. If device port capacity becomes a concern, a new leaf switch can be added. This architecture can also support using both chassis switches and fixed-port switches to accommodate connectivity types and budgets. One flaw of the spine-and-leaf architecture, however, is the number of ports needed to support each leaf. When adding a new spine, each leaf must have redundant paths connected to the new spine. For this reason, the number of ports needed can grow incredibly quickly and reduces the number of ports available for other purposes.
As demand grows in your data center, and east-west traffic increases, finding the right architecture for your needs is important. We spend a lot of time talking about the software-defined data center, and VMware NSX; however, addressing the core topology of your data center switching can be just as important. Spine-and-leaf can greatly impact your current workloads allowing for easier expandability, more efficient use of equipment, and higher redundancy.