Arthur Cole spoke with Brandon Hoff, director of product management for Emulex.
Virtual LANs were first developed as a means to help enterprises cope with the increased traffic from server virtualization. In the cloud, however, even vLANs can't seem to keep up. What's needed is an entirely new breed of dynamic network infrastructure that can scale to astronomical levels while maintaining support for multi-tenant environments. Emulex's Brandon Hoff says the most promising development of late is overlay networking (ON), which combines Layer 2 and Layer 3 technologies in support of elastic compute architectures. The question is which of the two ON standards currently under development will capture the market.
Cole: It seems that virtual LAN technology was a lifesaver when enterprises were trying to cope with the extra traffic from virtualized servers. How is the technology holding up in the cloud?
Hoff: vLAN technology has been a great first step for enterprises that are expanding their virtual infrastructure because the hypervisor can present multiple networks for all types of communications — the hypervisor management interface can be on one network, each tenant in a multi-tenant data center can have their own network for isolation, and multiple networks can be moved around the data center to support virtual machine migration operations over L2 networks. But as the enterprise evolves its infrastructure beyond virtualization to cloud computing, it needs to focus on application consolidation and virtual infrastructure deployment that can support dynamic and elastic compute environments, as well as the new network requirements that are being driven by the cloud. It needs to address requirements for elastic and dynamic compute infrastructures at scale. In the enterprise, a handful of tenants, such as enterprise divisions or departments, may live in the on-premise virtual infrastructure, but in the cloud the virtualized infrastructure must serve multiple customers, individual divisions or departments inside each customer, and be flexible enough that virtual workloads can be moved into, out of and between clouds, on demand.
The current practice of host-centric segmentation based on vLANs and L2 subnets, with server clusters separated by L3 physical IP networks, limit today’s multi-tenant cloud infrastructure from meaningful network scaling for longer-term growth. Basically, the number of vLANs available to the cloud isn’t large enough.
To scale existing networking technologies for multi-tenant infrastructures, enterprises need to deploy new solutions that can enable VM communication and migration across Layer 3 boundaries without impacting connectivity. At the same time, they must also ensure isolation for thousands of logical network segments and maintain existing VM IP addresses and Media Access Control (MAC) addresses no matter where the workload is migrated.
The answer is overlay networking. Overlay networking creates a virtual network where the traffic from each VM is mapped to a specific virtual network; the packets are encapsulated in a MAC-in-IP format, and then routed transparently over the existing physical infrastructure. There are currently two formats for building overlay networks under consideration in the Internet Engineering Task Force (IETF). These are Network Virtualization using Generic Routing Encapsulation (NVGRE), initiated by Microsoft and others, and Virtual eXtensible Local Area Networks (VXLAN), initiated by VMware and others. Both standards are designed to enable the efficient and fluid movement of virtual resources across cloud infrastructures for large-scale and cloud-scale VM deployments.
Cole: How exactly will they help vLANs scale to cloud levels?
Hoff: NVGRE and VXLAN each provide L2 overlay schemes over an L3 network to enable virtual L2 connections running over two or more physical L3 networks, while giving the appearance to the virtual workloads that they share the same physical L2 subnet — even though the workloads may be running anywhere. This enables inter-VM communications or VM migrations across L3 networks that operate as if they belonged to the same L2 subnet, allowing for cost-effective, cloud-level scalability. Both of these standards encapsulate Ethernet L2 Frames into an IP packet marked by a new 24-bit identifier. These new identifiers enable more than 16 million L2 logical networks to operate within the same administrative domain, a scalability improvement of many orders of magnitude over the 4,094 vLAN segment limit prescribed by the IEEE.
NVGRE is designed to encapsulate Ethernet L2 Frames in a GRE packet. GRE is an existing protocol that has been in use for a long time. Now, it is extended all the way back to the server as the end-point and managed through software-defined networking.
VXLAN is a new format designed to encapsulate Ethernet L2 Frames in a UDP packet. One benefit of the VXLAN standard is that it does define some control plane functionality, which may be beneficial in some network environments.
It will be interesting to see how these two standards evolve since, at least initially, they will co-exist. Microsoft has built NVGRE into Windows Server 2012 and VMware has built VXLANs into VMware 5.1 — both of which are expected to launch this year.
Cole: As more logical layers are stacked on top of each other, don't enterprises, or even telecommunications companies, run the risk of hampering network performance across the board?
Hoff: For enterprise data centers, and for telecommunication companies as well, east-west traffic was initially limited to within a rack or between a handful of racks that were on a physical L2 network. Today, east-west traffic is growing to become a dominant data center traffic type. As hybrid clouds expand and consolidate multiple customers on to the same physical infrastructure with overlay networking, there is a risk of hampering network performance. For example, a high performance workload that requires high-bandwidth connections to multiple servers cannot be individually moved into the cloud because the network connection may become a bottleneck. Also, extending an overlay network across low-bandwidth L3 network links can also cause performance problems. To ensure that each workload gets the network resources it needs, network virtualization management tools are essential.
While overlay networks can run independently of the physical network, they are dependent on the performance and availability of the underlying network. Therefore overlay network policies must reflect the underlying infrastructure, and likewise, physical network changes must take into account the virtual networks being supported. Software management layers that integrate virtual machine management with virtual and physical network management are an important piece of the puzzle. These are technologies that Emulex is also bringing to the market as overlay networks become more popular.