Networking in VMware World

Terminologies in VMware Networking:

VMware Networking

The key virtual networking components provided by Virtual Infrastructure are:

    • Virtual Ethernet Adapters, used by individual virtual machines.
      In most cases, a virtual machine uses only one of the three types of virtual adapters.
      • vmxnet: works only if VMware Tools is installed in the guest operating system. The vmxnet adapter is designed for high performance. In most cases, when you select the flexible network adapter, this is the adapter used after VMware Tools is installed in the guest operating system.
      • vlance:  a virtual device that provides strict emulation of the AMD Lance PCNet32 Ethernet adapter. It is compatible with most 32-bit guest operating systems. This adapter is used when you select the flexible network adapter but VMware Tools is not installed in the guest operating system.
      • e1000: a virtual device that provides strict emulation of the Intel E1000 Ethernet adapter. This is the virtual Ethernet adapter used in 64-bit virtual machines. It is also available in 32-bit virtual machines.

The other virtual network adapter types are:

      • vswif: a paravirtualized device similar to vmxnet that is used only by the ESX Server service console.
      • vmknic: a virtual device in the VMkernel, the software layer that manages most of the physical resources on the ESX Server host. The vmknic is used by the TCP/IP stack that services VMotion, NFS and software iSCSI clients that run at the VMkernel level, and remote console traffic.

NOTE: The speed and duplex settings found in physical networking are not relevant in the virtual network, because all the data transfer takes place in the host system’s RAM, nearly instantaneously and without the possibility of collisions or other signaling-related errors.

  • Virtual Switches (vSwitch), which connects virtual machines to each other and connect both virtual machines and the ESX Server service console to external networks.
  • Uplink & Uplink Port: Physical Ethernet adapters serve as bridges between virtual and physical networks. In VMware Infrastructure, they are called uplinks, and the virtual ports connected to them are called uplink ports. A single host may have a maximum of 32 uplinks (Physical Ethernet adapters on server). In order for a virtual switch to provide access to more than one VLAN, the physical switch ports to which its uplinks are connected must be in trunking mode. It is important to prune the VLANs, keeping only those that are required for the virtual switch. Failure to do so can cause unnecessary overhead on the ESX Server host because it must process broadcast traffic for all VLANs trunked to it (As in virtual switch standard edition, port group can either be configured to access one single VLAN or allow all VLAN traffic, so VLAN prune on physical access switch is very important).
  • Virtual Port: Ports on Virtual Switch which are connected with virtual Ethernet adapters as well as physical Ethernet adapters. Thus, uplink ports are also virtual ports.
  • Port Groups: You can think of port groups as templates for creating virtual ports with particular sets of specifications. You can create a maximum of 512 port groups on a single host. Port groups make it possible to specify that a given virtual machine should have a particular type of connectivity on every host on which it might run.

NIC Teaming in VMware world

NIC Teaming in VMware world is connecting a single virtual switch to multiple physical Ethernet adapters with load balancing & failover, which is quite similar to EtherChannel or PortChannel technology. All adapters in the NIC team must be attached to the same physical switch or an appropriate set of stacked physical switches. That switch or set of stacked switches must be 802.3ad-compliant and configured to use that link-aggregation standard in static mode (that is, with no LACP). All adapters must be active. You should make the setting on the virtual switch and ensure
that it is inherited by all port groups within that virtual switch.

  • Teaming state — which physical Ethernet adapters are actually transporting data — is maintained for each port group. Teaming state transitions are mostly transparent to virtual Ethernet adapters. Virtual machines cannot tell when a failover has occurred or which physical adapter is carrying any given frame.

Implement Tips: it is recommended to attach all VMs, which belong to the same subnet, to a single virtual switch, so that communication among them can be forwarded within that virtual switch. If they are attached to different virtual switch, their interconnection relies on layer 3 routing in the real network, and traffic will firstly leave VMware host and come back again through different local physical network port which further forwards them to another virtual switch.

one virtual switch can have several Teamed NIC group as its egress for different type of traffic.

one physical Ethernet port can not be attached to 2 virtual switch at the same time.

Best Practice for NIC Teaming

  • Spanning Tree Protocol (STP) – disable the STP on switch ports that ESXi Host connects. Configuring PortFast Access or PortFast Trunk switch port will save about 30 secs when physical switch initiates.
  • EtherChannel negotiation like PAgP or LACP – must be disabled because they are not supported.
  • Disable Trunking Negotiation (DTP) further save about 4 secs.
  • Disable Failback policy (which is enabled by default) on NIC teaming to avoid Ethernet port switch between active and faulty frequently.

Steps to configure NIC Teaming in VMware ESXi hsot:

Procedures to configure NIC Teaming in VMware ESXi host can be summarized to below steps:

  • Create vSwitch (optional): VMs who belongs to the same VLAN are recommended to attach to the same vSwitch so that traffic among them originates and terminates within vSwitch and does not rely on outside networks. In this case, there is no need to create 2nd vSwitch.【Note that VMs belong to different VLANs can also be attached to the same vSwitch, however, those VMs should be associated with different Port Groups. Besides, Trunk mode should be used if there is only one pair of teamed NIC for uplink】one vSwitch can connects to several Port Groups and several pairs of teamed NIC at the same time, but one physical Ethernet port can be attached to only a single vSwitch at a time.
    Create vSwitch
  • Create Port Group and associate it with relevant vnics of VMs:
    Create Port Group
  • Physical NIC Teaming: Attach physical Ethernet Ports to vSwitch and specify Load Balance mechanism as well as NIC teaming policy. You can either team 8 physical Ethernet ports together or team them into 4 pairs of teamed NICs with different parameters (access or trunk, VLAN info etc) on each pair.
    VMware NIC Teaming

How VMware Virtual Switch supports VLAN:

Virtual Switch Tagging (VST mode): port on Virtual Switch can be configured to access only a single VLAN, the VMware equivalent of access port in physical network world. This is the most common configuration. In this mode, you provision one port group on a virtual switch for each VLAN, then attach the virtual machine’s virtual adapter to the port group instead of the virtual switch directly. The virtual switch port group tags all outbound frames and removes tags for all inbound frames. Use of this mode requires that the physical switch provide a trunk.

Virtual Guest Tagging (VGT mode): port on Virtual Switch configured to access multiple VLANs. You may install an 802.1Q VLAN trunking driver inside the virtual machine, and tags will be preserved between the virtual machine networking stack and external switch when frames are passed from or to virtual switches. Use of this mode requires that the physical switch provide a trunk.

External Switch Tagging (EST mode) — You may use external switches for VLAN tagging. This is similar to a physical network, and VLAN configuration is normally transparent to each individual physical server. There is no need to provide a trunk in these environments.

Promiscuous Mode: virtual switch supports copying packets to a mirror port. By using what is called promiscuous mode, ESX Server makes a virtual switch port act as a SPAN port or mirror port. This capability makes it possible to debug using a sniffer or to run monitoring applications such as IDS.

Basic Concepts to bear in mind:

A virtual machine can be configured with one or more virtual Ethernet adapters, each of which each has its own IP address and MAC address.

Network traffic cannot flow directly from one virtual switch to another virtual switch within the same host. The virtual switch does not allow traffic to pass from one VLAN to another, communication between VLANs is treated the same as communication between virtual switches — it is not allowed.

VMware Infrastructure 3 enforces a single-tier networking topology. In other words, there is no way to interconnect multiple virtual switches, thus the network cannot be configured to introduce loops. As a result, Spanning Tree Protocol  (STP) is not needed and is not present.

Reference URL:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004088

http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

http://docwiki.cisco.com/wiki/QoS_Design_Considerations_for_Virtual_UC_with_UCS