The Z420 that I used has an Intel E5-1620 processor and 8GB of RAM. I purchased it on eBay for around $300. I just checked and a Z420 with an E5-2680 and 32GB of RAM is going for around $375. The extra RAM and faster processor would be nice.
I added a 10Gb PCIe card from eBay - 666172-001 10GB MELLANOX PCIe 10GBe ETHERNET NIC for $16.00. That allowed me to connect the Z420 to the customer's switch and run iPerf3. That worked great but I wanted a way to verify that the Z420 could actually run at a full 10Gb and be able to show the customer before I test.
So, I decided to buy another 10Gb adapter and use a VM to test from the Z420's Ubuntu 18.04 host to the guest. I purchased the exact same adapter because the Mellanox driver was already installed and I knew it worked well.
Using KVM and Virt-manager
I decided to use KVM as the hypervisor and Virt-manager as the manager instead of VMware Workstation for this application. There are a few reasons I decided to use KVM:- VMware workstation is a proprietary package that costs $199. Plus you have to keep maintenance on it or buy it again when the next version comes out. It's a great tool and I use it on my laptop, but I didn't want to spend the money on my server.
- KVM is built into Linux and Virt-manager is a free, open-source tool. That means that KVM is automatically upgraded when the kernel is updated and Virt-manager will be updated by the Ubuntu package manager.
- I'm studying Software Defined Networking and Linux is a big part of that. For example, NetAPP has a KVM based version of their SAN controller and customers will need help installing and optimizing it. There are also a lot of Linux servers running KVM in data centers and as a network engineer I want to know how to configure them for network access.
I am going to write another blog on how to get KVM up and running. It was a great experience and I learned quite a bit in the process since I had to install Virt-manager, create the bridge, modify firewall rules, and troubleshoot a communications issues that ultimately was caused by my Docker install. How much more fun could you have on a Saturday afternoon?
The results
I had a 10Gb 16 port switch in my lab and a couple of short DAC cables so connecting the two 10Gb adapters to my network was easy enough. Once I had KVM installed and the bridges created, I grabbed my Perfsonar toolkit ISO and built a CentOS 7 Perfsonar VM. The process was almost identical to VMware Workstation.Virt-manager makes it easy to clone VMs so before I started configuring the VM I cloned it using "virt-clone --original Perfsonar4-1 --name Perfsonar4-2 --auto-clone" from the terminal. The tool takes care of changing MAC addresses but you will need to change the hostname, ssh keys, etc.
Here is what Virt-manager looked like with the two VMs created:
Networking in Virt-manager is similar to VMware Workstation. Here is a screenshot of the bridges:
In the VM details, you select the Bridge to use. Virt-manager lists the Bridge name, BR0 in this case, and the physical interface on the host, ens3.
One thing I learned is that you need to use the virtio device model. Initially, I selected E1000 based on my experience with VMware ESXi and it took me a few minutes to figure out why I was getting 941Gbps when testing!
I was worried that the E5-1620 wouldn't have enough power to run the Z420 and the VM at 10Gb but it worked no problem. CPU utilization on the VM ran around 65% most of the time and maxed out at 77%.
I only gave the VM 2 vCPUs, if it had maxed out, I would have been able to add another one. Here is a screenshot of HTOP that I grabbed on the VM during the test:
Here is the output from iPerf3 on the Z420. Notice there were no retries (Retr) after the first second and the Congestion Window (Cwnd) was very consistent.
mhubbard@Z420:~$ iperf3 -c 192.168.10.187 -P4 -O2 Connecting to host 192.168.10.187, port 5201 [ 5] local 192.168.10.185 port 47044 connected to 192.168.10.187 port 5201 [ 7] local 192.168.10.185 port 47046 connected to 192.168.10.187 port 5201 [ 9] local 192.168.10.185 port 47048 connected to 192.168.10.187 port 5201 [ 11] local 192.168.10.185 port 47050 connected to 192.168.10.187 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 310 MBytes 2.60 Gbits/sec 35 404 KBytes (omitted) [ 7] 0.00-1.00 sec 267 MBytes 2.24 Gbits/sec 78 288 KBytes (omitted) [ 9] 0.00-1.00 sec 236 MBytes 1.98 Gbits/sec 85 372 KBytes (omitted) [ 11] 0.00-1.00 sec 239 MBytes 2.00 Gbits/sec 111 349 KBytes (omitted) [SUM] 0.00-1.00 sec 1.03 GBytes 8.82 Gbits/sec 309 (omitted) - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 1.00-2.00 sec 281 MBytes 2.36 Gbits/sec 0 417 KBytes (omitted) [ 7] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 407 KBytes (omitted) [ 9] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec 0 440 KBytes (omitted) [ 11] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec 0 450 KBytes (omitted) [SUM] 1.00-2.00 sec 1.09 GBytes 9.39 Gbits/sec 0 (omitted) - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 0.00-1.00 sec 279 MBytes 2.34 Gbits/sec 0 428 KBytes [ 7] 0.00-1.00 sec 280 MBytes 2.35 Gbits/sec 0 407 KBytes [ 9] 0.00-1.00 sec 279 MBytes 2.34 Gbits/sec 0 491 KBytes [ 11] 0.00-1.00 sec 280 MBytes 2.35 Gbits/sec 0 452 KBytes [SUM] 0.00-1.00 sec 1.09 GBytes 9.38 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 428 KBytes [ 7] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 417 KBytes [ 9] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 509 KBytes [ 11] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec 0 475 KBytes [SUM] 1.00-2.00 sec 1.09 GBytes 9.39 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 437 KBytes [ 7] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 420 KBytes [ 9] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec 0 533 KBytes [ 11] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec 0 498 KBytes [SUM] 2.00-3.00 sec 1.09 GBytes 9.37 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 3.00-4.00 sec 280 MBytes 2.35 Gbits/sec 0 465 KBytes [ 7] 3.00-4.00 sec 280 MBytes 2.35 Gbits/sec 0 427 KBytes [ 9] 3.00-4.00 sec 280 MBytes 2.35 Gbits/sec 0 533 KBytes [ 11] 3.00-4.00 sec 280 MBytes 2.35 Gbits/sec 0 498 KBytes [SUM] 3.00-4.00 sec 1.09 GBytes 9.39 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 4.00-5.00 sec 280 MBytes 2.35 Gbits/sec 0 530 KBytes [ 7] 4.00-5.00 sec 280 MBytes 2.35 Gbits/sec 0 430 KBytes [ 9] 4.00-5.00 sec 281 MBytes 2.35 Gbits/sec 0 533 KBytes [ 11] 4.00-5.00 sec 280 MBytes 2.35 Gbits/sec 0 498 KBytes [SUM] 4.00-5.00 sec 1.09 GBytes 9.40 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 619 KBytes [ 7] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec 0 438 KBytes [ 9] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 533 KBytes [ 11] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec 0 498 KBytes [SUM] 5.00-6.00 sec 1.09 GBytes 9.38 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 0 735 KBytes [ 7] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 0 445 KBytes [ 9] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec 0 533 KBytes [ 11] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec 0 501 KBytes [SUM] 6.00-7.00 sec 1.09 GBytes 9.38 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 871 KBytes [ 7] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 448 KBytes [ 9] 7.00-8.00 sec 281 MBytes 2.35 Gbits/sec 0 539 KBytes [ 11] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec 0 513 KBytes [SUM] 7.00-8.00 sec 1.09 GBytes 9.39 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 1.05 MBytes [ 7] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 448 KBytes [ 9] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 550 KBytes [ 11] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 523 KBytes [SUM] 8.00-9.00 sec 1.09 GBytes 9.39 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 9.00-10.00 sec 281 MBytes 2.36 Gbits/sec 0 1.25 MBytes [ 7] 9.00-10.00 sec 281 MBytes 2.35 Gbits/sec 0 479 KBytes [ 9] 9.00-10.00 sec 280 MBytes 2.35 Gbits/sec 0 576 KBytes [ 11] 9.00-10.00 sec 280 MBytes 2.35 Gbits/sec 0 546 KBytes [SUM] 9.00-10.00 sec 1.10 GBytes 9.41 Gbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender [ 5] 0.00-10.03 sec 2.74 GBytes 2.35 Gbits/sec receiver [ 7] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender [ 7] 0.00-10.03 sec 2.74 GBytes 2.35 Gbits/sec receiver [ 9] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender [ 9] 0.00-10.03 sec 2.74 GBytes 2.35 Gbits/sec receiver [ 11] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender [ 11] 0.00-10.03 sec 2.74 GBytes 2.35 Gbits/sec receiver [SUM] 0.00-10.00 sec 10.9 GBytes 9.39 Gbits/sec 0 sender [SUM] 0.00-10.03 sec 11.0 GBytes 9.39 Gbits/sec receiver
References
Convert code to HTML for BloggerHow to clone existing KVM virtual machine images on Linux
Install And Set Up KVM On Ubuntu 18.04 Bionic Beaver Linux
Create and Run Virtual Machines With virt-manager
Predictable Network Interface Names
libvirt Networking Handbook
KVM/Networking
Ubuntu 16.04 kvm bridges not working
bridge networking not working on ubuntu 16.04
Configure network interface as DHCP client on RHEL7 Linux
Open firewall port on CentOS 7
KVM - Create a virtual machine with 2 bridges interfaces
Testing 10Gb links with iPerf3 ensures optimal network performance for high-speed data transfers. Just like Skynode enhances my gaming experience, accurate link testing with iPerf3 is essential for reliable network efficiency.
ReplyDelete