Tip:
Highlight text to annotate it
X
Welcome back everyone! This is part 2 in our 10Gb Home Network Series, and in this video
we are going to be focusing on peer-to-peer networking. If you have not seen the first
video in this series, make sure you stop here and check out the introduction video since
it covers some important concepts with 10Gb networking. Alright! Now that you have seen
the first video, bragged to all your friends, and considered the possibilities for your
excessively fast home network, it's time to have some fun.
The main point of this video is to have some fun and keep things simple. That means a minimal
shopping list for our trip to the virtual store. Check the video description for quick
links. All you need is two computers, two 10Gb network cards, and one cable. That's
it! No ridiculously expensive 10Gb switch or magic required to participate. Man, this
kind of fun without an expensive switch should be illegal. Okay, time for specifics with
our shopping list. And to that point, compatibility is a very important consideration if you are
going to do this the right way. The two computers you select need to each have an open PCIe
x8 (Gen1) slot. You need a Gen2 PCIe slot if the 10Gb card has dual ports. If you are
uncertain, be sure to check the requirements of the card you purchase. Next on the list
is two 10Gb SFP+ Network Interface Cards (NIC). Some popular brands include Intel, Mellanox,
and Chelsio. Each vendor has different feature sets on the various 10Gb NICs they offer.
Make sure you purchase the same two cards for maximum compatibility. And don't forget
to consider what operating system you are using with the 10Gb NIC. Support for 10Gb
is not as broad as the 1Gb market, so getting the right driver for your operating system
will be vital to maximize your performance. For this video I'm going to be demonstrating
the Mellanox ConnectX-2 Ethernet Adapter. At the time of this recording I purchased
two cards for $18 each on EBay. After-market prices are sure to fluctuate, especially if
everyone rushes to E-Bay for these cards. Brand new, 10Gb cards are very expensive,
because vendors are targeting the enterprise market. Lastly you will need, at least, one
Direct Attach Copper (DAC) cable, so you can directly connect PC number 1 to PC number
2. These cables come in three different lengths: 1m, 3m, and 5m. That's 3ft, 10ft, and 16ft.
This is called a peer-to-peer connection. This is a great way to keep things simple
and cheap. Make sure you check the video description, section titled Shopping List, for helpful
links to the items I just mentioned.
Now that you have your geek gear list, it's time to pull the trigger on some hardcore
upgrades for your network. A few places I frequent are Amazon and EBay. If you don't
want to pay full price, look to EBay. Before making any purchases, consider the following
suggestions. 1) Make sure the card you purchase has a full height bracket. A lot of the cards
listed came from servers with half-height brackets. 2) Get a single port card for reduced
price. Consider a dual port card if you plan on making multiple connections to the same
system. Example: PC #1 connects to your storage server and PC #2 connects to the same storage
server. This scenario would require three cards: 2 single port cards, and one dual port
card. 3) Check your operating system Hardware Compatibility List (HCL) to verify the card
you are about to purchase works with your operating system. I will list combinations
I have tested in the video description for your convenience. 4) Get matching cards to
maximize your compatibility. 5) Measure the distance between systems you plan to connect,
so you know how long the DAC cable needs to be. The longer cables tend to have more bulk
and less maneuverability.
Use the links in the video description to make your purchase and unbox when the gear
arrives. Ahh yes, that will be a glorious day indeed!
Install day has arrived! Crack your workstations open. Install the 10Gb Ethernet adapter into
an open PCIe x8 slot securely, and button things back up. Repeat this procedure for
workstation #2. Now connect up the two systems with the Direct Attach Copper (DAC) cable
as shown. If you insert the cable module facing the proper direction, you should feel the
module click into place. If you need to remove the DAC module, you will need to push the
cable towards the card while you pull the release strap. This should allow the module
to be removed. If the module does not freely slide out, do not force it out by pulling
hard. Try again as the video shows. Now that both adapters are installed and the copper
cable is connecting them together, you officially have a 10Gb network. At least physically speaking.
Now, celebrate early with your best Truffle Shuffle dance moves. Next we are going to
verify that your Ethernet adapter is not being throttled by the PCIe interface.
10Gb Ethernet adapters are going to tax your system one way or another, so It's important
to rule out your PCIe bus speed as being a limiting factor. On Windows run "msinfo32"
(System Information), and search for the name of your Ethernet adapter. In my case I would
search for "Mellanox". You can also try searching for "10G". Once you find your Ethernet adapter,
look at how it registers. Mine displays as "Mellanox ConnectX - PCIe 2.0 5GT/s". Or,
using PowerShell run Get-NetAdapterHardwareInfo and check the PCIeLinkSpeed and width column.
Long story short an x8 PCIe 2.0 slot will give you 32Gbs. That's plenty to handle a
single (10Gbps) or dual (20Gbps) port card. On the other hand an x8 PCIe 1.1 slot will
only give you 16Gbps, so that is fine for a single port. But if you use a dual port
card here, expect reduced bandwidth. Many will attempt to use older systems, so I just
want people to understand what the limiting factors are. If you are on a Linux system
execute "dmesg | grep -i 10g". The log may indicate you are running at reduced capacity
if that is the case.
Now that your hardware is installed, it's time to load a driver and update the firmware
on the Ethernet adapter. Start by downloading the appropriate driver for your card. You
may have to check the "legacy" section on the drivers page as you will be working with
older model Ethernet adapters. Once you have the driver downloaded, start the install and
walk through the installation screens. You will need to restart your computer at this
point for the drivers to register with your operating system. Now that the drivers are
installed, we need to verify the latest firmware is loaded on the Ethernet adapter. Often this
step is overlooked, but important to avoid compatibility and performance problems. Download
the firmware update tool. In my case the Mellanox utility is called "mlxup". I copied it to
the Windows slash System32 directory so it would be in the system path. Run the update
utility to query your card and see what version firmware is already installed. Next you download
the matching firmware if you are not running the latest version. The process will be different
for Intel and Chelsio cards. Once you have the firmware downloaded and extracted, you
can run the update utility to flash the Ethernet adapter with the latest version. In my case
it was already up to date. Repeat this procedure for the other computer.
It's finally time to configure our new adapters. The idea is configure your 10Gb Ethernet adapters
on a totally different network. This will avoid network path issues and simplify the
network configuration. Use the displayed configuration slide to set your 10Gb adapter settings. Continue,
when you are finished, with the network and advanced property settings. Next open File
Explorer and browse to c:\windows\system32\drivers\etc and edit the "hosts" file. You will need administrative
privileges to edit the file. You will add two entries making it easier to reference
systems on your high speed network segment. The hosts file is a local version of a DNS
entry.
In this segment we will be setting up tools to aid in the testing and use of your new
10Gb network. The first tool is called iPerf. It is a command line utility, but there is
a visual front end you can add called jPerf. This simplifies the tool and makes it a little
easier to use. The second tool is called SoftPerfect RAM Disk. This allows you to create a RAM
disk on each system and transfer using high speed RAM. This is a good method for ruling
out your hard disk drive as a performance bottle-neck. Start by installing jPerf, which
should include the iPerf binary (linked in the video description). Run a jPerf server
instance on PC #2 and start jPerf client on PC #1. Configure client settings as displayed.
Or you can just use the command line if you prefer. Match the settings I use, and execute
a 30 second job to get a sense of how your system is performing. Bring up Task Manager
and monitor your compute and network utilization to verify nothing odd is happening and things
are functioning as you expect. Everyone's speeds will vary for the following reasons:
processor speed, processor degree of parallelism, RAM amount, bus speeds, hard drive speed,
protocol, and of course network bandwidth. Your limiting factor could be on the send
or receive side of the equation too.
Where iPerf is great for testing network throughput, SoftPerfect RAM Disk is great for file transfers
without being inhibited by potentially slow disk drives. Download and install the RAM
Disk software on both workstations. Configure RAM Disk as shown in the video and adjust
to your available RAM. Next, find a large file that will fit in the newly created RAM
Drive, share your RAM Drive on workstation #2, and connect to it using the hosts file
alias we created earlier. Using this method we are connecting workstation #1 to workstation
#2 using the 10Gb network. If you just use the normal host name, Windows will likely
connect to the remote system using your standard 1Gb connection, so be aware.
Now that you've seen what can be done with 10Gb Ethernet speeds, I'm sure you are wondering
how else you can take advantage of this tech. I know I was! So I started investigating my
options for expanding 10Gb Ethernet to the remaining systems in my Home Area Network
(HAN). I quickly realized 10Gb switches were outrageously priced. So I did what any geek
confronted with a technical challenge would do. I built my own switch! Check out my third
video, in the 10Gb Home Network Series, where I teach you how to build your own switch and
bypass the ridiculous pricing for 10Gb commodity switches. Don't forget to subscribe, like
and comment to join the discussion.