Myricom labeled Myri-10G as “a convergence at 10-Gigabit/s data rates of Myrinet technology with Ethernet.” An insightful pundit amusingly called Myri-10G “10-Gigabit Ethernet with special sauce.” Marketing phrases and sound bites aside, Myri-10G products immediately won customers, some for conventional 10-Gigabit Ethernet networking applications, some for HPC applications, and some for special applications such as IPTV video streaming. Why? Myri-10G Network Interface Cards (NICs) deliver uncompromising, wire-speed performance – a Myricom focus and tradition – at an affordable price, and are fully interoperable with the 10-Gigabit Ethernet products of other companies. Myri-10G switches scale economically to thousands of ports, a capability not offered by the traditional Ethernet-switch vendors.
Myri-10G with one SFP+ port
10G-PCIE-8B-S+E
Myri-10G network port: SFP+ (Small Form factor Pluggable) socket for an SFP+ transceiver, 10+10 Gbit/s data rate, full-duplex. The SFP+ socket provides SFI 10.3125 GBaud 64b/66b-encoded signaling, electrical power, and the standard SFP+ management interface. The socket does not require the use of SFP+ transceivers purchased from Myricom. The socket may also be used with SFP+-terminated "Direct Attach" twinax copper cables up to 5m, or with SFP+-terminated EOE cables. The port can operate with either Ethernet or Myrinet protocols at the Data Link layer. When operating in Ethernet mode, the port supports Ethernet flow control as defined by IEEE 802.3x. The allowed length of the fiber cable depends upon the SFP+ transceiver and the quality of the fiber, but in Myrinet mode must not exceed 200m due to Myrinet flow control.
This two-port NIC appears to the host operating system as a single PCI Express device. Firmware provides failover between the two network ports with a response time that can be set to as little as ~10µs. Concurrent wire-speed operation from two 10Gb ports requires higher data rates to/from the host than a "Gen 1" (2.5 GT/s) PCI Express x8 slot can provide. If your application requires two ports for performance and failover, see the 10G-PCIE2-8B2-2S NIC, which has a "Gen 2" (5 GT/s) PCI Express x8 host port (see below).
Specifications:
Myri-10G network ports (2): SFP+ (Small Form factor Pluggable) sockets for SFP+ transceivers, either 10GBase-SR or 10GBase-LR, 10+10 Gbit/s data rate, full-duplex. The SFP+ sockets provide SFI 10.3125 GBaud 64b/66b-encoded signaling, electrical power, and the standard SFP+ management interface.
Myri-10G with two SFP+ ports for failover
10G-PCIE-8B-2S+E
Myri-10G with two SFP+ ports for performance and failover
10G-PCIE2-8B2-2S+E
This NIC appears to the host operating system as two independent PCI Express devices, one for each network port. Each of the two devices (Lanai-Z8ES chips) can carry traffic at rates approaching 10Gb/s (~9.9 Gb/s with a 9KB MTU, or ~9.5 Gb/s with a 1500B MTU). Together, the two devices can carry traffic between the host and the network at rates approaching 20Gb/s.
If both devices are using the bundled Myri10GE Ethernet software distribution and if the hosts have sufficient memory bandwidth, the netperf TCP_STREAM test with two streams shows an aggregate bandwidth of ~19.8 Gb/s with a 9KB MTU, and ~18.9 Gb/s with a 1500B MTU. These netperf 2.4.3 tests were between pre-release Intel Nehalem hosts running the RHEL5 kernel 2.6.18-92.el5. Performance will vary with the host and operating system.
Specifications:
Myri-10G network ports (2): SFP+ (Small Form factor Pluggable) sockets for SFP+ transceivers, either 10GBase-SR or 10GBase-LR, 10+10 Gbit/s data rate, full-duplex. The SFP+ sockets provide SFI 10.3125 GBaud 64b/66b-encoded signaling, electrical power, and the standard SFP+ management interface.
Myri-10G network port: 10GBase-CX4, 10+10 Gbit/s data rate, full-duplex. The port is fully compliant with the IEEE 802.3ak specifications for the 10GBase-CX4 10-Gigabit Ethernet Physical layer (PHY), and can operate with either Ethernet or Myrinet protocols at the Data Link layer. When operating in Ethernet mode, the port supports Ethernet flow control as defined by IEEE 802.3x. The port may be connected with standard 10GBase-CX4 cables up to 15m in length.
Myri-10G with one CX4 port
10G-PCIE-8B-C+E
Myri-10G with one QSFP port
10G-PCIE-8B-QP+E
For even higher performance:
See the 10G-PCIE2-8B2-2QP NIC (below), which has a "Gen 2" (5 GT/s) PCI Express x8 host port and two QP ports for up to nearly 20 Gb/s aggregate unidirectional throughput, or nearly 40 Gb/s aggregate bidirectional performance.
Although the Myri-10G "QP" Physical layer (PHY) can be used for 10-Gigabit Ethernet applications, QP ports are generally used for HPC applications with MX software and Myrinet network protocols. A QP port uses an industry-standard QSFP (Quad Small Form-factor Pluggable) socket with XAUI signaling per IEEE 802.3ae, electrical power, and the standard QSFP management interface. The advantages of QP ports are low cost, low latency, low power, and the flexibility of being able to use either low-cost, passive, copper cables (10G-QC-xM) up to 5m or Electrical-Optical-Electrical (EOE) fiber cables (10G-QP-xM) up to 200m.
Specifications:
Myri-10G network port: QSFP (Quad Small Form factor Pluggable) socket for QSFP-terminated passive copper cables, QSFP-terminated EOE cables, or QSFP transceivers, 10+10 Gbit/s data rate, full-duplex. The QSFP socket provides XAUI signaling per IEEE 802.3ae, electrical power, and the standard QSFP management interface. The port can operate with either Ethernet or Myrinet protocols at the Data Link layer. When operating in Ethernet mode, the port supports Ethernet flow control as defined by IEEE 802.3x. When operating in Myrinet mode, the fiber link is limited to 200m by Myrinet flow control.
This NIC appears to the host operating system as two independent PCI Express devices, one for each network port. Each of the two devices (Lanai-Z8ES chips) can carry traffic at rates approaching 10Gb/s (1.25 GBytes/s). In hosts with sufficient memory and I/O bandwidth, the two devices can together carry traffic between the host and the network at rates approaching 20Gb/s (2.5 GBytes/s) unidirectional and 40Gb/s (5 GBytes/s) bidirectional. If both devices are using the optional MX-10G message-passing system, traffic can be combined using MX NIC bonding, resulting in a unidirectional (MPI PingPong) data rate of ~2.4 GByte/s and a bidirectional (MPI SendRecv) data rate of ~4.7 GByte/s
Myri-10G with two QSFP ports
10G-PCIE2-8B2-2QP+E
SFP+ Transceivers
10G-SFP
These industry-standard SFP+ (Small Form factor Pluggable 10.3125 GBaud) optical-fiber transceivers are supplied to Myricom from multiple sources. Myricom has qualified the transceivers from each source for performance and reliability. Some of the SFP+ transceivers supplied by Myricom have Myricom labeling, as shown, and some have the manufacturers' standard labeling.
The 10GBase-SR transceivers operate at an 850nm optical wavelength, and are designed for 50µm (50/125) multimode fiber at distances of 26-82m depending upon the fiber, or at distances up to 300m over 50µm 2000MHz·km multimode fiber.
The 10GBase-LR transceivers operate at a 1310nm optical wavelength, and are designed for 9µm (9/125) single-mode fiber up to 10km, but are commonly used up to larger distances with high-quality fiber.