10G ETHERNET LAYER 1 OVERVIEW
Think your`re an networking ethernet guru ? tcpdump kung fu master? eat jumbo packets for breakfast? Then welcome to the "Layer 1" dungeon, forget everything you know. We are talking about transceiver, clock recovery, DC balance, encoding symbol sets, scrambling, lane skew and more. For 10G, 40G and 100G networks you will be surprised that, what data you think is going down the wire, is actually a completely different set of 1`s and 0`s in the cable.
Lets start with the basic building blocks of a 10G ethernet system. The diagram above maps out the major components in english (not 802.3 clause 49 legalize).
MAC Media Access Control. Usually this is considered the lowest of low level components, however this is today's starting point. Its connected to the PHY using XAUI or RXAUI protocol which is an 8b/10b encoded bit-stream and includes everything required to extract packets. Note the "extract packets" part as if your from a software background the idea of always sending data on every single cycle, regardless of idle or not, seems quite un-natural. We will get to that a bit later, for now consider the MAC`s interface a full line rate no gap stream of bits.
PHY. Traditionally the PHY was a completely separate chip, e.g the classic NetLogic AEL2005 which handled all the nasty encoding/decoding serialization/deserialization at 10Ghz. This separation required a standardized protocol between the MAC & PHY thus the XAUI protocol was created. The PHY`s main job is to translate the XAUI data stream into a serial link which is connected to the SFP+ transceiver's. Now days the PHY is fully integrated and part of the ASIC or FPGA making the separation more logical than a real physical connector.
PCS Physical Coding Sublayer. For 10GBaseR the PCS is fairly simple compared to 40G, 100G and some of the more exotic backplane protocols. Why simpler? the PCS for 40G/100GBaseR must handle alignment, distribution, lane skew, error correction and synchronization, all of which are not part of the 10GBaseR spec. Straight 10GBaseR PCS only requires 64b66b encode/decode and scramble/descramble logic. One of the great design decisions about 40G and 100G is, it utilizes the same basic 10G foundations, with fairly minor variations for the different protocols. This substantially reduces the technology risk and resources required to support 40G/100G as most of the ground work has already been done. In short, the primary purpose of the PCS is to translate the XAUI bitstream into a 64b/66b encoding and scramble the bits that are then sent to the PMA unit for serialization.
PMA Physical Medium Attachment This unit converts the scrambled 64b/66b bitstream into a high speed 10.3125Ghz serial link or decodes the 10.3125Ghz serial link into an 64b/66b bitstream. On the receiver side it also performs clock recovery which we will go into detail in a different post. Typically most of the work is done by the hardmacro transceiver's on the chip, as FPGA`s and ASIC`s can not run a 10Ghz clock across the chip, but it can run very high clocks in very small areas of the chip, which is exactly what the hardware designers did.
PMD/MDI Medium Dependant Interface At this point, its about what kind of SFP+ transceiver's is plugged in. This is where the rubber meets the road, as the 10Ghz serial stream of 1's and 0's is converted into electrical impulses in the case of Copper cabling, or converted into photons when fiber is used.
For 10Gbit network capture devices, the MAC + PCS + PMA are combined into the same FPGA chip. All major fpga vendors support high speed on-chip transceiver`s, since the Xilinx Virtex6 and Altrea StratixV. Combinding the functionality enables reduction in latency, reduction in costs and an increase in flexibility. Also our own 10G, 40G and 100G capture systems implementing the entire MAC+PCS+PMA in a single fpga, without any PHY chip. We will dig into each of the above components in signficiant detail over the next few posts, stay tuned.
More posts on Layer 1 Ethernet
