Tuesday, July 24, 2007

Multicore versus TOE

Intel's LAN division rolled out its latest 10G Ethernet controller yesterday, firing a round for CPU vendors who want to keep the job of handling high-end Ethernet tasks on their multicore processors. The Intel 82598 is its first to use its I/O Acceleration Technology (I/OAT) at 10 Gbits/s.

Even Intel admits there is a performance benefit on some high-end Ethernet controllers for the alternative, providing TCP offload engines (TOEs), and it won't rule out the possibility it may offering such a product some day. HP is so far opposed to the I/OAT approach, opting for TOEs. Intel says they are working on them and it claims design wins in all the other top sever vendors, though Intel admits server makers use a mix of I/OAT and TOE.

There are many more shoes to drop on this debate as 10G Ethernet ramps up over the next two years. At least one more new startup is about to come out of hiding on this issue. Stay tuned for more on Monday.

Meanwhile, it is interesting to note, Intel is already gearing up for the so-called Convergence Enhanced Ethernet by providing a pre-standard version of per-priority pause on its 10G controller. CEE aims to lay a foundation for running Fibre Channel protocols on Ethernet in the 2010 timeframe—or maybe sooner if this new Intel part is any indication of commercial trends.


Anonymous said...

Whether a TOE or a RNIC, the facts speak for themselves - they provide a performance boost for a variety of applications at a lower performance / watt than using a dedicated core or thread. The larger question is whether this benefit is fleeting or not as the number of cores increases to where they are essentially "free" and customers are willing to spend cores rather than I/O devices to meet their performance or performance / watt requirements.

Intel believes many core will win out in the end. Sun is willing to bet at least one processor line on the same outcome. For interconnects built around 10 Gbps technology, they may have the processing power to ship something credible in 2010. For interconnects built around 40 and 100 Gbps technology, it remains to be seen whether they can credibly throw enough cores and associated memory and I/O bandwidth have anything credible. A key observation is that I/O technologies progress faster than processor which is faster than memory. As a result, the ability to construct a credible, balanced solution remains elusive or at best, of limited success over time. There will always be a new challenge or customer requirement that makes differentiation in I/O technologies both compelling and required. The advent hybrid computing efforts by many vendors is proof positive that in a very dynamic and changing landscape, status quo thinking and execution will not win the day.

It may be that in 2015 there will be so many cores or a combination of computational elements will make the need for distinct I/O devices moot. That appears to be what Intel is banking on occurring. However, there is a significant amount of time between now and then and until Intel demonstrates an ability to integrate 10 GbE into its processors (some assert Sun's Niagara while an interesting experiment isn't ready for high volume deployment), the jury will remain out on whether they can really eliminate the need for discrete TOE and RNIC I/O devices. At least there are a good number of I/O IHV who are banking that processor-level integration remains a significant challenge and risk else they are all out of jobs.

steve said...

Intel's announcement validates the assertions which have been made by Solarflare and other stateless offload based Ethernet controller vendors, namely that TCP offload is at best a point solution and is already unnecessary at the 10Gb/s speed node when using the commodity server chipsets which have been available since 2006.

Of more importance than TCP offload (which has been plagued by poor performance, compatibility and reliability), are the recent platform improvements enabled by the chipset and operating system vendors. Intel has opened up its asynchronous DMA engine, renaming it from I/OAT to QuickData and its benefits are now available through both Microsoft and Linux to stateless offload controllers from all hardware vendors. Microsoft's Scalable Networking Pack supports Receive Side Scaling which significantly improves cache locality and hence performance for multi-core architectures.

The previous comment noted that server IO performance will likely be again challenged by 40 and 100 Gb/s networking. In the HPC space, server performance is continually challenged by IO performance whereas in the Ethernet space this occurs in a step-wise manner. It is no coincidence that the HPC space has settled on OS-bypass as the means to achieve sustainable high-performance using high-end interconnects such as Infiniband. If history repeats itself again as it has for the 100Mb/s, 1Gb/s, 10Gb/s Ethernet cycles, then by the time commodity servers start to make significant use of Ethernet at the next speed grade, they will be more than capable of handling the bandwidth.

It is also worth noting that TCP offload is not a new innovation to be compared against just one platform innovation such as I/OAT, rather it should be viewed a competing platform architecture which has appeared cyclically over the last 20 years.

Anonymous said...

What is amusing is that if protocol off-load is so irrelevant due to processor / chipset / software improvements, then why is it ok for InfiniBand, Fibre Channel, SAS, and so forth? People get religious about TOE but the fundamental question is applicable to all I/O interconnects. In the end, will there be nothing but a configurable physical layer with all I/O protocols being done in the processors? It is a bit hard to argue protocol off-load is bad on one hand and good on another just because there remains this religious aura around what one is allowed to do just because the protocol in question is TCP.