Monday, April 02, 2007

Data center nets circa 2009

I have sunk my teeth into this CEE concept for enhanced Ethernet and I am not letting go. This week I want to dig into what the data center networks will look like in 2009. That means I need to get more clarity on CEE, Infiniband, Fibre Channel, iSCSI and any other contenders to paint a picture of where we are headed.

Everybody wants to be the converged network for networking, clustering and storage. What's the reality? If you have any insights you want to share, post them here or drop me a line at

The concept may be all the more interesting because CEE hits at the same time as integrated AMD Fusion and Intel Nehalem processors that might include key underpinnings for I/O.

1 comment:

yaron said...


It seems like people are short sighted when it comes to new technology introduction
Technologies like FC, iSCSI, iWarp, and InfiniBand all started with the same buzz like CEE/FCoE
Then comes the realization phase (in the famous Gartner curves), and only few years later decent products hit the market and adoption ramps up.

CEE & FCoE are much more then changing a packet structure, they break some of the fundamental assumptions in Eth, switches are going to be cut-through and lossless, the good old STP protocol would be replaced by its multi-path equivalent, FC and Eth management would need to be layered one on top of the other, etc’. today CEE & FCoE are in the hands of architects and marketing guys (a.k.a paper tiger), they would need to travel a long path in standard committees, then developers would see its not that trivial and would need to nail all the bugs, then we will see a first generation of products, we all know how those would function and integrate or interoperate, a bunch of customers will do POCs, and then one day its going to mature.

This cycle is not new, every technology went through that path, it never took less then 3-5 years (check the history), even if we give CEE some discount it will not mature before 2010.

As a reminder all those new capabilities that are planned for CEE already work in InfiniBand, and in production sites with thousands of nodes, by the time CEE matures InfiniBand will be deployed with its 40 Gb/s flavor.

Many customers today are constraint by the 1GbE speed, and can’t afford buying a Cisco or a Force10 10GbE director at $8,000 a port, an InfiniBand director port costs under $500 and consume 90% less power, so it makes a much more viable server fabric option for them, not to mention InfiniBand already does all those nice things CEE will do in few years, and runs storage 4X faster then FC, that’s why InfiniBand adoption is growing rapidly, and why its not going to die so soon. Even Cisco understands that which is why they invest in InfiniBand.

Another point to mention is that any of those technologies that provide physical layer consolidation, do not address the problem, FCoXX while running over Ethernet (or IB), preserve the FC management model as a separate entity, the key technologies that are needed are ones that will lead to consolidation at the higher layers.

So in my opinion IB and Ethernet would probably live side by side for a while, IB will gain more critical mass mainly due to the slow execution and complexity on the 10GbE side, in the future (2010+) when CEE will hit the market they would split the market where IB will be the high-end offering and CEE the lower-end one, in the very long run I’m not sure how much it all matters since the real problems are not in the physical layers but in the application and management.

Yaron, Voltaire (CTO)