Tuesday, November 14, 2006

…but just another bird in the flock

The outlook for Infiniband is more mixed beyond the rarified world of supercomputers that demand top performance. On Monday, Mellanox Technologies, the last remaining IB merchant chip supplier, rolled out a hybrid architecture supporting IB and Ethernet. ConnectX even has limited support in software for Fibre Channel. For the full story click here.

The message is that the mainstream data center will use multiple networks. Convergence will come in silicon embracing those nets.

Myricom has already started embracing Ethernet. Following a similar trend, Fibre Channel leader QLogic acquired Infiniband switch maker SilverStorm Technologies in October. It had acquired earlier this year Infiniband card maker PathScale.

In this mixed world, Infiniband has a future as the highest performance interconnect at the high end and one of the many nets with a role to play in the mainstream.

A couple interesting data points:

--The Infiniband chip business became profitable for Mellanox last year for the first time since the company was founded in 1999 with profits of $3 million on sales of $42 million in 2005, according to S1 documents filed with the SEC in September.

--To keep its edge, Mellanox will demonstrate 40 Gbit/s Infiniband links with one microsecond latency over the ConnectX chips early next year. It is developing 10Gb/s serdes blocks designed to enable up Infiniband links at up to 120Gb/s in the more distant future. --rbm


Anonymous said...

Mellanox will never reach 1 microsecond latency with Infiniband. Infiniband has never gone under 3.5 us. And it will not change much in the future because the RDMA/Infiniband model is not designed well for such a purpose.

The only way to reach very low latency would be to do as Pathscale/Qlogic: use a IB-compatible hardware for marketing purpose, but run a completely different (and IB-incompatible) protocol/implementation (Infinipath) for performance purpose. In the end, Infinipath is actually doing what Quadrics and Myricom have doing for a while: proprietary and specialized hardware and protocol designed for low latency. Infiniband has not been designed this way, it is not going to change if Mellanox keeps doing infiniband-like interconnects.

Anonymous said...

The above comment sounds very much like a competitor speaking rather than an objective assessment.

The fact is all of these numbers are contrived no matter the interconnect. The zero-byte ping-pong test is pure nonsense from a practical world especially in any configuration that scales out to a decent fabric diameter. The only real accurate measure of whether an interconnect is provding better value over another is whether the applications perform better or not. So much of the micro-benchmarks at best approximate only a subset of the application's operation. All too often when the application is actually executed these numbers don't add up. So, give me a real workload to evaluate benefits. I may not agree with all of the assertions made by any camp but I tend to agree more often with a stepwise, refined analysis on workloads. It is there where the benefits of any interconnect can shine or be shown to be duds.

As for the sub-microsecond nonsense, well, the IB proponents state they will have such a number in 2007. It will likely be based using the same techniques as used in any of its competitors. Will it influence anyone especially customers? Unlikely as more and more customers are moving to require run workload X with your interconnect and prove it is better. They are saying they've had enough of the marketing spin and want real performance demonstrated or take a hike. It is not clear whether the adoption rates or decline in the top500 are an indication that the customers see proof or they are buying hype but with the costs continuing to climb for a solution (which goes so much further than just an interconnect), one hopes they are doing credible analysis else they are not getting their money's worth and that is shameful especially when so many are state or federal government funded.