Thursday, November 16, 2006

White paper: HT beats PCIe in latency

The HyperTransport Consortium recently posted a white paper documenting the latency advantage of HT over PCI Express for communications systems. The paper "presents a couple of usage scenarios involving read accesses and derives the latency performance of each," according to author Brian Holden who chairs the technical working group of the HT Consortium.

The paper measures latency for short packet reads on HT at 147-168 nanoseconds compared to 240-273 for PCIe. For long packet reads HT weighs in at 576-630ns compared to 885-1,008ns for PCIe.

The paper may be timed to exploit the weakness of PCIe in comms now that the related Advanced Switching Interconnect has gone belly up (see my posting of Aug. 23-24). ASI was supposed to bring comms-friendly features to PCIe, such as support for multiple host CPUs.

Despite HT's technical edge in latency, my sense is big comms OEMs like Cisco are moving away from the parallel HT and toward PCIe in their designs because it is more broadly supported. (Intel's PC volumes far outstrip those of HT backer AMD). And PCIe performance may not be the very best, but it is good enough.

I'd love to hear any other opinions from the comms world, so post away! –rbm


interconnect Guy said...

Cut thru switch latency with QoS is required in Telecom systems.
RapidIO Switches are providing less the 100ns latency with full 10G data rates. Like carrrier class switchs of old. This type of dependable, consitant latency is required and its the latency and the QoS that sets PCI-e on its knees, with every vendor cooking its own Virtual Channel implementation in PCI-e; there is no way to get consistant latency during traffic loads. Since PCI-e needs loads of buffering for its VC's most vendors (well the only one with a 'switch') only has one Virtual Channel. The spec does not set a min. number of VC'c so its game on for vendors, and the losser is the user.
So its key to talk to vendors about latency and QoS, low deterministic latency is important, but when doing embedded realtime applications, QoS can be a bigger factor. HT is just not a player, no PICMG or ATCA standard, HT is not for Telecom, a few old old Cisco designs, and HT is parrelel so latency should be even lower.

Anonymous said...

The PCIe latency numbers are pretty much what is measured in real Intel platforms so the analysis seems reasonable in that aspect.

Cisco has their own interconnect technology for many comms offerings. They were founding promoters of HT and have continued to invest. So there is both a historical and technical basis for their continued use. Communications isn't something one can pass a broad stroke to come to only one answer. Communications is broken purposefully into a control plane and a data plane for just this reason as the mix of components used for each may be different. However, most designs use the same physical topology making latency a major issue especially as the speeds of routers, telecomm applications, and so forth increase. Which is best really comes down to the target cost model and latency requirements. Both have advantages and disadvantages.

Anonymous said...

interconnect guy clearly loves rapidio. unfortunately nobody else does so I don't see the point in even mentioning it in a generic comparison with HT and PCIe. rapidio will simply will NOT be adopted outside some military and wireless niches. so sticking with the mainstream, no suprise PCIe will win in generic comms - it may not be the best but it's good enough. customers who need more will just add their special sauce to the standard and do an asic, or convince some startup to do it for them with promise of payback from all those routers shipped. hey, good old PCI sucks too, but look how many PCI-based comms systems are out there. There's a long list of standards and products that were clearly superior that didn't make it because the inventors lost sight of ensuring mass market acceptance and having a broad ecosystem. (My personal favorite being Fast Ethernet vs. VG-AnyLAN back in the 90's. HP backed a great standard, but they were the only one selling it.)