Thursday, April 10, 2008

Moving at 10G rates

I've been amazed to see how rapidly the broad group of companies backing Fibre Channel over Ethernet has been moving, given how much there is to do to create this converged data center network. But after some further digging yesterday I see progress is not quite as fast as their marketing departments would have us believe.

The first FCoE products released this week are still pre-standard and many are less than elegant. The reality is the real product wave will hit sometime early next year, as I reported last night.

Still, the FCoE group has been gaining momentum in monthly meetings. A separate CEE Authors group is trying to crank up the pace on the (notoriously slow) IEEE standards efforts.

I give special kudos to Cisco for driving in the diamond lane on this initiative. Their Nexus 5000 shows not only fast time-to-market with an ASIC-laden design but an innovative business approach with its spin-in of start-up Nuova Systems. This tactic, first tried with Andiamo, helped Cisco attract outside data center expertise it needed with people such as Ed Bugnion, the former CTO of VMware who is now Nuova's chief technology officer.

A tip of the hat from the Interconnects blog for the nice work by the folks in Little Italy up on San Jose's Tasman drive.


Anonymous said...

In the current environment, the Ethernet fabric is leveraged across multiple customers, and customer traffic is segregated and isolated by VLANs and firewalls.

Fibre Channel traffic is on a separate leveraged fabric, and customer traffic is segregated and isolated by VSANs and FC zones.

So, we decide to move to a unified fabric. Due to product limitations, FCoE is available at the access layer only. Fibre Channel traffic now has to pass through firewalls, since it is encapsulated in an Ethernet jacket.

What are the ramifications of this?
What does this do to throughput over the newly-unified traffic?
What will this do to security models and practices?

Anonymous said...

FCoE is a layer 2 only protocol - it cannot traverse IP routers. Some call this a major limitation while others call it a reasonable trade-off. Given there is no IP involved and most firewalls operate at the transport / IP layers, it is not clear what your issue is?

As for actual convegence, well, you have to take a hard look at the bandwidth provisioned, how one overcomes the limitations of Ethernet's spanning tree, and so forth to determine whether there is really any physical convergence beyond simple layer 2 protocol convergence. If you want to ask the tougher questions that go beyond the marketing spin machines, then take a look at what customers are being asked to jump through to make FCoE happen. It isn't a silver or magic bullet and it isn't one that lends itself to the current IT transformation process. It is at best a green field technology that will have a number of issues to resolve before it can really replace native FC solutions or be used in place of iSCSI - both technologies are actually well tested with many petabytes of storage deployed. FCoE is a reasonable idea in some aspects but it isn't as easy as those very hand waving marketing types would have you believe. Will it happen? Yes because the players involved have deep cash pockets. Will it be the preferred technology for the future? Very unclear and something people need to take a very hard look at before investing heavily.

Rick Merritt said...

These comments on the limits of an FCoE approach are well taken, especially the note that FCoE is for layer 2 only and has no role yet at the router.

I'd like to hear more about this from both the pro and con sides.

Anonymous said...

FCoE Pros:

(a) It's Ethernet, Stupid. Ethernet volume economics for h/w

(b) Retain FC management experience - perhaps this is really a con but Cisco, et. al. says it is a good thing

(c) Retain FC gateway mentality, oh, but wait, Cisco Nexus does not do gateways only TTM Nuova does

(d) Requires new Ethernet standards, new Ethernet hardware, new Ethernet - great profit for vendors, poor customers

(e) Serial 16G FC isn't needed - of course, FC could follow Ethernet via multi-lane technology / DWDM

(f) Well.... trying to find another pro so instead I'll repeat the mantra that never fails - It's Ethernet, Stupid and that makes it right no matter what.

FCoE Cons:
1. It's Ethernet, Stupid. Not a 1:1 functional match to FC or even IB so differences equate to higher cost

2. Increased software management costs for customers, higher margins for vendors - convergence isn't free and customers can expect to be charged a premium for that software on top of actually low manufacturing cost hardware.

3. Doesn't really solve much in cable management - don't buy the marketing spin. If care about cable reduction then buy blades which provide greater consolidation at a much lower cost point in spite of the blogs from some who claim a proprietary network enclosure is better than a proprietary blade enclosure. Really should look at the margin difference between the two and ask why customers are not demanding greater price concessions from the network enclosures.

4. Still requires FC management training so paying for FC experience one way or the other

5. Still not IP based so can't replace iSCSI or value propositions derived from being IP based

6. Yet another gateway and convergence protocol to divert customers into buying new panacea technology yet to be proven in the market so expect lots of money to be poured down the drain while the kinks are worked out. Great for vendors' pockets but not necessarily customers. Could result in many customers focusing on simple 8G FC upgrade if they really need the bandwidth - and many do not - while letting bleeding edge and marketing take the risk and languishing on the vine.

Rick Merritt said...

What I hear is a valid concern about identifying any hidden costs in adopting what is still to some extent a new technology with FCoE.

I'd love to see a detailed and even-handed analysis of the overall news costs and savings.

Anonymous said...

The primary benefit of FCoE, IMO, is not to replace Fibre Channel or iSCSI SANs or enable I/O convergence. Rather it is to allow new servers to connect to existing FC SANs "for free". All servers come with at least 2 Ethernet ports and are connected to Ethernet switches (GbE today, 10GbE in the future) while Fibre Channel HBAs and switch ports are a substantial cost adder per server.

Once the FCoE stack is available in operating systems, every server will natively support FCoE "out of the box" just like they natively support iSCSI today. You only need to install a FCoE-to-FC gateway (e.g Cisco Nexus 5000)somewhere in the data center network to allow your new servers to access the FC SAN.

If you use iSCSI, great, you're already reaping this value. There are technical challenges to solve with FCoE, of course, but I believe the expanded SAN access and lower cost will be attractive.

Rick Merritt said...

This is a popular and practical school of techno-thought: Nothing dies and everythig needs to work together.

Anonymous said...

Storage was centralized for years, then de-centralized, and now the argument is to re-centralize. All equates to higher costs for the storage solution though a potential net savings for the IT operation. Not sure I buy the argument that every server will want FCoE though I do agree that a software only solution similar to what is found in iSCSI will allow such connectivity.

Does not really matter since there are so many customer variants with combinations of culture, cost, ability to absorb change, ego - you get the picture - that all of these technologies will be used to some capacity and people can point to their success no matter the outcome. FCoE is not driven by customer demand but by vendors recognizing an opportunity to create some amount of customer demand for the technology on the surface. Some though would argue that the real reason is to change the customer control point and that is what Nexus is all about - Cisco trying to establish that their network management and hardware products are the right place to build an IT operation. Server and storage providers are just cookiee cutter building blocks that can come from many vendors while networking can only come from one. Same applies to VMware which is trying to parlay the virtualization space into the higher level resource management control point for customers. The equity stake Cisco placed in VMware may be an acknowledgement of the two vendors trying to largely divide up the profitable portions of the IT operation while leaving the lower margin volume hardware problems to the others. Control points are where it is at and FCoE is just one more example of how that control point argument is being played out. Check out the blogs in Cisco's web site for more discussion on where they are headed (the recent foray into appliances illustrates this point as well).