networkZONE Products for the week of June 17, 2002
Broadcom Says . . .
Raw Bandwidth Rules - Broadcom's Carrier-Class Switch
Fabric Guarantees Per Flow QoS, Graceful Redundancy Without Complex Metering
Mechanisms
Broadcom Corporation is now offering a highly integrated carrier-class
switch fabric chipset that can scale from 80 Gigabits per second (Gbps)
up to 1.2 Terabits per second (Tbps), optimizing performance while guaranteeing
end-to-end subscriber bandwidth. Boadcom's new switch fabric chipset provides
the building blocks for transferring data among high speed line cards in
ATM and Multi-Protocol Label Switching (MPLS) multi-service switches, core
enterprise switches, metro edge routers, and core IP routers as well as
in next-generation fiber channel switches.
The new Broadcom chipset enables equipment vendors to build a range of systems that accelerate the deployment of high-speed Internet Protocol (IP)-based service level agreements (SLA) supporting voice, video and data traffic. With this capability, service providers can support new revenue generating offerings, such as Transparent Local Area Network Services (TLS), while maintaining SLA guarantees on a per subscriber basis. These services will ultimately enable corporate IT managers to ubiquitously connect multiple enterprise networks across a Wide Area Network (WAN), with the manageability and flexibility of an enterprise Local Area Network (LAN).
"Up until now, there hasn't been a scalable switch fabric architecture for IP-based networks that can be leveraged across multiple platforms and provide the same guaranteed bandwidth and carrier-class reliability as ATM," said Robert Rango, Vice President and General Manager of Broadcom's Wide Area Networking Business Unit. "Our switch chipset provides equipment vendors with a highly integrated, low power, modular back plane solution for emerging MPLS traffic engineered networks with true Quality of Service (QoS)."
Utilizing the company's field-proven integrated 3.125 Gbps Serializer/Deserializer (SerDes) technology, Broadcom's switch fabric chipset achieves the highest level of throughput in its class, supporting wire-speed performance on all packet sizes. It also features the ability to scale from 80 Gbps up to 1.2 Tbps and guarantee bandwidth regardless of network congestion. This capability allows equipment manufacturers to leverage a single hardware and software design team to build modular line cards that are portable across multiple platforms. While this approach enables platforms to cost-effectively scale with fabric bandwidth, it also creates operational savings for service providers by reducing inventory costs through the reuse of line cards into a range of high- and low-end platforms.
Utilizing a combination of virtual output queuing, per output port queuing and per-subscriber flow control, Broadcom's switch fabric guarantees bandwidth on a per-subscriber basis even if other subscribers are violating the agreed upon SLA. As a result, manufacturers can build equipment that can guarantee bandwidth up to the full line rate, enabling service providers to ensure SLA's on a per subscriber basis. Furthermore, service providers can dynamically adjust bandwidth guarantees remotely, providing a cost-effective method of changing a subscriber's SLA.
Broadcom's new switch fabric chipset also offers redundancy with a SerDes level of granularity. This unique approach utilizes the system bandwidth more effectively, does not require standby chips for link failures, and gives manufacturers the flexibility to use different redundancy schemes for various levels of reliability. The entire connection bandwidth is always available to all subscribers while maintaining carrier-class reliability.
Two New Products: BCM8320 and BCM8332
The switch fabric chipset consists of the Broadcom BCM8320 Fabric Interface
Chip and BCM8332 Switch Fabric Chip. The BCM8320 contains a bi-directional
dual 10-Gbps CSIX interface with integrated queue management, scheduling,
memory and 16 integrated 3.125 Gbps SerDes. The BCM8332 implements a 32-port
switch with integrated queue management, scheduling, memory and 32 integrated
3.125 Gbps SerDes. Broadcom's switch fabric chipset requires no external
memory and no external schedulers, and uses less than half the power of
its nearest competitor.
The BCM8320 and BCM8332 are protocol independent with a standards-based CSIX interface, providing a flexible solution that can deliver guaranteed QoS to multiple protocol line cards, including Ethernet, ATM, Packet over SONET and TDM. Manufacturers can use the BCM8320 and BCM8332 as the core building blocks for the development of switches and routers, supporting multiple protocols. Using Broadcom's chips as the foundation for equipment that implements protocols such as IP over MPLS combined with Resource-Reservation Protocol (RSVP) enables network operators to effectively manage their high-speed networks and avoid bottlenecks.
The BCM8320 and BCM8332 are part of Broadcom's MetroSwitch family of products, which provide highly integrated solutions for the transmission of packet-based voice, video, and data traffic in the Metro Area Network (MAN). The MetroSwitch family leverages Broadcom's unique expertise in integrating high-speed CMOS physical layer blocks with advanced packet processing and switching technologies. All MetroSwitch products offer standards-compliant interfaces for direct connection to 2.5 Gbps and 10 Gbps network processors, traffic managers and internally-developed customer Application Specific Integrated Circuits (ASICs).
Reference Platform Information
To reduce system design time and accelerate customer time to market with
the BCM8320 and BCM8332, Broadcom has developed a switch chassis reference
platform, the BCM98300, consisting of up to eight BCM98320 reference line
cards and up to four BCM98332 switch fabric reference line cards.
The BCM98320 reference line card is built using the BCM8320 Fabric Interface Chip and the Broadcom BCM5632, a twelve-Gigabit Ethernet plus one 10-Gigabit Ethernet switch processor. This line card also offers multiple large scale FPGAs that allow the user to implement traffic management algorithms.
The BCM98320 switch fabric reference line card also includes an XAUI
interface port as well as a HyperTransport port that can be used to connect
to other Broadcom reference designs. The BCM98300 switch chassis reference
platform comes complete with software driver support and a full Application
Programming Interface (API) suite. Schematics and layout files are also
included.
analogZONE Says . . .
Broadcom's most recent attempt at a Terabit switch fabric was based on the assumption that carriers will start to need guaranteed QoS on a per-flow basis for packet-based traffic. Broadcom says that equipment providers must-haves include, support for SLAs, scalability to multi (100s) of Gigabits, reliability, and multi-Gigabit user interfaces which can be broken down and "retailed" on a 10- or 100-Mbit/s basis. I'd agree with this assessment and am very impressed with the solution that the wizards of Irvine have come up with.
If they have gotten the architecture right, this market may be a place where Broadcom can leverage its ability to cram enormous amounts of logic and analog circuitry onto a CMOS chip into a tremendous advantage. While the claims that they make for these chips seem a bit overstated at first glance, long conversations with Eric Hayes, the product line manager for the switch fabric, gave me enough insights on their inner workings that I think they stand a good chance on delivering on them. This faith is reinforced by the fact that it is already sampling (Radlan is one of first customers.)
In essence, the chip set is a switch controller and switch fabric that uses a decentralized queue management system and virtual output queuing to provide strictly enforced per-flow QoS management and SLA enforcement capability. Switches of varying sizes can be built out of two basic elements, the BCM8320 queue manager/scheduler, and the BCM8332 switch fabric element.
Before we look more closely at the chips themselves, it might be a good idea to get a quick overview of how these two elements work together. Note: The following explanation follows next is a bit tricky to follow, but I'll do my best to keep you from getting a heavy dose of MEGO.
The 8320 queue manager has 16, 2.5 Gbit/s (3.125 Gbits/s if you include 8B/10B line coding) integrated bi-directional SREDES ports, that can connect to any one of the 8332's 32 SERDES ports. Unless I misunderstood something (or Broadcom is lying through its teeth), the fully non-blocking switch architecture, combined with a unique double-output queue management system (more on this later) guarantees that every switching path can run full-out at max bandwidth without suffering any congestion.
For a "small" application, you can plug two controllers into a single switch fabric element to yield a 40-Gbit switch. If you want to build a symmetric switch fabric (where the capacity of the switch is equal to the capacity of the feeds), the largest structure you can create is 16 switch elements connected to 32 queue managers. If you can handle some level of blocking, you can build even larger "asymmetric" architectures which oversubscribe the switch fabric.
The switch fabric is fed from line side of the BCM8320 queue manager/scheduler via a pair of 10-Gbit CSIX interfaces. It accepts CSIX-standard C-frames, breaks up the random-length packets into uniform-length cells, and classifies them for scheduling. In the process, it also converts the CSIX header into its own format, and adds overhead capacity in the SERDES stream to support the burden of cell headers and the extra bits generated when packet/cell conversion results in cells that are only partially full of data. The headers are used by the 8320's integrated queue managers as they direct them to an on-chip RAM that serves as a virtual output queue (VoQ), where each flow is queued according to its destination and class of service. Traffic is drawn from the output queues according to a very straightforward weighted round-robin (WRR) algorithm and shipped across the SERDES link to the switch fabric itself.
The 8332 switch fabric element, a non-blocking 32-by32 port switch fabric that can take the full capacity of each of its 10-Gbit/s ports and direct it to any other port on a packet-by-packet basis. It also has its own shallow buffers for internal flow control and preventing its pipelines from "starving."
This second output buffering stage, combined with the fabric's full-rate "any-to-any" capabilities means that every path has its own queuing and flow control. This allows the architecture to guarantee QoS on a per-subscriber basis - even when the switch is fully saturated with traffic. With enough bandwidth to assure no best-effort is needed, the scheduling algorithm can be much simpler (and more deterministic) than other switches which must work around blocking and throughput issues.
While most good switch fabrics can run at a significant fraction of their rated capacity, they usually begin to have difficulty maintaining rigid quality of service on all queues when the loading goes much over 60% or 70%. From what I can tell from two hours of conversation with Mr. Hayes, this new architecture from Broadcom delivers deterministic switching results whether the pipes are 5% or 95% full.
The 8332's any-to-any architecture is also a key technology for highly-effective redundancy. Since every SERDES link between switch fabric and controller is independent, the scheduler has multiple paths at its disposal to ensure a stream of traffic can find its way to the desired output port. This means that in a symmetrical application a single link failure simply degrades the overall capacity of the switch, not a particular connection. One of the benefits is that you actually get to use the extra capacity built into a redundant system, not simply leaving large sections of switch fabric idle until a fail-over.
One of the few obvious problems with this sort of architecture is that it needs lots of RAM to create the huge numbers of virtual output queues on the scheduler and switch fabric. Thankfully, Broadcom has had lots of experience in producing highly-integrated chips. While their on-chip RAM capabilities are relatively new, the working silicon they have seems to indicate that they are as proficient at this as they are large oceans of gates and integrated PHY-layer devices. It's a good thing too, since the transistor count that Eric gave me for the 8332 (the exact number is "classified") makes the device look like a very large, specialized RAM wrapped in a thin candy-coating of logic and driver devices.
The other apparent problem with this architecture is that it suffers from n-squared complexity problems as it scales upward. As the port size grows, the pin count starts to grow, but the size of the buffer RAM grows even faster. It's hard to imagine that we'll see even a doubling of the current port count without resorting to a clever work-around of one sort or another.
But, for the moment, the scaling limits are mostly an academic point. With a fully-built-out switch fabric, 32 BCM8320s and 16 BCM8332s yield a switch with a total full-duplex capacity of 640 Gbit/s (Broadcom advertises this as 1.2 Terabit/s half-duplex). Since today's top-line core routers are boasting capacities of 100 Gbit/s, with 320 on the horizon for a year or two, the chip set should be in the "sweet spot" for performance and capacity for 2-4 years, or longer. By that time, I expect Broadcom's designers will have either come up with a work-around to the scaling limits of the current architecture, or developed a follow-on architecture that keeps this series of switch chips in a leadership position.
Unless I'm missing something about the overall design, this relatively straightforward chip set should be able to deliver as-advertised on most or all of its specs. This of course assumes that chips this large can actually be produced with sufficient yields and timing consistency to be commercially viable. While no small feat in itself, it is something that Broadcom has done before.
While I occasionally quarrel with Broadcom about overstated claims for its products, I think that they are spot-on this time and have developed a winning chip set that will meet or exceed expectations.
The BCM8320 and BCM8332 will be available to sample
in the third quarter of 2002 with the BM8320 priced at $400 in 10-k piece
lots in a FCBGA-1936, and the BCM8332 will be priced at $600 also in 10-k
piece lots in a FCBGA-1521.
|