• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

NVIDIA Commercializes Silicon Photonics with InfiniBand and Ethernet Switches

AleksandarK

News Editor
Staff member
Joined
Aug 19, 2017
Messages
2,887 (1.04/day)
NVIDIA has developed co-packaged optics (CPO) technology with TSMC for its upcoming Quantum-X InfiniBand and Spectrum-X Ethernet switches, integrating silicon photonics directly onto switch ASICs. The engineering approach reduces power consumption by 3.5x. It decreases signal loss from 22 dB to 4 dB compared to traditional pluggable optics, addressing critical power and connectivity limitations in large-scale GPU deployments, especially in 10,000+ GPU systems. The architecture incorporates continuous wave laser sources within the switch chassis, consuming 2 W per port, compared to the 10 W required by conventional externally modulated lasers in pluggable modules. This configuration, combined with integrated optical engines that use 7 W versus 20 W for traditional digital signal processors, reduces total optical interconnect power from approximately 72 MW to 21.6 MW in a 400,000 GPU data center scenario.

Specifications for the Quantum 3450-LD InfiniBand model include 144 ports running at 800 Gb/s, delivering 115 Tb/s of aggregate bandwidth using four Quantum-X CPO sockets in a liquid-cooled chassis. The Spectrum-X lineup features the SN6810 with 128 ports at 800 Gb/s (102.4 Tb/s) and the higher-density SN6800 providing 512 ports at 800 Gb/s for 409.6 Tb/s total throughput. The Quantum-X InfiniBand implementation uses a monolithic switch ASIC with six CPO modules supporting 36 ports at 800 Gb/s, while the Spectrum-X Ethernet design employs a multi-chip approach with a central packet processing engine surrounded by eight SerDes chiplets. Both architectures utilize 224 Gb/s signaling per lane with four lanes per port. NVIDIA's Quantum-X switches are scheduled for availability in H2 2025, with Spectrum-X models following in H2 2026.



Below is a video with more technical specs, in case you are interested, with nice showcase of what these switches are capable of.

View at TechPowerUp Main Site | Source
 
Joined
Jan 3, 2021
Messages
3,987 (2.59/day)
Location
Slovenia
Processor i5-6600K
Motherboard Asus Z170A
Cooling some cheap Cooler Master Hyper 103 or similar
Memory 16GB DDR4-2400
Video Card(s) IGP
Storage Samsung 850 EVO 250GB
Display(s) 2x Oldell 24" 1920x1200
Case Bitfenix Nova white windowless non-mesh
Audio Device(s) E-mu 1212m PCI
Power Supply Seasonic G-360
Mouse Logitech Marble trackball, never had a mouse
Keyboard Key Tronic KT2000, no Win key because 1994
Software Oldwin
Joined
May 22, 2010
Messages
423 (0.08/day)
Processor R7-7700X
Motherboard Gigabyte X670 Aorus Elite AX
Cooling Scythe Fuma 2 rev B
Memory no name DDR5-5200
Video Card(s) Some 3080 10GB
Storage dual Intel DC P4610 1.6TB
Display(s) Gigabyte G34MQ + Dell 2708WFP
Case Lian-Li Lancool III black no rgb
Power Supply CM UCP 750W
Software Win 10 Pro x64
But i thought infiniband was a deprecated technology with everything going to ethernet now, with ultraethernet and whatnot
 
Top