The AI data center buildout requires massive bandwidth from accelerator to accelerator and from accelerator to CPU. At the core of that bandwidth bridge is PCIe technology, which constantly needs to evolve to satisfy massive bandwidth requirements. Today, PCI-SIG, the working group behind the PCI and PCIe connector, is releasing details about the almost ready 0.9 version of the PCIe 7.0 connector and its final specifications. The latest PCIe 7.0 will bring 128 GT/s speeds, with a bi-directional bandwidth of 512 GB/s in the x16 lane configuration. Targeting applications like 800G Ethernet, AI/ML, cloud, quantum computing, hyperscalers, military/aerospace, and cloud providers all need massive bandwidth for their respective applications and use cases to work flawlessly.

Interestingly, as PCIe doubles bandwidth over the traditional three-year cadence, high bandwidth for things like storage is becoming available on fewer and fewer lanes. For example, PCIe 3.0 with x16 lanes delivers 32 GB/s of bi-directional bandwidth. And now, PCIe 7.0 delivers that same bandwidth on only a single x1 lane. Some other goals of the new PCIe 7.0 include significant improvements in channel parameters and signal integrity while enhancing power efficiency and maintaining the protocol's low-latency characteristics. All while ensuring complete backward compatibility with previous generations of the standard. Notably, the PCIe 7.0 standard uses PAM4 signaling, which was first presented for PCIe 6.0. Here is a nice PAM4 signaling primer if you want to learn more about PAM4 signaling. Below are the specifications of PCIe generations and their respective characteristics. We expect to see final version v1.0 by end of the year, and some PCIe 7.0 accelerators next year.

Artificial intelligence has been a hot topic this year, and everything is now an AI processor, from CPUs to GPUs, NPUs, and many others. However, it was only a matter of time before we saw an integration of AI processing elements into the networking chips. Today, Broadcom announced its new Ethernet switching silicon called Trident 5-X12. The Trident 5-X12 delivers 16 Tb/s of bandwidth, double that of the previous Trident generation while adding support for fast 800G ports for connection to Tomahawk 5 spine switch chips. The 5-X12 is software-upgradable and optimized for dense 1RU top-of-rack designs, enabling configurations with up to 48x200G downstream server ports and 8x800G upstream fabric ports. The 800G support is added using 100G-PAM4 SerDes, which enables up to 4 m DAC and linear optics.

However, this is not only a switch chip on its own. Broadcom has added AI processing elements in an inference engine called NetGNT (Networking General-purpose Neural-network Traffic-analyzer). It can detect common traffic patterns and optimize data movement across the chip. Specifically, the company has listed an example of the system doing AI/ML workloads. In that case, NetGNT performs intelligent traffic analysis to avoid network congestion in these workloads. For example, it can detect the so-called "incast" patterns in real-time, where many flows converge simultaneously on the same port. By recognizing the start of incast early, NetGNT can invoke hardware-based congestion control techniques to prevent performance degradation without added latency.