Broadcom Inc. today announced the availability of its 3.5D eXtreme Dimension System in Package (XDSiP) platform technology, enabling consumer AI customers to develop next-generation custom accelerators (XPUs). The 3.5D XDSiP integrates more than 6000 mm² of silicon and up to 12 high bandwidth memory (HBM) stacks in one packaged device to enable high-efficiency, low-power computing for AI at scale. Broadcom has achieved a significant milestone by developing and launching the industry's first Face-to-Face (F2F) 3.5D XPU.

The immense computational power required for training generative AI models relies on massive clusters of 100,000 growing to 1 million XPUs. These XPUs demand increasingly sophisticated integration of compute, memory, and I/O capabilities to achieve the necessary performance while minimizing power consumption and cost. Traditional methods like Moore's Law and process scaling are struggling to keep up with these demands. Therefore, advanced system-in-package (SiP) integration is becoming crucial for next-generation XPUs. Over the past decade, 2.5D integration, which involves integrating multiple chiplets up to 2500 mm² of silicon and HBM modules up to 8 HBMs on an interposer, has proven valuable for XPU development. However, as new and increasingly complex LLMs are introduced, their training necessitates 3D silicon stacking for better size, power, and cost. Consequently, 3.5D integration, which combines 3D silicon stacking with 2.5D packaging, is poised to become the technology of choice for next-generation XPUs in the coming decade.

AMD has recently filed a patent revealing plans to implement "multi-chip stacking" in future Ryzen SoCs, as Wccftech reports, quoting a post on X from @coreteks: "New patent from AMD shows how future Zen SoCs could look. Basically a novel packaging design that enables compact chip stacking and interconnection by having them partially overlap, as in this figure. The dotted line is a larger die stacked on top of those smaller ones". The patent details a new approach where smaller chiplets partially overlap with a larger die, creating space for additional components and functions on the same die. This strategy aims to improve the efficiency of the contact area, thus making room for higher core counts, larger caches, and increased memory bandwidth within the same die size. The proposed stacking will reduce the physical distance between components through overlapping chiplets, thus minimizing interconnect latency and achieving faster communication between different chip parts. The design will also improve power management, as the segregated chiplets allow for better control of each unit through power gating.

Even if long-time rival Intel has lost some of its momentum (and market share) this year, AMD's chance to push ahead with its intention to become number one in the market is to continue to innovate. In the same way that its 3D V-Cache technology made the X3D processor lineup so successful, this chip stacking approach could play a major role in future AMD Ryzen SoCs. It seems that AMD is committed to moving away from the monolithic design era and taking the road of multi-chiplet; however, it can be a long wait until (and if) this chip stacking will complete the journey from patents to design, production, and final product.