A Chinese tech forum ChipHell user who goes by zcjzcj11111 sprung up a fascinating take on what the next-generation AMD "Navi 48" GPU could be, and put their imagination on a render. Apparently, the "Navi 48," which powers AMD's series-topping performance-segment graphics card, is a dual chiplet-based design, similar to the company's latest Instinct MI300 series AI GPUs. This won't be a disaggregated GPU such as the "Navi 31" and "Navi 32," but rather a scale-out multi-chip module of two GPU dies that can otherwise run on their own in single-die packages. You want to call this a multi-GPU-on-a-stick? Go ahead, but there are a couple of changes.

On AMD's Instinct AI GPUs, the chiplets have full cache coherence with each other, and can address memory controlled by each other. This cache coherence makes the chiplets work like one giant chip. In a multi-GPU-on-a-stick, there would be no cache coherence, the two dies would be mapped by the host machine as two separate devices, and then you'd be at the mercy of implicit or explicit multi-GPU technologies for performance to scale. This isn't what's happening on AI GPUs—despite multiple chiplets, the GPU is seen by the host as a single PCI device with all its cache and memory visible to software as a contiguously addressable block.

Nikon Corporation (Nikon) is developing a digital lithography system with resolution of one micron (L/S) and high productivity for advanced semiconductor packaging applications. This product is scheduled to be released in Nikon's fiscal year 2026.

The rapid adoption of artificial intelligence (AI) technology is driving demand for integrated circuits (ICs) for data centers. In the field of advanced packaging, including chiplets, the size of packages is increasing with the miniaturization of wiring patterns. This will lead to heightened demand for panel level packages that use glass and other materials suitable for larger packages, requiring exposure equipment that combines high resolution with a large exposure area. To meet these demands, Nikon is developing digital exposure equipment that combines the high-resolution technology of its semiconductor lithography systems, which has been cultivated over many decades, along with the excellent productivity made possible with the multi-lens technology of its FPD lithography systems.

Synopsys, Inc. today announced the industry's first complete UCIe IP solution operating at up to 40 Gbps per pin to address the increased compute performance requirements of the world's fastest AI data centers. The UCIe interconnect, the de facto standard for die-to-die connectivity, is critical for high-bandwidth, low-latency die-to-die connectivity in multi-die packages, enabling more data to travel efficiently across heterogeneous and homogeneous dies, or chiplets, in today's AI data center systems.

Synopsys' 40G UCIe IP supports both organic substrate and high-density, advanced packaging technologies to give designers the flexibility to explore the packaging options that best fit their needs. The complete Synopsys 40G UCIe IP solution, including PHY, controller, and verification IP, is a key component of Synopsys' comprehensive and scalable multi-die solution for fast heterogeneous integration from early architecture exploration to manufacturing.

E&R Engineering Corp. hosted an event on August 28, 2024, in Taipei, Taiwan, where they launched the "E-Core System." This initiative, a combination of "E&R" and "Glass Core" inspired by the sound of "Ecosystem," led to the establishment of the "Glass Substrate Supplier E-Core System Alliance." The alliance aims to combine expertise to promote comprehensive solutions, providing equipment and materials for next-generation advanced packaging with glass substrates to both domestic and international customers.

E&R's E-Core Alliance includes Manz AG, Scientech for wet etching, HYAWEI OPTRONICS for AOI optical inspection, Lincotec, STK Corp., Skytech, Group Up for sputtering and ABF lamination equipment, and other key component suppliers such as HIWIN, HIWIN MIKROSYSTEM, Keyence Taiwan, Mirle Group, ACE PILLAR CHYI DING), and Coherent.

During the Semicon Taiwan 2024 summit event, TSMC VP of Advanced Packaging Technology, Jun He, spoke about the importance of merging AI chip memory and logic chips using 3D IC technology. He predicted that by 2030 the worldwide semiconductor industry would hit the $1 trillion milestone with HPC and AI leading 40 percent of the market share. In 2027, TSMC will introduce the 2.5D CoWoS technology that includes eight A16 process chipsets and 12 HBM4. AI processors that use this technology will not only be much cheaper to produce but will also provide engineers with a greater level of convenience. Engineers will have the option to write new codes into them instead. Manufacturers are cutting the SoC and HBM architectural conversion and mass production costs down to nearly one-fourth.

Nevertheless, the increasing production capacities of 3D IC technology remain the main challenge, as the size of chips and the complexity of manufacturing are decisive factors. However, the higher the size of the chips, the more chiplets are added, and thus the performance is improved, but this now makes the process even more complicated and is associated with more risks of misalignment, breakage, and extraction failure.

Ampere has announced a significant update to its product roadmap, highlighting the upcoming 512-core AmpereOne Aurora processor. This new chip is specifically designed to address the growing demands of cloud-native AI computing.

The newly announced AmpereOne Aurora 512 cores processor integrates AI acceleration and on-chip High Bandwidth Memory (HBM), promising three times the performance per rack compared to current AmpereOne processors. Aurora is designed to handle both AI training and inference workloads, indicating Ampere's commitment to becoming a major player in the AI computing space.

AMD reportedly plans to incorporate glass substrates into its high-performance system-in-packages (SiPs) sometimes between 2025 and 2026. Glass substrates offer several advantages over traditional organic substrates, including superior flatness, thermal properties, and mechanical strength. These characteristics make them well-suited for advanced SiPs containing multiple chiplets, especially in data center applications where performance and durability are critical. The adoption of glass substrates aligns with the industry's broader trend towards more complex chip designs. As leading-edge process technologies become increasingly expensive and yield gains diminish, manufacturers turn to multi-chiplet designs to improve performance. AMD's current EPYC server processors already incorporate up to 13 chiplets, while its Instinct AI accelerators feature 22 pieces of silicon. A more extreme testament is Intel's Ponte Vecchio, which utilized 63 tiles in a single package.

Glass substrates could enable AMD to create even more complex designs without relying on costly interposers, potentially reducing overall production expenses. This technology could further boost the performance of AI and HPC accelerators, which are a growing market and require constant innovation. The glass substrate market is heating up, with major players like Intel, Samsung, and LG Innotek also investing heavily in this technology. Market projections suggest explosive growth, from $23 million in 2024 to $4.2 billion by 2034. Last year, Intel committed to investing up to 1.3 trillion Won (almost one billion USD) to start applying glass substrates to its processors by 2028. Everything suggests that glass substrates are the future of chip design, and we await to see first high-volume production designs.

MediaTek announced today at COMPUTEX 2024 that the company has joined Arm Total Design, a fast-growing ecosystem that aims to accelerate and simplify the development of products based on Arm Neoverse Compute Subsystems (CSS). Arm Neoverse CSS is designed to meet the performance and efficiency needs of AI applications in the data center, infrastructure systems, telecommunications, and beyond.

"Together with Arm, we're enabling our customers' designs to meet the most challenging workloads for AI applications, maximizing performance per watt," said Vince Hu, Corporate Vice President at MediaTek. "We will be working closely with Arm as we expand our footprint into data centers, utilizing our expertise in hybrid computing, AI, SerDes and chiplets, and advance packaging technologies to accelerate AI innovation from the edge to the cloud."

At the Intel Vision 2024 customer and partner conference, Intel introduced the Intel Gaudi 3 accelerator to bring performance, openness and choice to enterprise generative AI (GenAI), and unveiled a suite of new open scalable systems, next-gen products and strategic collaborations to accelerate GenAI adoption. With only 10% of enterprises successfully moving GenAI projects into production last year, Intel's latest offerings address the challenges businesses face in scaling AI initiatives.

"Innovation is advancing at an unprecedented pace, all enabled by silicon - and every company is quickly becoming an AI company," said Intel CEO Pat Gelsinger. "Intel is bringing AI everywhere across the enterprise, from the PC to the data center to the edge. Our latest Gaudi, Xeon and Core Ultra platforms are delivering a cohesive set of flexible solutions tailored to meet the changing needs of our customers and partners and capitalize on the immense opportunities ahead."

Intel Core Ultra "Lunar Lake-MX" will be the company's bulwark against Apple's M-series Pro and Max chips, designed to power the next crop of performance ultraportables. The MX codename extension denotes MoP (memory-on-package), which sees stacked LPDDR5X memory chips share the package's fiberglass substrate with the chip, to conserve PCB footprint, and give Intel greater control over the right kind of memory speed, timings, and power-management features suited to its microarchitecture. This is essentially what Apple does with its M-series SoCs powering its MacBooks and iPad Pros. Igor's Lab scored the motherlode on the way Intel has restructured the various components across its chiplets, and the various I/O wired to the package.

When compared to "Meteor Lake," the "Lunar Lake" microarchitecture sees a small amount of "re-aggregation" of the various logic-heavy components of the processor. On "Meteor Lake," the CPU cores and the iGPU sat on separate tiles—Compute tile and Graphics tile, respectively, with a large SoC tile sitting between them, and a smaller I/O tile that serves as an extension of the SoC tile. All four tiles sat on top of a Foveros base tile, which is essentially an interposer—a silicon die that facilitates high-density microscopic wiring between the various tiles that are placed on top of it. With "Lunar Lake," there are only two tiles—the Compute tile, and the SoC tile.

Last year, Arm introduced its Neoverse Compute Subsystem (CSS) for the N2 and V2 series of data center processors, providing a reference platform for the development of efficient Arm-based chips. Major cloud service providers like AWS with Graviton 4 and Trainuium 2, Microsoft with Cobalt 100 and Maia 100, and even NVIDIA with Grace CPU and Bluefield DPUs are already utilizing custom Arm server CPU and accelerator designs based on the CSS foundation in their data centers. The CSS allows hyperscalers to optimize Arm processor designs specifically for their workloads, focusing on efficiency rather than outright performance. Today, Arm has unveiled the next generation CSS N3 and V3 for even greater efficiency and AI inferencing capabilities. The N3 design provides up to 32 high-efficiency cores per die with improved branch prediction and larger caches to boost AI performance by 196%, while the V3 design scales up to 64 cores and is 50% faster overall than previous generations.

Both the N3 and V3 leverage advanced features like DDR5, PCIe 5.0, CXL 3.0, and chiplet architecture, continuing Arm's push to make chiplets the standard for data center and cloud architectures. The chiplet approach enables customers to connect their own accelerators and other chiplets to the Arm cores via UCIe interfaces, reducing costs and time-to-market. Looking ahead, Arm has a clear roadmap for its Neoverse platform. The upcoming CSS V4 "Adonis" and N4 "Dionysus" designs will build on the improvements in the N3 and V3, advancing Arm's goal of greater efficiency and performance using optimized chiplet architectures. As more major data center operators introduce custom Arm-based designs, the Neoverse CSS aims to provide a flexible, efficient foundation to power the next generation of cloud computing.

Raytheon, an RTX business, has been awarded a $20 million contract through the Strategic and Spectrum Missions Advanced Resilient Trusted Systems (S2MARTS) consortium to develop a next-generation multi-chip package for use in ground, maritime and airborne sensors. Under the contract, Raytheon will package state-of-the-art commercial devices from industry partners like AMD to create a compact microelectronics package that will convert radio frequency energy to digital information with more bandwidth and higher data rates. The integration will result in new system capabilities designed with higher performance, lower power consumption and reduced weight.

"By teaming with commercial industry, we can incorporate cutting-edge technology into Department of Defense applications on a much faster timescale," said Colin Whelan, president of Advanced Technology at Raytheon. "Together, we will deliver the first multi-chip package that features the latest in interconnect ability - which will provide new system capabilities to our warfighters."

According to the report from a journal called Fundamental Research, researchers from the Institute of Computing Technology at the Chinese Academy of Sciences have developed a 256-core multi-chiplet processor called Zhejiang Big Chip, with plans to scale up to 1,600 cores by utilizing an entire wafer. As transistor density gains slow, alternatives like multi-chiplet architectures become crucial for continued performance growth. The Zhejiang chip combines 16 chiplets, each holding 16 RISC-V cores, interconnected via network-on-chip. This design can theoretically expand to 100 chiplets and 1,600 cores on an advanced 2.5D packaging interposer. While multi-chiplet is common today, using the whole wafer for one system would match Cerebras' breakthrough approach. Built on 22 nm process technology, the researchers cite exascale supercomputing as an ideal application for massively parallel multi-chiplet architectures.

Careful software optimization is required to balance workloads across the system hierarchy. Integrating near-memory processing and 3D stacking could further optimize efficiency. The paper explores lithography and packaging limits, proposing hierarchical chiplet systems as a flexible path to future computing scale. While yield and cooling challenges need further work, the 256-core foundation demonstrates the potential of modular designs as an alternative to monolithic integration. China's focus mirrors multiple initiatives from American giants like AMD and Intel for data center CPUs. But national semiconductor ambitions add urgency to prove domestically designed solutions can rival foreign innovation. Although performance details are unclear, the rapid progress shows promise in mastering modular chip integration. Combined with improving domestic nodes like the 7 nm one from SMIC, China could easily create a viable Exascale system in-house.

During the recent IEDM conference, TSMC previewed its process roadmap for delivering next-generation chip packages packing over one trillion transistors by 2030. This aligns with similar long-term visions from Intel. Such enormous transistor counts will come through advanced 3D packaging of multiple chipsets. But TSMC also aims to push monolithic chip complexity higher, ultimately enabling 200 billion transistor designs on a single die. This requires steady enhancement of TSMC's planned N2, N2P, N1.4, and N1 nodes, which are slated to arrive between now and the end of the decade. While multi-chipset architectures are currently gaining favor, TSMC asserts both packaging density and raw transistor density must scale up in tandem. Some perspective on the magnitude of TSMC's goals include NVIDIA's 80 billion transistor GH100 GPU—among today's largest chips, excluding wafer-scale designs from Cerebras.

Yet TSMC's roadmap calls for more than doubling that, first with over 100 billion transistor monolithic designs, then eventually 200 billion. Of course, yields become more challenging as die sizes grow, which is where advanced packaging of smaller chiplets becomes crucial. Multi-chip module offerings like AMD's MI300X and Intel's Ponte Vecchio already integrate dozens of tiles, with PVC having 47 tiles. TSMC envisions this expansion to chip packages housing more than a trillion transistors via its CoWoS, InFO, 3D stacking, and many other technologies. While the scaling cadence has recently slowed, TSMC remains confident in achieving both packaging and process breakthroughs to meet future density demands. The foundry's continuous investment ensures progress in unlocking next-generation semiconductor capabilities. But physics ultimately dictates timelines, no matter how aggressive the roadmap.

Ayar Labs, a leader in silicon photonics for chip-to-chip connectivity, will showcase its in-package optical I/O solution integrated with Intel's industry-leading Agilex Field-Programmable Gate Array (FPGA) technology. In demonstrating 5x current industry bandwidth at 5x lower power and 20x lower latency, the optical FPGA - packaged in a common PCIe card form factor - has the potential to transform the high performance computing (HPC) landscape for data-intensive workloads such as generative artificial intelligence (AI), machine learning, and support novel new disaggregated compute and memory architectures and more.

"We're on the cusp of a new era in high performance computing as optical I/O becomes a 'must have' building block for meeting the exponentially growing, data-intensive demands of emerging technologies like generative AI," said Charles Wuischpard, CEO of Ayar Labs. "Showcasing the integration of Ayar Labs' silicon photonics and Intel's cutting-edge FPGA technology at Supercomputing is a concrete demonstration that optical I/O has the maturity and manufacturability needed to meet these critical demands."

Socionext today announced a collaboration with Arm and TSMC for the development of an innovative power-optimized 32-core CPU chiplet in TSMCʼs 2 nm silicon technology, delivering scalable performance for hyperscale data center server, 5/6G infrastructure, DPU and edge-of- network markets.

The engineering samples are targeted to be available in 1H2025. This advanced CPU chiplet proof-of-concept using Arm Neoverse CSS technology is designed for single or multiple instantiations within a single package, along with IO and application-specific custom chiplets to optimize performance for a variety of end applications.

Zero ASIC, a semiconductor startup, came out of stealth today to announce early access to its one-of-a-kind ChipMaker platform, demonstrating a number of world firsts:

• 3D chiplet composability enabling billions of new silicon products
• Fully automated no-code chiplet-based chip design
• Zero install interactive RTL-based chip emulation
• Roadmap to 100X reduction in chip development costs

"Custom Application Specific Integrated Circuits (ASICs) offer 10-100X cost and energy advantage over commercial off the shelf (COTS) devices, but the enormous development cost makes ASICs non-viable for most applications," said Andreas Olofsson, CEO and founder of Zero ASIC. "To build the next wave of world changing silicon devices, we need to reduce the barrier to ASICs by orders of magnitude. Our mission at Zero ASIC is to make ordering an ASIC as easy as ordering catalog parts from an electronics distributor."

In an extraordinary leap forward for the chiplet industry, the groundbreaking collaboration between the Open Compute Project Foundation (OCP) and JEDEC is set to usher in a new era of innovation. By merging the capabilities and open standards of OCP's Chiplet Data Extensible Markup Language (CDXML) and JEDEC's JEP30 PartModel Guidelines, this partnership, initiated in late 2022, promises to revolutionize chiplet design, manufacturing and integration. The result will be a unified structure that supports both chiplets and general electronic parts within the overarching purview of JEDEC.

In a significant development, the integration of OCP CDXML into JEP30 has reached a critical milestone, enabling chiplet builders to provide standardized chiplet part descriptions to their customers electronically. This advancement opens the door to automating System in Package (SiP) design and assembly using chiplets. The chiplet descriptions encompass crucial information for SiP builders, including thermal properties, physical and mechanical requirements, behavior specifications, power and signal integrity properties, testing in-package and security parameters.

Tenstorrent, a company that sells AI processors and licenses AI and RISC-V IP, announced today that it selected Samsung Foundry to bring Tenstorrent's next generation of AI chiplets to market. Tenstorrent builds powerful RISC-V CPU and AI acceleration chiplets, aiming to push the boundaries of compute in multiple industries such as data center, automotive and robotics. These chiplets are designed to deliver scalable power from milliwatts to megawatts, catering to a wide range of applications from edge devices to data centers.

To ensure the highest quality and cutting-edge manufacturing capabilities for its chiplet, Tenstorrent has selected Samsung's Foundry Design Service team, known for their expertise in silicon manufacturing. The chiplets will be manufactured using Samsung's state-of-the-art SF4X process, which boasts an impressive 4 nm architecture.

TSMC today announced the new 3Dblox 2.0 open standard and major achievements of its Open Innovation Platform (OIP) 3DFabric Alliance at the TSMC 2023 OIP Ecosystem Forum. The 3Dblox 2.0 features early 3D IC design capability that aims to significantly boost design efficiency, while the 3DFabric Alliance continues to drive memory, substrate, testing, manufacturing, and packaging integration. TSMC continues to push the envelope of 3D IC innovation, making its comprehensive 3D silicon stacking and advanced packaging technologies more accessible to every customer.

"As the industry shifted toward embracing 3D IC and system-level innovation, the need for industry-wide collaboration has become even more essential than it was when we launched OIP 15 years ago," said Dr. L.C. Lu, TSMC fellow and vice president of Design and Technology Platform. "As our sustained collaboration with OIP ecosystem partners continues to flourish, we're enabling customers to harness TSMC's leading process and 3DFabric technologies to reach an entirely new level of performance and power efficiency for the next-generation artificial intelligence (AI), high-performance computing (HPC), and mobile applications."

NVIDIA's upcoming Blackwell GPU architecture, expected to succeed the current Ada Lovelace architecture, is gearing up to make some significant changes. While we don't have any microarchitectural leaks, rumors are circulating that Blackwell will have different packaging and die structures. One of the most intriguing aspects of the upcoming Blackwell is the mention of a Multi-Chip Module (MCM) design for the GB100 data-center GPU. This advanced packaging approach allows different GPU components to exist on separate dies, providing NVIDIA with more flexibility in chip customization. This could mean that NVIDIA can more easily tailor its chips to meet the specific needs of various consumer and enterprise applications, potentially gaining a competitive edge against rivals like AMD.

While Blackwell's release is still a few years away, these early tidbits paint a picture of an architecture that isn't just an incremental improvement but could represent a more significant shift in how NVIDIA designs its GPUs. NVIDIA's potential competitor is AMD's upcoming MI300 GPU, which utilized chiplets in its designs. Chiplets also provide ease of integration as smaller dies provide better wafer yields, meaning that it makes more sense to switch to smaller dies and utilize chiplets economically.

What's New: Intel today announced one of the industry's first glass substrates for next-generation advanced packaging, planned for the latter part of this decade. This breakthrough achievement will enable the continued scaling of transistors in a package and advance Moore's Law to deliver data-centric applications.

"After a decade of research, Intel has achieved industry-leading glass substrates for advanced packaging. We look forward to delivering these cutting-edge technologies that will benefit our key players and foundry customers for decades to come."
-Babak Sabi, Intel senior vice president and general manager of Assembly and Test Development

d-Matrix, the leader in high-efficiency generative AI compute for data centers, has closed $110 million in a Series-B funding round led by Singapore-based global investment firm Temasek. The goal of the fundraise is to enable d-Matrix to begin commercializing Corsair, the world's first Digital-In Memory Compute (DIMC), chiplet-based inference compute platform, after the successful launches of its prior Nighthawk, Jayhawk-I and Jayhawk II chiplets.

d-Matrix's recent silicon announcement, Jayhawk II, is the latest example of how the company is working to fundamentally change the physics of memory-bound compute workloads common in generative AI and large language model (LLM) applications. With the explosion of this revolutionary technology over the past nine months, there has never been a greater need to overcome the memory bottleneck and current technology approaches that limit performance and drive up AI compute costs.

QuickLogic Corporation, a developer of embedded FPGA (eFPGA) IP, ruggedized FPGAs and Endpoint AI/ML solutions, and YorChip, a pioneering startup specializing in UCIe-compatible IP, have formed a strategic partnership to revolutionize the world of FPGA chiplets. The collaboration will result in a groundbreaking lineup of FPGA chiplets optimized for low power consumption and low cost, opening new possibilities for a wide range of applications, including the fast-growing edge IoT and AI/ML markets.

According to Yole Group, a market research company, by 2023, they expect chiplet adoption will lead to a TAM of chiplet-based integrated circuits in excess of $200B, across the consumer, automotive defense, aerospace, industrial, and medical markets. Since discrete FPGAs are already prevalent in those same markets, wide adoption of eFPGA-based UCIe (Unified Chiplet Interconnect Express) enabled chiplets is expected, and QuickLogic and YorChip are well-positioned to capitalize on this growth opportunity.

The GPU at the heart of the China-exclusive AMD Radeon RX 7900 GRE (Golden Rabbit Edition) sparked much curiosity. It is a physically different GPU from the one found in desktop Radeon RX 7900 XT and RX 7900 XTX graphics cards. AMD wouldn't go through all that effort designing a whole different GPU just for a limited edition graphics card, which means this silicon could find greater use for the company—for example, this could be the package AMD uses for its upcoming mobile RX 7900 series. AMD wouldn't go through all the effort designing a first-party MBA (made by AMD) PCB for the silicon just for the RX 7900 GRE, and so this PCB, with this particular version of the "Navi 31" silicon, could see a wider global launch, probably as the rumored Radeon RX 7800 XT, or something else (although with a different set of specs from the RX 7900 GRE).

We compared the sizes of the new "Navi 31" package found in the RX 7900 GRE, with those of the regular "Navi 31" powering the RX 7900 XT/XTX, the previous-generation "Navi 21" powering the RX 6900 XT, and the NVIDIA AD103 silicon powering the desktop GeForce RTX 4080. There are some interesting findings. The new smaller "Navi 31" package is visibly smaller than the one powering the RX 7900 XT/XTX. It is a square package, compared to the larger rectangular one, and has a significantly thinner metal reinforcement brace. What's interesting is that the 5 nm GCD is still surrounded by six 6 nm MCDs. We don't know if they've disabled two of the six MCDs, or whether they're dummies. AMD uses dummy chiplets as structural reinforcement in some of its EPYC server processors. The dummies spread some of the mounting pressure applied by the IHS or cooling solution, so the logic behind surrounding the GCD with six of these MCDs could be the same.