This week, at the 2024 IEEE International Electron Devices Meeting (IEDM), imec, a world-leading research and innovation hub in nanoelectronics and digital technologies, proposes a novel 3D integrated charge-coupled device (CCD) that can operate as a block-addressable buffer memory, in support of data-intensive compute applications. Memory operation is demonstrated on a planar proof-of-concept CCD structure which can store 142 bits. Implementing an oxide semiconductor channel material (such as IGZO) ensures sufficiently long retention time and enables 3D integration in a cost-efficient, 3D NAND-like architecture. Imec expects the 3D CCD memory density to scale far beyond the DRAM limit.

The recent introduction of the compute express link (CXL) memory interface provides opportunities for new memories to complement DRAM in data-intensive compute applications like AI and ML. One example is the CXL type-3 buffer memory, envisioned as an off-chip pool of memories that 'feeds' the various processor cores with large data blocks via a high-bandwidth CXL switch. This class of memories meets different specifications than byte-addressable DRAM, which increasingly struggles to maintain the cost-per-bit-trend scaling line.

During the prestigious IEDM 2024 conference, NVIDIA presented its vision for the future AI accelerator design, which the company plans to chase after in future accelerator iterations. Currently, the limits of chip packaging and silicon innovation are being stretched. However, future AI accelerators might need some additional verticals to gain the required performance improvement. The proposed design at IEDM 24 introduces silicon photonics (SiPh) at the center stage. NVIDIA's architecture calls for 12 SiPh connections for intrachip and interchip connections, with three connections per GPU tile across four GPU tiles per tier. This marks a significant departure from traditional interconnect technologies, which in the past have been limited by the natural properties of copper.

Perhaps the most striking aspect of NVIDIA's vision is the introduction of so-called "GPU tiers"—a novel approach that appears to stack GPU components vertically. This is complemented by an advanced 3D stacked DRAM configuration featuring six memory units per tile, enabling fine-grained memory access and substantially improved bandwidth. This stacked DRAM would have a direct electrical connection to the GPU tiles, mimicking the AMD 3D V-Cache on a larger scale. However, the timeline for implementation reflects the significant technological hurdles that must be overcome. The scale-up of silicon photonics manufacturing presents a particular challenge, with NVIDIA requiring the capacity to produce over one million SiPh connections monthly to make the design commercially viable. NVIDIA has invested in Lightmatter, which builds photonic packages for scaling the compute, so some form of its technology could end up in future NVIDIA accelerators

Today at the IEEE International Electron Devices Meeting (IEDM) 2024, Intel Foundry unveiled new breakthroughs to help drive the semiconductor industry forward into the next decade and beyond. Intel Foundry showcased new material advancements that help improve interconnections within a chip, resulting in up to 25% capacitance by using subtractive ruthenium. Intel Foundry also was first to report a 100x throughput improvement using a heterogeneous integration solution for advanced packaging to enable ultra-fast chip-to-chip assembly. And to further drive gate-all-around (GAA) scaling, Intel Foundry demonstrated work with silicon RibbonFET CMOS and with gate oxide module for scaled 2D FETs for improved device performance.

"Intel Foundry continues to help define and shape the roadmap for the semiconductor industry. Our latest breakthroughs underscore the company's commitment to delivering cutting-edge technology developed in the U.S., positioning us well to help balance the global supply chain and restore domestic manufacturing and technology leadership with the support of the U.S. CHIPS Act," says Sanjay Natarajan, Intel senior vice president and general manager of Intel Foundry Technology Research.