SK hynix showcased its next-generation technologies and strengthened key partnerships at the TSMC 2024 Technology Symposium held in Santa Clara, California on April 24. At the event, the company displayed its industry-leading HBM AI memory solutions and highlighted its collaboration with TSMC involving the host's CoWoS advanced packaging technology.

TSMC, a global semiconductor foundry, invites its major partners to this annual conference in the first half of each year so they can share their new products and technologies. Attending the event under the slogan "Memory, the Power of AI," SK hynix received significant attention for presenting the industry's most powerful AI memory solution, HBM3E. The product has recently demonstrated industry-leading performance, achieving input/output (I/O) transfer speed of up to 10 gigabits per second (Gbps) in an AI system during a performance validation evaluation.

SMART Modular Technologies, Inc. ("SMART"), a division of SGH (Nasdaq: SGH) and a global leader in memory solutions, solid-state drives, and advanced memory, announces its new family of Add-In Cards (AICs) which implements the Compute Express Link (CXL) standard and also supports industry standard DDR5 DIMMs. These are the first in their class, high-density DIMM AICs to adopt the CXL protocol. The SMART 4-DIMM and 8-DIMM products enable server and data center architects to add up to 4 TB of memory in a familiar, easy-to-deploy form factor.

"The market for CXL memory components for data center applications is expected to grow rapidly. Initial production shipments are expected in late 2024 and will surpass the $2 billion mark by 2026. Ultimately, CXL attach rates in the server market will reach 30% including both expansion and pooling use cases," stated Mike Howard, vice president of DRAM and memory markets at TechInsights, an intelligence source to semiconductor innovation and related markets.

Samsung Electronics, a world leader in advanced semiconductor technology, unveiled the expansion of its Compute Express Link (CXL) memory module portfolio and showcased its latest HBM3E technology, reinforcing leadership in high-performance and high-capacity solutions for AI applications.

In a keynote address to a packed crowd at Santa Clara's Computer History Museum, Jin-Hyeok Choi, Corporate Executive Vice President, Device Solutions Research America - Memory at Samsung Electronics, along with SangJoon Hwang, Corporate Executive Vice President, Head of DRAM Product and Technology at Samsung Electronics, took the stage to introduce new memory solutions and discuss how Samsung is leading HBM and CXL innovations in the AI era. Joining Samsung on stage was Paul Turner, Vice President, Product Team, VCF Division at VMware by Broadcom and Gunnar Hellekson, Vice President and General Manager at Red Hat to discuss how their software solutions combined with Samsung's hardware technology is pushing the boundaries of memory innovation.

SK hynix is displaying its latest AI memory technologies at NVIDIA's GPU Technology Conference (GTC) 2024 held in San Jose from March 18-21. The annual AI developer conference is proceeding as an in-person event for the first time since the start of the pandemic, welcoming industry officials, tech decision makers, and business leaders. At the event, SK hynix is showcasing new memory solutions for AI and data centers alongside its established products.

Showcasing the Industry's Highest Standard of AI Memory
The AI revolution has continued to pick up pace as AI technologies spread their reach into various industries. In response, SK hynix is developing AI memory solutions capable of handling the vast amounts of data and processing power required by AI. At GTC 2024, the company is displaying some of these products, including its 12-layer HBM3E and Compute Express Link (CXL)1, under the slogan "Memory, The Power of AI". HBM3E, the fifth generation of HBM2, is the highest-specification DRAM for AI applications on the market. It offers the industry's highest capacity of 36 gigabytes (GB), a processing speed of 1.18 terabytes (TB) per second, and exceptional heat dissipation, making it particularly suitable for AI systems. On March 19, SK hynix announced it had become the first in the industry to mass-produce HBM3E.

SK hynix unveiled a new consumer product based on its latest solid-state drive (SSD), PCB01, which boasts industry-leading performance levels at GPU Technology Conference (GTC) 2024. Hosted by NVIDIA in San Jose, California from March 18-21, GTC is one of the world's leading conferences for AI developers. Applied to on-device AI PCs, PCB01 is a PCIe fifth-generation SSD which recently had its performance and reliability verified by a major global customer. After completing product development in the first half of 2024, SK hynix plans to launch two versions of PCB01 by the end of the year which target both major technology companies and general consumers.

Optimized for AI PCs, Capable of Loading LLMs Within One Second
Offering the industry's highest sequential read speed of 14 gigabytes per second (GB/s) and a sequential write speed of 12 GB/s, PCB01 doubles the speed specifications of its previous generation. This enables the loading of LLMs required for AI learning and inference in less than one second. To make on-device AIs operational, PC manufacturers create a structure that stores an LLM in the PC's internal storage and quickly transfers the data to DRAMs for AI tasks. In this process, the PCB01 inside the PC efficiently supports the loading of LLMs. SK hynix expects these characteristics of its latest SSD to greatly increase the speed and quality of on-device AIs.

MemVerge, a leader in AI-first Big Memory Software, has joined forces with Micron to unveil a groundbreaking solution that leverages intelligent tiering of CXL memory, boosting the performance of large language models (LLMs) by offloading from GPU HBM to CXL memory. This innovative collaboration is being showcased in Micron booth #1030 at GTC, where attendees can witness firsthand the transformative impact of tiered memory on AI workloads.

Charles Fan, CEO and Co-founder of MemVerge, emphasized the critical importance of overcoming the bottleneck of HBM capacity. "Scaling LLM performance cost-effectively means keeping the GPUs fed with data," stated Fan. "Our demo at GTC demonstrates that pools of tiered memory not only drive performance higher but also maximize the utilization of precious GPU resources."

Cadence's digital and custom/analog flows are certified on the Intel 18A process technology. Cadence design IP supports this node from Intel Foundry, and the corresponding process design kits (PDKs) are delivered to accelerate the development of a wide variety of low-power consumer, high-performance computing (HPC), AI and mobile computing designs. Customers can now begin using the production-ready Cadence design flows and design IP to achieve design goals and speed up time to market.

"Intel Foundry is very excited to expand our partnership with Cadence to enable key markets for the leading-edge Intel 18A process technology," said Rahul Goyal, Vice President and General Manager, Product and Design Ecosystem, Intel Foundry. "We will leverage Cadence's world-class portfolio of IP, AI design technologies, and advanced packaging solutions to enable high-volume, high-performance, and power-efficient SoCs in Intel Foundry's most advanced process technology. Cadence is an indispensable partner supporting our IDM2.0 strategy and the Intel Foundry ecosystem."

Samsung Electronics Co., Ltd a world leader in advanced memory technology, today announced that for the first time in the industry, it has successfully verified Compute Express Link (CXL) memory operations in a real user environment with open-source software provider Red Hat, leading the expansion of its CXL ecosystem. Due to the exponential growth of data throughput and memory requirements for emerging fields like generative AI, autonomous driving and in-memory databases (IMDBs), the demand for systems with greater memory bandwidth and capacity is also increasing. CXL is a unified interface standard that connects various processors, such as CPUs, GPUs and memory devices through a PCIe interface that can serve as a solution for limitations in existing systems in terms of speed, latency and expandability.

"Samsung has been working closely with a wide range of industry partners in areas from software, data centers and servers to chipset providers, and has been at the forefront of building up the CXL memory ecosystem," said Yongcheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "Our CXL partnership with Red Hat is an exemplary case of collaboration between advanced software and hardware, which will enrich and accelerate the CXL ecosystem as a whole."

Alphawave Semi (LSE: AWE), a global leader in high-speed connectivity for the world's technology infrastructure, today announced successful collaboration with Keysight Technologies, a market-leading design, emulation, and test solutions provider, demonstrating interoperability between Alphawave Semi's PCIe 6.0 64 GT/s Subsystem (PHY and Controller) Device and Keysight PCIe 6.0 64 GT/s Protocol Exerciser, negotiating a link to the maximum PCIe 6.0 data rate. Alphawave Semi, already on the PCI-SIG 5.0 Integrators list, is accelerating next-generation PCIe 6.0 Compliance Testing through this collaboration.

Alphawave Semi's leading-edge silicon implementation of the new PCIe 6.0 64 GT/s Flow Control Unit (FLIT)-based protocol enables higher data rates for hyperscale and data infrastructure applications. Keysight and Alphawave Semi achieved another milestone by successfully establishing a CXL 2.0 link setting the stage for future cache coherency in the datacenter.

Samsung Electronics, a world leader in advanced memory technology, today announced that for the first time in the industry, it has successfully verified Compute Express Link (CXL) memory operations in a real user environment with open-source software provider Red Hat, leading the expansion of its CXL ecosystem.

Due to the exponential growth of data throughput and memory requirements for emerging fields like generative AI, autonomous driving and in-memory databases (IMDBs), the demand for systems with greater memory bandwidth and capacity is also increasing. CXL is a unified interface standard that connects various processors, such as CPUs, GPUs and memory devices through a PCIe interface that can serve as a solution for limitations in existing systems in terms of speed, latency and expandability.

Today at the "AI Everywhere" event, Intel launched its 5th Gen Intel Xeon processors (code-named Emerald Rapids) that deliver increased performance per watt and lower total cost of ownership (TCO) across critical workloads for artificial intelligence, high performance computing (HPC), networking, storage, database and security. This launch marks the second Xeon family upgrade in less than a year, offering customers more compute and faster memory at the same power envelope as the previous generation. The processors are software- and platform-compatible with 4th Gen Intel Xeon processors, allowing customers to upgrade and maximize the longevity of infrastructure investments while reducing costs and carbon emissions.

"Designed for AI, our 5th Gen Intel Xeon processors provide greater performance to customers deploying AI capabilities across cloud, network and edge use cases. As a result of our long-standing work with customers, partners and the developer ecosystem, we're launching 5th Gen Intel Xeon on a proven foundation that will enable rapid adoption and scale at lower TCO." -Sandra Rivera, Intel executive vice president and general manager of Data Center and AI Group.

Thanks to the leaked slides obtained by @InstLatX64, we have more details and some performance estimates about Intel's upcoming 5th Generation Xeon "Emerald Rapids" CPUs, boasting a significant performance leap over its predecessors. Leading the Emerald Rapids family is the top-end SKU, the Xeon 8592+, which features 64 cores and 128 threads, backed by a massive 480 MB L3 cache pool. The upcoming lineup shifts from a 4-tile to a 2-tile design to minimize latency and improve performance. The design utilizes the P-Core architecture under the Raptor Cove ISA and promises up to 40% faster performance than the current 4th Generation "Sapphire Rapids" CPUs in AI applications utilizing Intel AMX engine. Each chiplet has 35 cores, three of which are disabled, and each tile has two DDR5-5600 MT/s memory controllers, which operate two memory channels each and translating that into eight-channel design. There are three PCIe controllers per die, making it six in total.

Newer protocols and AI accelerators also back the upcoming lineup. Now, the Emerald Rapids family supports the Compute Express Link (CXL) Types 1/2/3 in addition to up to 80 PCIe Gen 5 lanes and enhanced Intel Ultra Path Interconnect (UPI). There are four UPI controllers spread over two dies. Moreover, features like the four on-die Intel Accelerator Engines, optimized power mode, and up to 17% improvement in general-purpose workloads make it seem like a big step up from the current generation. Much of this technology is found on the existing Sapphire Rapids SKUs, with the new generation enhancing the AI processing capability further. You can see the die configuration below. The 5th Generation Emerald Rapids designs are supposed to be official on December 14th, just a few days away.

SK hynix presented its leading AI and high-performance computing (HPC) solutions at Supercomputing 2023 (SC23) held in Denver, Colorado between November 12-17. Organized by the Association for Computing Machinery and IEEE Computer Society since 1988, the annual SC conference showcases the latest advancements in HPC, networking, storage, and data analysis. SK hynix marked its first appearance at the conference by introducing its groundbreaking memory solutions to the HPC community. During the six-day event, several SK hynix employees also made presentations revealing the impact of the company's memory solutions on AI and HPC.

Displaying Advanced HPC & AI Products
At SC23, SK hynix showcased its products tailored for AI and HPC to underline its leadership in the AI memory field. Among these next-generation products, HBM3E attracted attention as the HBM solution meets the industry's highest standards of speed, capacity, heat dissipation, and power efficiency. These capabilities make it particularly suitable for data-intensive AI server systems. HBM3E was presented alongside NVIDIA's H100, a high-performance GPU for AI that uses HBM3 for its memory.

MSI, a leading global server provider, is showcasing its latest GPU and CXL memory expansion servers powered by AMD EPYC processors and 4th Gen Intel Xeon Scalable processors, which are optimized for enterprises, organizations and data centers, at SC23, booth #1592 in the Colorado Convention Center in Denver from November 13 to 16.

"The exponential growth of human- and machine-generated data demands increased data center compute performance. To address this demand, liquid cooling has emerged as a key trend, said Danny Hsu, General Manager of Enterprise Platform Solutions. "MSI's server platforms offer a well-balanced hardware foundation for modern data centers. These platforms can be tailored to specific workloads, optimizing performance and aligning with the liquid cooling trend."

SK hynix showcased its next-generation memory semiconductor technologies and solutions at the OCP Global Summit 2023 held in San Jose, California from October 17-19. The OCP Global Summit is an annual event hosted by the world's largest data center technology community, the Open Compute Project (OCP), where industry experts gather to share various technologies and visions. This year, SK hynix and its subsidiary Solidigm showcased advanced semiconductor memory products that will lead the AI era under the slogan "United Through Technology".

SK hynix presented a broad range of its solutions at the summit, including its leading HBM(HBM3/3E), CXL, and AiM products for generative AI. The company also unveiled some of the latest additions to its product portfolio including its DDR5 RDIMM, MCR DIMM, enterprise SSD (eSSD), and LPDDR CAMM devices. Visitors to the HBM exhibit could see HBM3, which is utilized in NVIDIA's H100, a high-performance GPU for AI, and also check out the next-generation HBM3E. Due to their low-power consumption and ultra-high-performance, these HBM solutions are more eco-friendly and are particularly suitable for power-hungry AI server systems.

Jung-Bae Lee, President and Head of Memory Business of Samsung Electronics, the world's largest NAND memory supplier, has noted in the blog post that Samsung plans to develop its 9th Generation V-NAND memory with over 300 layers, aiming for mass production in 2024. Samsung's V-NAND uses a double-stack structure and is expected to have more active layers than its competitors' 3D NAND memory, such as SK Hynix's forthcoming 321-layer memory. The increase in layers allows Samsung to enhance storage density and performance in its future 3D NAND devices, focusing on input/output (I/O) speed. While the specific performance details of Samsung's 9th Generation V-NAND remain undisclosed, the memory is expected to be used in next-generation PCIe SSDs with the PCIe 5.0 standard.

Jung-Bae Lee has noted: "New structural and material innovations will be critical in the upcoming era of sub-10-nanometer (nm) DRAM and 1,000-layer vertical V-NAND. As such, we are developing 3D stacked structures and new materials for DRAM while increasing layer count, decreasing height, and minimizing cell interference for V-NAND." The 9th installment of V-NAND, scheduled for 2024, is utilizing 11 nm-class DRAM. Additionally, the blog post reassures the commitment to CXL Memory Modules (CMM), which will enable the composable infrastructure of next-generation systems, especially with high-capacity SSDs powered by V-NAND.

Phison Electronics, a global leader in NAND controllers and storage solutions, announced today that the company has expanded its portfolio of PCIe 5.0 high-speed transmission solutions with PCIe 5.0, CXL 2.0 compatible redriver and retimer data signal conditioning IC products. Leveraging the company's deep expertise in PCIe engineering, Phison is the only signal conditioners provider that offers the widest portfolio of multi-channel PCIe 5.0 redriver and retimer solutions and PCIe 5.0 storage solutions designed specifically to meet the data infrastructure demands of artificial intelligence and machine learning (AI+ML), edge computing, high-performance computing, and other data-intensive, next-gen applications. At the 2023 Open Compute Project Global Summit, the Phison team is showcasing its expansive PCIe 5.0 portfolio, demonstrating the redriver and retimer technologies alongside other enterprise NAND flash, illustrating a holistic vision for a PCIe 5.0 data ecosystem to address the most demanding applications of the AI-everywhere era.

"Phison has focused industry-leading R&D efforts on developing in-house, chip-to-chip communication technologies since the introduction of the PCIe 3.0 protocol, with PCIe 4.0 and PCIe 5.0 solutions now in mass production, and PCIe 6.0 solutions now in the design phase," said Michael Wu, President & General Manager, Phison US. "Phison's accumulated experience in high-speed signaling enables our team to deliver retimer and redriver design solutions that are optimized for top signal integration, low power usage, and high temperature endurance, to deliver interface speeds for the most challenging compute environments."

Ever since the creation of A64FX for the Fugaku supercomputer, Fujitsu has been plotting the development of next-generation CPU design for accelerating AI and general-purpose HPC workloads in the data center. Codenamed Monaka, the CPU is the latest creation for TSMC's 2 nm semiconductor manufacturing node. Based on Armv9-A ISA, the CPU will feature up to 150 cores with Scalable Vector Extensions 2 (SVE2), so it can process a wide variety of vector data sets in parallel. Using a 3D chiplet design, the 150 cores will be split into different dies and placed alongside SRAM and I/O controller. The current width of the SVE2 implementation is unknown.

The CPU is designed to support DDR5 memory and PCIe 6.0 connection for attaching storage and other accelerators. To bring cache coherency among application-specific accelerators, CXL 3.0 is present as well. Interestingly, Monaka is planned to arrive in FY2027, which starts in 2026 on January 1st. The CPU will supposedly use air cooling, meaning the design aims for power efficiency. Additionally, it is essential to note that Monaka is not a processor that will power the post-Fugaku supercomputer. The post-Fugaku supercomputer will use post-Monaka design, likely iterating on the design principles that Monaka uses and refining them for the launch of the post-Fugaku supercomputer scheduled for 2030. Below are the slides from Fujitsu's presentation, in Japenese, which highlight the design goals of the CPU.

Intel Corporation today announced its intent to separate its Programmable Solutions Group (PSG) operations into a standalone business. This will give PSG the autonomy and flexibility it needs to fully accelerate its growth and more effectively compete in the FPGA industry, which serves a broad array of markets, including the data center, communications, industrial, automotive, aerospace and defense sectors. Intel also announced that Sandra Rivera, executive vice president at Intel, will assume leadership of PSG as chief executive officer; Shannon Poulin has been named chief operating officer.

Standalone operations for PSG are expected to begin Jan. 1, 2024, with ongoing support from Intel. Intel expects to report PSG as a separate business unit when it releases first-quarter 2024 financials. Over the next two to three years, Intel intends to conduct an IPO for PSG and may explore opportunities with private investors to accelerate the business's growth, with Intel retaining a majority stake.

Avicena, a privately held company headquartered in Sunnyvale, CA, is demonstrating its LightBundle multi-Tbps chip-to-chip interconnect technology at the European Conference for Optical Communications (ECOC) 2023 in Glasgow, Scotland (https://www.ecocexhibition.com/). Avicena's microLED-based LightBundle architecture breaks new ground by unlocking the performance of processors, memory and sensors, removing key bandwidth and proximity constraints while simultaneously offering class leading energy efficiency.

"As generative AI continues to evolve, the role of high bandwidth-density, low-power and low latency interconnects between xPUs and HBM modules cannot be overstated", says Chris Pfistner, VP Sales & Marketing of Avicena. "Avicena's innovative LightBundle interconnects have the potential to fundamentally change the way processors connect to each other and to memory because their inherent parallelism is well-matched to the internal wide and slow bus architecture within ICs. With a roadmap to multi-terabit per second capacity and sub-pJ/bit efficiency these interconnects are poised to enable the next era of AI innovation, paving the way for even more capable models and a wide range of AI applications that will shape the future."

SK hynix announced on September 22 that it showcased its latest memory technologies and products at Intel Innovation 2023 held September 19-20 in the western U.S. city of San Jose, California. Hosted by Intel since 2019, Intel Innovation is an annual IT exhibition which brings together the technology company's customers and partners to share the latest developments in the industry. At this year's event held at the San Jose McEnery Convention Center, SK hynix showcased its advanced semiconductor memory products which are essential in the generative AI era under the slogan "Pioneer Tomorrow With the Best."

Products that garnered the most interest were HBM3, which supports the high-speed performance of AI accelerators, and DDR5 RDIMM, a DRAM module for servers with 1bnm process technology. As one of SK hynix's core technologies, HBM3 has established the company as a trailblazer in AI memory. SK hynix plans to further strengthen its position in the market by mass-producing HBM3E (Extended) from 2024. Meanwhile, DDR5 RDIMM with 1bnm, or the 5th generation of the 10 nm process technology, also offers outstanding performance. In addition to supporting unprecedented transfer speeds of more than 6,400 megabits per second (Mbps), this low-power product helps customers simultaneously reduce costs and improve ESG performance.

AMD today rolled out the new compacted Socket SP6 server platform designed for smaller servers locally deployed at the edge by organizations. With CPU core-counts of up to 64-core/128-thread, these processors are based on the "Zen 4c" microarchitecture, which comes with identical IPC and ISA to "Zen 4," but with smaller L3 cache available per core. The EPYC 8004 series targets traditional data-centers located on-site for organizations. Even if the heavy-lifting of the IT for them is performed by remote data-centers or cloud providers, organizations still need smaller edge server deployments. The EPYC 8004 series caters to a different kind of servers than the ones the lower core-count models of EPYC 9004 "Genoa" do.

With the EPYC 8004 series, AMD is debuting a new smaller CPU socket called SP6. The socket measures 58.5 mm x 75.4 mm, compared to the 76.0 mm x 80.0 mm of Socket SP5 powering EPYC 9004 "Genoa" and EPYC 97x4 "Bergamo." Socket SP5 is an LGA with a pin count of 4,844, compared to SP5, which is LGA-6096. The first line of processors for this socket, the EPYC 8004 series, are codenamed "Siena." These are very much part of the 4th Gen EPYC series, a lineage it shares with "Genoa" for data-center servers, "Genoa-X" for compute servers, and "Bergamo" for high-density cloud.

To address customers' growing needs, Intel expanded its Intel Agilex FPGA portfolio and broadened its Programmable Solutions Group (PSG) offerings to handle the increased demand for customized workloads, including enhanced AI capabilities, and to provide lower total cost of ownership (TCO) and more complete solutions. These new products and technologies will be the focus of Intel's FPGA Technology Day (IFTD) on Sept. 18, where hardware engineers, software developers and system architects can interact with Intel and partner experts.

FPGAs play an important role in Intel's portfolio by offering flexible and customizable platform capabilities for demanding applications and workloads. Intel FPGAs solve customer challenges from cloud to edge with AI capabilities across silicon, IP and software. Intel's latest announcements illustrate how the company's increased investment in its FPGA portfolio is unfolding. So far in 2023, Intel has released 11 of 15 expected new products - more new product introductions than ever in Intel's FPGA business. As disclosed in its second quarter 2023 earnings call, Intel reported that its PSG business unit delivered 35% revenue growth year-over-year, marking the third consecutive quarter of record revenue.

Supermicro, Inc., a Total IT Solution Provider for Cloud, AI/ML, Storage, and 5G/Edge, is delivering a high-throughput, low latency E3.S storage solutions supporting the industry's first PCIe Gen 5 drives and CXL modules to meet the demands of large AI Training and HPC clusters, where massive amounts of unstructured data must be delivered to the GPUs and CPUs to achieve faster results.

Supermicro's Petascale systems are a new class of storage servers supporting the latest industry standard E3.S (7.5 mm) Gen 5 NVMe drives from leading storage vendors for up to 256 TB of high throughput, low latency storage in 1U or up to a half petabyte in 2U. Inside, Supermicro's innovative symmetrical architecture reduced latency by ensuring the shortest signal paths for data and maximized airflow over critical components, allowing them to run at optimal speeds. With these new systems, a standard rack can now hold over 20 Petabytes of capacity for high throughput NVMe-oF (NVMe over Fabrics) configurations, ensuring that GPUs remain saturated with data. Systems are available with either the 4th Gen Intel Xeon Scalable processors or 4th Gen AMD EPYC processors.

Lightelligence, the global leader in photonic computing and connectivity systems, today announced Photowave, the first optical communications hardware designed for PCIe and Compute Express Link (CXL) connectivity, unleashing next-generation workload efficiency.

Photowave, an Optical Networking (oNET) transceiver leveraging the significant latency and energy efficiency of photonics technology, empowers data center managers to scale resources within or across server racks. The first public demonstration of Photowave will be at Flash Memory Summit today through Thursday, August 10, in Santa Clara, Calif.