GlobalFoundries (Nasdaq: GFS) (GF) today announced plans to create a new center for advanced packaging and testing of U.S.-made essential chips within its New York manufacturing facility. Supported by investments from the State of New York and the U.S. Department of Commerce, the first-of-its-kind center aims to enable semiconductors to be securely manufactured, processed, packaged and tested entirely onshore in the United States to meet the growing demand for GF's silicon photonics and other essential chips needed for critical end markets including AI, automotive, aerospace and defense, and communications.

Growth in AI is driving the adoption of silicon photonics and 3D and heterogeneously integrated (HI) chips to meet power, bandwidth and density requirements in datacenters and edge devices. Silicon photonics chips are also positioned to address power and performance needs in automotive, communications, radar, and other critical infrastructure applications.

Ultra Accelerator Link Consortium (UALink) has announced the expansion of its Board of Directors with the election of Alibaba Cloud Computing Ltd., Apple Inc., and Synopsys Inc. The new Board members will leverage their industry knowledge to advance development and industry adoption of UALink - a high-speed, scale-up interconnect for next-generation AI cluster performance.

"Alibaba Cloud believes that driving AI computing accelerator scale-up interconnection technology by defining core needs and solutions from the perspective of cloud computing and applications has significant value in building the competitiveness of intelligent computing supernodes," said Qiang Liu, VP of Alibaba Cloud, GM of Alibaba Cloud Server Infrastructure. "The UALink consortium, as a leader in the interconnect field of AI accelerators, has brought together key members from the AI infrastructure industry to work together to define interconnect protocol which is natively designed for AI accelerators, driving innovation in AI infrastructure. This will strongly promote the innovation of AI infrastructure and improve the execution efficiency of AI workloads, contributing to the establishment of an open and innovative industry ecosystem."

Halo 5: Guardians has remained exclusive to Microsoft Xbox gaming systems since launching back in October 2015, during the Xbox One era—an approaching ten-year anniversary has fans wishing for a possible port over to PC platforms. Hopes were elevated late last week due to an intriguing image popping up on Lenovo's Legion Go S product page. A certain promo image depicts screen content featuring a Steam menu session with an entry for Halo 5. This discovery (courtesy of Rebs Gaming) prompted much online chatter about the possible PC port being unveiled at the next Xbox Developer_Direct—scheduled for broadcast on January 23.

343 Industries (now Halo Studios) has ported five titles over to PC—neatly compiled in the form of Halo: Master Chief Collection—but the absence of Guardians has been a lingering sore spot. Over the past weekend, Lenovo senior community manager—Ben Green—dampened expectations. In a curt response to a query (on social media) regarding the alleged leak, he shot down the notion that Lenovo had inadvertently leaked the existence of a Halo 5: Guardians PC port. It is possible that Lenovo's marketing department had mocked-up the image in error. Microsoft's gaming division has committed to spreading its most visible IPs to "rival" platforms, so the upcoming Xbox Developer_Direct could harbor a surprise or two.

JEDEC Solid State Technology Association, the global leader in the development of standards for the microelectronics industry, today announced the publication of JESD220G: Universal Flash Storage 4.1. In addition, an update to the complementary JESD223F UFS Host Controller Interface (UFSHCI) version 4.1 standard has also been published. Developed for mobile applications and computing systems requiring high performance with low power consumption, UFS 4.1 offers faster data access and improved performance over the earlier version of the standard while maintaining hardware compatibility to UFS 4.0. Both standards are available for download from the JEDEC website.

"JEDEC members are continually innovating to shape the standards that will drive the next generation of mobile devices and advanced applications, and the committee's dedication to ongoing improvements to the UFS series is paving the way for future innovation," said Mian Quddus, Chairman of the JEDEC Board of Directors and the JC-64 Committee for Embedded Memory Storage and Removable Memory Cards.

According to a recent investigative report, UK-based Imagination Technologies faces allegations of transferring sensitive GPU intellectual property to Chinese companies with potential military connections. The UK-China Transparency organization claims that following its 2020 acquisition by China-controlled investment firm Canyon Bridge, Imagination provided complete access to its GPU IP to Chinese entities with military connections. The report suggests this included sharing detailed architectural documentation typically reserved for premier clients like Apple. At the center of the controversy are Chinese firms Moore Threads and Biren Technology, which have emerged as significant players in China's AI and GPU sectors. The report indicates Moore Threads maintains connections with military GPU suppliers, while Biren Technology has received partial Russian investment.

The organization argues that Canyon Bridge, which has ties to the state-owned China Reform enterprise, helped these technological transfers to benefit China's military-industrial complex. Imagination Technologies has defended its actions, maintaining that all licensing agreements comply with industry standards. The allegations have sparked renewed debate about foreign ownership of strategic technology assets and the effectiveness of current export controls. When Canyon Bridge acquired Imagination in 2020, security experts raised concerns about potential military applications of the firm's technology. UKCT plans to release additional findings, including information from legal disputes involving Imagination's previous management. Rising concerns over technology transfers have prompted governments to reassess export controls and corporate oversight in the semiconductor industry, as nations struggle to balance international commerce with national security priorities. We are yet to see official government response to this situation.

Despite having a rather successful launch, Hi-Fi Rush was slated to be cancelled when Tango Gameworks, the studio behind the unorthodox rhythm game, was to be shut down by Microsoft in mid 2024. Shortly after that unfortunate turn of events, Tango was acquired by Krafton, the publisher behind PUBG, and Hi-Fi Rush was given a second wind. That acquisition also got fans hopeful for a sequel, and there were even rumors floating around that a sequel was in the works. Now, in a recent interview with IGN, a few Tango Gameworks and Hi-Fi Rush higher-ups—Hi-Fi Rush director, John Johanas, studio head and creative director, Colin Mack, and Hi-Fi Rush project manager, Kazuaki Egashira—confirmed that this is not necessarily the case.

While a sequel had been in development when Krafton acquired the studio, Mack says that they "are not yet at the stage where we can say specifically." That said, the studio is apparently strongly leaning towards a sequel. The justification for not yet committing to Hi-Fi Rush 2 is that the team has a fondness for the game, its characters, and its world, so they want to make a sequel that is respectful and cherishes the IP. The team also acknowledges that "it is important to try new things," and Johanas admits that Tango is considering multiple opportunities. Throughout the interview, there's an emphasis on keeping things fresh, with the team seemingly under the impression that revisiting Hi-Fi Rush in a sequel without changing it, it would become outdated. Whatever comes out of the Hi-Fi Rush universe, it's apparent that the developers working on it don't see a sequel as a cash cow. For fans of the game, there is at least hope, since Mack says that the studio hopes to hire around 50 new employees in January 2025, doubling its workforce and supposedly allowing for more freedom for creativity and experimentation.

VeriSilicon today announced the launch of its latest Vitality architecture Graphics Processing Unit (GPU) IP series, designed to deliver high-performance computing across a wide range of applications, including cloud gaming, AI PC, and both discrete and integrated graphics cards.

VeriSilicon's new generation Vitality GPU architecture delivers exceptional advancements in computational performance with scalability. It incorporates advanced features such as a configurable Tensor Core AI accelerator and a 32 MB to 64 MB Level 3 (L3) cache, offering both powerful processing power and superior energy efficiency. Additionally, the Vitality architecture supports up to 128 channels of cloud gaming per core, addressing the needs of high concurrency and high image quality cloud-based entertainment, while enabling large-scale desktop gaming and applications on Windows systems. With robust support for Microsoft DirectX 12 APIs and AI acceleration libraries, this architecture is ideally suited for a wide range of performance-intensive applications and complex computing workloads.

A high-stakes trial between technology giants Arm and Qualcomm has revealed deeper tensions in the semiconductor industry, as Arm seeks the destruction of chip designs from Qualcomm's $1.4 billion Nuvia acquisition. The case, being heard in Delaware federal court, centers on a licensing dispute that could impact the future of AI-powered Windows PCs. Arm CEO Rene Haas took the stand Monday, adding allegations that Qualcomm violated licensing agreements following its 2021 acquisition of chip startup Nuvia. The issue is whether Qualcomm should pay Nuvia's higher royalty rates for using Arm's intellectual property rather than its own lower rates. Internal documents revealed Nuvia's rates were "many multiples" higher than Qualcomm's, with the acquisition potentially reducing Arm's revenue by $50 million.

During cross-examination, Qualcomm's legal team challenged Arm's motives, suggesting the dispute is part of a broader strategy to confront a customer increasingly viewed as a competitor. When presented with documents outlining potential plans for Arm to design its own chips, Haas downplayed these ambitions, emphasizing that Arm has never entered chip manufacturing. Allegedly, Arm sent letters to Qualcomm's customers, including Samsung, warning about possible disruption if Nuvia's IP design before acquisition in 2021 must be destroyed. Haas defended these communications, citing frequent inquiries from industry partners.

Itch.io, the popular developer-friendly indie game marketplace, has revealed that the site has been taken down by a seemingly erroneous IP protection request by none other than Funko Pop, the pop culture collectible maker. According to Itch.io, the bizarre take-down happened as a result of Funko's automated IP protection provider, and it happened despite Itch.io taking action to prevent the take-down. The site has since been restored, with the offending page removed, but it took more than a little effort on the part of Itch.io's management. The whole situation seems to have been caused by poor communication on the part of Itch.io's domain registrar, Iwantmyname, since that's where the decision to remove the entire site, instead of just the offending page, seems to have originated.

According to posts on X by BrandShield, Funko's IP protection partner, the company's automated system flagged an apparent phishing attempt by an Itch.io subdomain, prompting BrandShield to request the removal of the offending subdomain, notifying both Itch.io and Iwantmyname. Itch.io says that it then removed the offending page and sent an e-mail confirming that it had taken the requisite action, but that e-mail apparently went unread, leaving Itch.io without a site for what appears to be around eight hours. Eventually, someone at Iwantmyname was able to address Itch.io's communications regarding the take-down, and the site was restored in the morning of Monday, December 9.

We have already confirmed that Intel is continuing the development of Arc gaming GPUs beyond the current Xe2 "Battlemage" series, with the new Xe3 "Celestial" architecture in the works. However, thanks to PCWorld's The Full Nerd podcast, Tom Petersen of Intel confirmed that the Xe3 IP has been finished, and the hardware teams are already working on the next Xe4 "Druid" GPU IP. "Our architects are way ahead of us, and they are already working on not the next thing but the next thing after the next thing," said Petersen, adding: "The way I would like to comment is our IP that's kind of called Xe3, which is the one after Xe2, that's pretty much baked, right. And so the software teams have a lot of work to do on Xe3. The hardware teams are off on the next thing, right. That's our cadence, that we need to keep going."

The base IP of next-generation Xe3 "Celestial" GPUs is done. That means the basic media engines, Xe cores, XMX matrix engines, ray tracing engines, and other parts of the gaming GPU are already designed and most likely awaiting trial fabrication. The software to support this Xe3 is also being developed while Intel's team is working on enabling more optimizations for the Xe2 "Battlemage" architecture, which we previewed recently. We assume that Intel's Xe GPU will now follow a stricter cadence of releases, with SKUs getting updated much faster, given that a lot is prepared for the future.

With the latest industry boom in AI, the demand for more compute power is greater than ever, and the recent industry forecast predicts that the global GPU market will exceed $98.5 billion in value by the year 2024. This staggering projection, outlined in the 2024 supply-side GPU market summary report by Jon Peddie Research (JPR), shows how far the GPU market has come. Once primarily associated with powering consumer gaming rigs with AMD or NVIDIA inside, GPUs have become a key part of our modern tech stack, worth almost $100 billion in 2024 alone. Nowadays, GPUs are found in many products, from smartphones and vehicles to internet-connected devices and data centers.

"Graphics processor units (GPUs) have become ubiquitous and can be found in almost every industrial, scientific, commercial, and consumer product made today," said Dr. Jon Peddie, founder of JPR. "Some market segments, like AI, have grabbed headlines because of their rapid growth and high average selling price (ASP), but they are low-volume compared to other market segments." The report also shows the wide range of companies that are actively participating in the GPU marketplace, including industry giants like AMD, NVIDIA, and Intel, as well as smaller players from China like Loongson Zhongke, Siroyw, and Lingjiu Micro. Besides the discrete GPU solutions, the GPU IP market is very competitive, and millions of chips are shipped with GPU IP every year. Some revenue estimates of Chinese companies are not public, but JPR is measuring it from the supply chain side, so these estimates are pretty plausible.

According to DigiTimes, NVIDIA is reportedly targeting the high-end segment for its first consumer CPU attempt. Slated to arrive in 2025, NVIDIA is partnering with MediaTek to break into the AI PC market, currently being popularized by Qualcomm, Intel, and AMD. With Microsoft and Qualcomm laying the foundation for Windows-on-Arm (WoA) development, NVIDIA plans to join and leverage its massive ecosystem of partners to design and deliver regular applications and games for its Arm-based processors. At the same time, NVIDIA is also scheduled to launch "Blackwell" GPUs for consumers, which could end up in these AI PCs with an Arm CPU at its core.

NVIDIA's partner, MediaTek, has recently launched a big core SoC for mobile called Dimensity 9400. NVIDIA could use something like that as a base for its SoC and add its Blackwell IP to the mix. This would be similar to what Apple is doing with its Apple Silicon and the recent M4 Max chip, which is apparently the fastest CPU in single-threaded and multithreaded workloads, as per recent Geekbench recordings. For NVIDIA, the company already has a team of CPU designers that delivered its Grace CPU to enterprise/server customers. Using off-the-shelf Arm Neoverse IP, the company's customers are acquiring systems with Grace CPUs as fast as they are produced. This puts a lot of hope into NVIDIA's upcoming AI PC, which could offer a selling point no other WoA device currently provides, and that is tried and tested gaming-grade GPU with AI accelerators.

Ultra Accelerator Link (UALink ) Consortium, led by Board Members from AMD, Amazon Web Services (AWS), Astera Labs, Cisco, Google, Hewlett Packard Enterprise (HPE), Intel, Meta and Microsoft, have announced the incorporation of the Consortium and are extending an invitation for membership to the community. The UALink Promoter Group was founded in May 2024 to define a high-speed, low-latency interconnect for scale-up communications between accelerators and switches in AI pods & clusters. "The UALink standard defines high-speed and low latency communication for scale-up AI systems in data centers"

According to Bloomberg, Arm Holding PLC, the holding company behind the Arm instruction set and Arm chip designs, just issued a 60-day notice period of license retirement to Qualcomm, its long-time partner. The UK-based ISA provider has notified Qualcomm that it will cancel the Arm ISA architectural license agreement after the contract-mandated 60-day notice. The issues between the two arose in 2022, just a year after Qualcomm acquired Nuvia and its IP. Arm filed a lawsuit claiming that the reason was "Qualcomm attempted to transfer Nuvia licenses without Arm's consent, which is a standard restriction under Arm's license agreements." To transfer Nuvia core licensing, Qualcomm would need to ask Arm first and create a new licensing deal.

The licensing reworking came just in time when Qualcomm experienced its biggest expansion. The new Snapdragon 8 Elite is being used in the mobile sector, the Snapdragon X Elite/Plus is being used in Copilot+ PCs, and the automotive sector is also getting the new Snapdragon Cockpit/Ride Elite chipsets. Most of that is centered around Nuvia Oryon core IP, a high-performance, low-power design. Arm's representatives declined to comment on this move for Bloomberg, while a Qualcomm spokesman noted that the British company was trying to "strong-arm a longtime partner."

In the past 13 years, global data production has surged, increasing an estimated 74 times. (1) Looking forward, McKinsey projects AI to spur 35% annual growth in enterprise SSD capacity demand, from 181 Exabytes (EB) in 2024 to 1,078EB in 2030. (2) To address this growing demand, ScaleFlux, a leader in data storage and memory technology, is announcing a significant expansion of its product portfolio. The company is introducing cutting-edge controllers for both NVMe SSDs and Compute Express Link (CXL) modules, reinforcing its leadership in innovative technology for the data pipeline. "With the release of three new ASIC controllers and key updates to its existing lineup, ScaleFlux continues to push the boundaries of SSD and memory performance, power efficiency, and data integrity," points out Hao Zhong, CEO and Co-Founder of the company.

Three New SoC Controllers to Transform Data Center Storage
ScaleFlux is proud to unveil three new SoC controllers designed to enhance data center, AI and enterprise infrastructure:

Synopsys, Inc. today announced the industry's first complete UCIe IP solution operating at up to 40 Gbps per pin to address the increased compute performance requirements of the world's fastest AI data centers. The UCIe interconnect, the de facto standard for die-to-die connectivity, is critical for high-bandwidth, low-latency die-to-die connectivity in multi-die packages, enabling more data to travel efficiently across heterogeneous and homogeneous dies, or chiplets, in today's AI data center systems.

Synopsys' 40G UCIe IP supports both organic substrate and high-density, advanced packaging technologies to give designers the flexibility to explore the packaging options that best fit their needs. The complete Synopsys 40G UCIe IP solution, including PHY, controller, and verification IP, is a key component of Synopsys' comprehensive and scalable multi-die solution for fast heterogeneous integration from early architecture exploration to manufacturing.

Xockets, inventor of Data Processing Unit (DPU) technology has launched a legal battle against NVIDIA and Microsoft. The lawsuit, filed in Texas, accuses the companies of forming an illegal cartel to avoid fair compensation for its patented DPU technology. Xockets claims that the Data Processing Unit technology its co-founder Parin Dalal invented in 2012 is fundamental to NVIDIA's GPU-enabled AI systems and Microsoft's AI platforms. The lawsuit alleges that NVIDIA has infringed on Xockets' patents since its 2020 acquisition of Mellanox, a deal NVIDIA's CEO Jensen Huang once called a "homerun." Xockets is seeking injunctions to halt the release of NVIDIA's new Blackwell GPU-enabled AI systems and Microsoft's use of these systems in their generative AI platforms.

The case touches on the bigger issues of intellectual property rights and the monopoly in the tech sector. Robert Cote, a Xockets board member, describes the suit as a fight against "Big Tech's predatory infringement playbook," accusing NVIDIA and Microsoft of making moves to devalue smaller companies' innovations. The AI revolution continues to transform the tech world, and this legal dispute may have a profound effect on the way intellectual property is valued and protected in the industry, possibly introducing new precedents for the relationship between tech giants and smaller innovators.

Demonstrating the depth and breadth of its technologies at Hot Chips 2024, Intel showcased advancements across AI use cases - from the data center, cloud and network to the edge and PC - while covering the industry's most advanced and first-ever fully integrated optical compute interconnect (OCI) chiplet for high-speed AI data processing. The company also unveiled new details about the Intel Xeon 6 SoC (code-named Granite Rapids-D), scheduled to launch during the first half of 2025.

"Across consumer and enterprise AI usages, Intel continuously delivers the platforms, systems and technologies necessary to redefine what's possible. As AI workloads intensify, Intel's broad industry experience enables us to understand what our customers need to drive innovation, creativity and ideal business outcomes. While more performant silicon and increased platform bandwidth are essential, Intel also knows that every workload has unique challenges: A system designed for the data center can no longer simply be repurposed for the edge. With proven expertise in systems architecture across the compute continuum, Intel is well-positioned to power the next generation of AI innovation." -Pere Monclus, chief technology officer, Network and Edge Group at Intel.

Texas Instruments (TI) (Nasdaq: TXN) and the U.S. Department of Commerce have signed a non-binding Preliminary Memorandum of Terms for up to $1.6 billion in proposed direct funding under the CHIPS and Science Act to support three 300 mm wafer fabs already under construction in Texas and Utah. In addition, TI expects to receive an estimated $6 billion to $8 billion from the U.S. Department of Treasury's Investment Tax Credit for qualified U.S. manufacturing investments. The proposed direct funding, coupled with the investment tax credit, would help TI provide a geopolitically dependable supply of essential analog and embedded processing semiconductors.

"The historic CHIPS Act is enabling more semiconductor manufacturing capacity in the U.S., making the semiconductor ecosystem stronger and more resilient," said Haviv Ilan, president and CEO of Texas Instruments. "Our investments further strengthen our competitive advantage in manufacturing and technology as we expand our 300 mm manufacturing operations in the U.S. With plans to grow our internal manufacturing to more than 95% by 2030, we're building geopolitically dependable, 300 mm capacity at scale to provide the analog and embedded processing chips our customers will need for years to come."

OPENEDGES Technology, Inc., the leading provider of memory subsystem intellectual property (IP), today announced the launch of the Universal Chiplet Interconnect Express (UCIe) Controller IP, named OUC. UCIe is an open industry standard for a die-to-die interconnect, and co-developed by industry giants including AMD, Arm, ASE Group, Google Cloud, Intel, Meta, Microsoft, Qualcomm, and Samsung. UCIe is becoming a new trend in the semiconductor industry due to its advantages, such as increased semiconductor circuit integration, reduced production costs, and higher yields. OPENEDGES is contributing to UCIe consortium as a contributing member.

The OUC, deriving its name from OPENEDGES UCIe Controller, is tailored for highly customizable, package-level integration, facilitating die-to-die interconnect and protocol connections. It creates an interoperable, multi-vendor ecosystem set to revolutionize chip integration methodologies across the industry.

Akeana, the company committed to driving dramatic change in semiconductor IP innovation and performance, has announced its official company launch approximately three years after its foundation, having raised over $100 million in capital, with support from A-list investors including Kleiner Perkins, Mayfield, and Fidelity. Today's launch marks the formal availability of the company's expansive line of IP solutions that are uniquely customizable for any workload or application.

Formed by the same team that designed Marvell's ThunderX2 server chips, Akeana offers a variety of IP solutions, including microcontrollers, Android clusters, AI vector cores and subsystems, and compute clusters for networking and data centers. Akeana moves the industry beyond the status quo of legacy vendors and architectures, like Arm, with equitable licensing options and processors that fill and exceed current performance gaps.

Intel today announced that its lead products on Intel 18A, Panther Lake (AI PC client processor) and Clearwater Forest (server processor), are out of the fab and have powered-on and booted operating systems. These milestones were achieved less than two quarters after tape-out, with both products on track to start production in 2025. The company also announced that the first external customer is expected to tape out on Intel 18A in the first half of next year.

"We are pioneering multiple systems foundry technologies for the AI era and delivering a full stack of innovation that's essential to the next generation of products for Intel and our foundry customers. We are encouraged by our progress and are working closely with customers to bring Intel 18A to market in 2025." -Kevin O'Buckley, Intel senior vice president and general manager of Foundry Services

According to The Information, OpenAI is reportedly in talks with Broadcom about developing a custom AI accelerator to power OpenAI's growing demand for high-performance solutions. Broadcom is a fabless chip designer known for a wide range of silicon solutions spanning from networking, PCIe, SSD controllers, and PHYs all the way up to custom ASICs. The latter part is what OpenAI wants to focus on, but all the aforementioned IP developed by Broadcom is of use in a data center. Suppose OpenAI decides to use Broadcom's solutions. In that case, the fabless silicon designer offers a complete vertical stack of products for inter-system communication using various protocols such as PCIe, system-to-system communication using Ethernet networking with Broadcom Tomahawk 6 and future revisions, alongside storage solutions and many other complimentary elements of a data center.

As a company skilled in making various IPs, it also makes ASIC solutions for other companies and has assisted Google in the making of its Tensor Processing Unit (TPU), which is now in its sixth generation. Google TPUs are massively successful as Google deploys millions of them and provides AI solutions to billions of users across the globe. Now, OpenAI wants to be part of the AI chip game, and Broadcom could come to the rescue with its already-established AI success and various other data center componentry to help make a custom AI accelerator to power OpenAI's infrastructure needed for the next generation of AI models. With each new AI model released by OpenAI, compute demand spikes by several orders of magnitude, and having an AI accelerator that exactly matches their need will help the company move faster and run even bigger AI models.

Qualitas Semiconductor Co., Ltd. has developed a new PCIe 6.0 PHY IP, marking a significant advance in computer interconnect technology. This new product, created using advanced 5 nm process technology is designed to meet the high-speed data transfer needs of the AI era. The Qualitas' PCIe PHY IP using 5 nm FinFet CMOS technology consists of hardmacro PMA and PCS compliant to PCIe Base 6.0 specification.

The PCIe 6.0 PHY IP can achieve transmission speeds up to 64GT/s per lane. When using all 16 lanes, it can transfer data at rates up to 256 GB/s. These speeds make it well-suited for data centers and self-driving car technologies, where rapid data processing is essential. Qualitas achieved this performance by implementing 100G PAM4 signaling technology. Highlighting the importance of the new IP, Qualitas CEO Dr. Duho Kim signaled the company's intent to continue pushing boundaries in semiconductor technology.

South Korean startup Panmnesia has unveiled an interesting solution to address the memory limitations of modern GPUs. The company has developed a low-latency Compute Express Link (CXL) IP that could help expand GPU memory with external add-in card. Current GPU-accelerated applications in AI and HPC are constrained by the set amount of memory built into GPUs. With data sizes growing by 3x yearly, GPU networks must keep getting larger just to fit the application in the local memory, benefiting latency and token generation. Panmnesia's proposed approach to fix this leverages the CXL protocol to expand GPU memory capacity using PCIe-connected DRAM or even SSDs. The company has overcome significant technical hurdles, including the absence of CXL logic fabric in GPUs and the limitations of existing unified virtual memory (UVM) systems.

At the heart of Panmnesia's solution is a CXL 3.1-compliant root complex with multiple root ports and a host bridge featuring a host-managed device memory (HDM) decoder. This sophisticated system effectively tricks the GPU's memory subsystem into treating PCIe-connected memory as native system memory. Extensive testing has demonstrated impressive results. Panmnesia's CXL solution, CXL-Opt, achieved two-digit nanosecond round-trip latency, significantly outperforming both UVM and earlier CXL prototypes. In GPU kernel execution tests, CXL-Opt showed execution times up to 3.22 times faster than UVM. Older CXL memory extenders recorded around 250 nanoseconds round trip latency, with CXL-Opt potentially achieving less than 80 nanoseconds. As with CXL, the problem is usually that the memory pools add up latency and performance degrades, while these CXL extenders tend to add to the cost model as well. However, the Panmnesia CXL-Opt could find a use case, and we are waiting to see if anyone adopts this in their infrastructure.Below are some benchmarks by Panmnesia, as well as the architecture of the CXL-Opt.