The 8th edition of the Samsung AI Forum was held on November 4th and 5th in Seoul, and among all the presentations and keynote speeches, one piece of information caught our attention. As reported by The Chosun Daily, Samsung is (again) turning its attention to Processing-in-Memory (PIM) technology, in what appears to be the company's latest attempt to keep up with its rival SK Hynix in this area. In 2021, Samsung introduced the world's first HBM-PIM, the chips showing impressive gains in performance (nearly double) while reducing energy consumption by almost 50% on average. PIM technology basically adds the processor functions necessary for computational tasks, reducing data transfer between the CPU and memory.

Now, the company hopes that PIM memory chips could replace HBM in the future, based on the advantages this next-generation memory technology possesses, mainly for artificial intelligence (AI) applications. "AI is transforming our lives at an unprecedented rate, and the question of how to use AI more responsibly is becoming increasingly important," said Samsung Electronics CEO Han Jong-hee in his opening remarks. "Samsung Electronics is committed to fostering a more efficient and sustainable AI ecosystem." During the event, Samsung also highlighted its partnership with AMD, which reportedly supplies AMD with its fifth-generation HBM, the HBM3E.

SK hynix unveiled an enhanced Accelerator-in-Memory based Accelerator (AiMX) card at the AI Hardware & Edge AI Summit 2024 held September 9-12 in San Jose, California. Organized annually by Kisaco Research, the summit brings together representatives from the AI and machine learning ecosystem to share industry breakthroughs and developments. This year's event focused on exploring cost and energy efficiency across the entire technology stack. Marking its fourth appearance at the summit, SK hynix highlighted how its AiM products can boost AI performance across data centers and edge devices.

Booth Highlights: Meet the Upgraded AiMX
In the AI era, high-performance memory products are vital for the smooth operation of LLMs. However, as these LLMs are trained on increasingly larger datasets and continue to expand, there is a growing need for more efficient solutions. SK hynix addresses this demand with its PIM product AiMX, an AI accelerator card that combines multiple GDDR6-AiMs to provide high bandwidth and outstanding energy efficiency. At the AI Hardware & Edge AI Summit 2024, SK hynix presented its updated 32 GB AiMX prototype which offers double the capacity of the original card featured at last year's event. To highlight the new AiMX's advanced processing capabilities in a multi-batch environment, SK hynix held a demonstration of the prototype card with the Llama 3 70B model, an open source LLM. In particular, the demonstration underlined AiMX's ability to serve as a highly effective attention accelerator in data centers.

SK hynix unveiled and demonstrated a prototype of AiMX (Accelerator-in-Memory based Accelerator), a generative AI accelerator card based on GDDR6-AiM, at the AI Hardware & Edge AI Summit 2023 held September 12-14 at the Santa Clara Marriott, California.Hosted annually by the UK marketing firm Kisaco Research, the AI Hardware & Edge AI Summit brings together global IT companies and high-profile startups to share their developments in artificial intelligence and machine learning. This is SK hynix's third time participating in the summit.

At the event, the company showcased the prototype of AiMX, an accelerator card that combines multiple GDDR6-AiMs to further enhance performance, along with the GDDR6-AIM itself under the slogan of "Boost Your AI: Discover the Power of PIM (Processing-In-Memory) with SK hynix's AiM (Accelerator in Memory)." As a low-power, high-speed memory solution capable of handling large amounts of data, AiMX is set to play a key role in the advancement of data-intensive generative AI systems. The performance of generative AI improves as it is trained on more data, highlighting the need for high-performance products which can be applied to an array of generative AI systems.

Samsung Electronics and Red Hat today announced a broad collaboration on software technologies for next-generation memory solutions. The partnership will focus on the development and validation of open source software for existing and emerging memory and storage products, including NVMe SSDs; CXL memory; computational memory/storage (HBM-PIM, Smart SSDs) and fabrics — in building an expansive ecosystem for closely integrated memory hardware and software. The exponential growth of data driven by AI, AR and the fast-approaching metaverse is bringing disruptive changes to memory designs, requiring more sophisticated software technologies that better link with the latest hardware advancements.

"Samsung and Red Hat will make a concerted effort to define and standardize memory software solutions that embrace evolving server and memory hardware, while building a more robust memory ecosystem," said Yongcheol Bae, Executive Vice President and Head of the Memory Application Engineering Team at Samsung Electronics. "We will invite partners from across the IT industry to join us in expanding the software-hardware memory ecosystem to create greater customer value."

SK hynix announced on February 16 that it has developed PIM (processing-in-memory), a next-generation memory chip with computing capabilities. It has been generally accepted that memory chips store data and CPU or GPU, like human brain, process data. SK hynix, following its challenge to such notion and efforts to pursue innovation in the next-generation smart memory, has found a breakthrough solution with the development of the latest technology.

SK hynix plans to showcase its PIM development at the world's most prestigious semiconductor conference, 2022 ISSCC, in San Francisco at the end of this month. The company expects continued efforts for innovation of this technology to bring the memory-centric computing, in which semiconductor memory plays a central role, a step closer to the reality in devices such as smartphones.