News Posts matching #DRAM

Return to Keyword Browsing

SK hynix Unveils Highest-Performing SSD for AI PCs at NVIDIA GTC 2024

SK hynix unveiled a new consumer product based on its latest solid-state drive (SSD), PCB01, which boasts industry-leading performance levels at GPU Technology Conference (GTC) 2024. Hosted by NVIDIA in San Jose, California from March 18-21, GTC is one of the world's leading conferences for AI developers. Applied to on-device AI PCs, PCB01 is a PCIe fifth-generation SSD which recently had its performance and reliability verified by a major global customer. After completing product development in the first half of 2024, SK hynix plans to launch two versions of PCB01 by the end of the year which target both major technology companies and general consumers.

Optimized for AI PCs, Capable of Loading LLMs Within One Second
Offering the industry's highest sequential read speed of 14 gigabytes per second (GB/s) and a sequential write speed of 12 GB/s, PCB01 doubles the speed specifications of its previous generation. This enables the loading of LLMs required for AI learning and inference in less than one second. To make on-device AIs operational, PC manufacturers create a structure that stores an LLM in the PC's internal storage and quickly transfers the data to DRAMs for AI tasks. In this process, the PCB01 inside the PC efficiently supports the loading of LLMs. SK hynix expects these characteristics of its latest SSD to greatly increase the speed and quality of on-device AIs.

Samsung Shows Off 32 Gbps GDDR7 Memory at GTC

Samsung Electronics showed off its latest graphics memory innovations at GTC, with an exhibit of its new 32 Gbps GDDR7 memory chip. The chip is designed to power the next generation of consumer and professional graphics cards, and some models of NVIDIA's GeForce RTX "Blackwell" generation are expected to implement GDDR7. The chip Samsung showed off at GTC is of the highly relevant 16 Gbit density (2 GB). This is important, as NVIDIA is rumored to keep graphics card memory sizes largely similar to where they currently are, while only focusing on increasing memory speeds.

The Samsung GDDR7 chip shown is capable of its 32 Gbps speed at a DRAM voltage of just 1.1 V, which beats the 1.2 V that's part of JEDEC's GDDR7 specification, which along with other power management innovations specific to Samsung, translates to a 20% improvement in energy efficiency. Although this chip is capable of 32 Gbps, NVIDIA isn't expected to give its first GeForce RTX "Blackwell" graphics cards that speed, and the first SKUs are expected to ship with 28 Gbps GDDR7 memory speeds, which means NVIDIA could run this Samsung chip at a slightly lower voltage, or with better timings. Samsung also made some innovations with the package substrate, which decreases thermal resistance by 70% compared to its GDDR6 chips. Both NVIDIA and AMD are expected to launch their first discrete GPUs implementing GDDR7, in the second half of 2024.

SK Hynix Begins Volume Production of Industry's First HBM3E

SK hynix Inc. announced today that it has begun volume production of HBM3E, the newest AI memory product with ultra-high performance, for supply to a customer from late March. The company made public its success with the HBM3E development just seven months ago. SK hynix being the first provider of HBM3E, a product with the best performing DRAM chips, extends its earlier success with HBM3. The company expects a successful volume production of HBM3E, along with its experiences also as the industry's first provider of HBM3, to help cement its leadership in the AI memory space.

In order to build a successful AI system that processes a huge amount of data quickly, a semiconductor package should be composed in a way that numerous AI processors and memories are multi-connected. Global big tech companies have been increasingly requiring stronger performance of AI semiconductor and SK hynix expects its HBM3E to be their optimal choice that meets such growing expectations.

2024 HBM Supply Bit Growth Estimated to Reach 260%, Making Up 14% of DRAM Industry

TrendForce reports that significant capital investments have occurred in the memory sector due to the high ASP and profitability of HBM. Senior Vice President Avril Wu notes that by the end of 2024, the DRAM industry is expected to allocate approximately 250K/m (14%) of total capacity to producing HBM TSV, with an estimated annual supply bit growth of around 260%. Additionally, HBM's revenue share within the DRAM industry—around 8.4% in 2023—is projected to increase to 20.1% by the end of 2024.

HBM supply tightens with order volumes rising continuously into 2024
Wu explains that in terms of production differences between HBM and DDR5, the die size of HBM is generally 35-45% larger than DDR5 of the same process and capacity (for example, 24Gb compared to 24Gb). The yield rate (including TSV packaging) for HBM is approximately 20-30% lower than that of DDR5, and the production cycle (including TSV) is 1.5 to 2 months longer than DDR5.

JEDEC Agrees to Relax HBM4 Package Thickness

JEDEC is currently presiding over standards for 6th generation high bandwidth memory (AKA HBM4)—the 12 and 16-layer DRAM designs are expected to reach mass production status in 2026. According to a ZDNET South Korea report, involved manufacturers are deliberating over HBM4 package thicknesses—allegedly, decision makers have settled on 775 micrometers (μm). This is thicker than the previous generation's measurement of 720 micrometers (μm). Samsung Electronics, SK Hynix and Micron are exploring "hybrid bonding," a new packaging technology—where onboard chips and wafers are linked directly to each other. Hybrid bonding is expected to be quite expensive to implement, so memory makers are carefully considering whether HBM4 warrants its usage.

ZDNET believes that JEDEC's agreement—settling on 775 micrometers (μm) for 12-layer and 16-layer stacked HBM4—could have: "a significant impact on the future packaging investment trends of major memory manufacturers. These companies have been preparing a new packaging technology, hybrid bonding, keeping in mind the possibility that the package thickness of HBM4 will be limited to 720 micrometers. However, if the package thickness is adjusted to 775 micrometers, 16-layer DRAM stacking HBM4 can be sufficiently implemented using existing bonding technology." A revised schedule could delay the rollout of hybrid bonding—perhaps pushed back to coincide with a launch of seventh generation HBM. The report posits that Samsung Electronics, SK Hynix and Micron memory engineers are about to focus on the upgrading of existing bonding technologies.

Global Top 10 Foundries Q4 Revenue Up 7.9%, Annual Total Hits US$111.54 Billion in 2023

The latest TrendForce report reveals a notable 7.9% jump in 4Q23 revenue for the world's top ten semiconductor foundries, reaching $30.49 billion. This growth is primarily driven by sustained demand for smartphone components, such as mid and low-end smartphone APs and peripheral PMICs. The launch season for Apple's latest devices also significantly contributed, fueling shipments for the A17 chipset and associated peripheral ICs, including OLED DDIs, CIS, and PMICs. TSMC's premium 3 nm process notably enhanced its revenue contribution, pushing its global market share past the 60% threshold this quarter.

TrendForce remarks that 2023 was a challenging year for foundries, marked by high inventory levels across the supply chain, a weak global economy, and a slow recovery in the Chinese market. These factors led to a downward cycle in the industry, with the top ten foundries experiencing a 13.6% annual drop as revenue reached just $111.54 billion. Nevertheless, 2024 promises a brighter outlook, with AI-driven demand expected to boost annual revenue by 12% to $125.24 billion. TSMC, benefiting from steady advanced process orders, is poised to far exceed the industry average in growth.

JEDEC Publishes GDDR7 Graphics Memory Standard

JEDEC Solid State Technology Association, the global leader in the development of standards for the microelectronics industry, is pleased to announce the publication of JESD239 Graphics Double Data Rate (GDDR7) SGRAM. This groundbreaking new memory standard is available for free download from the JEDEC website. JESD239 GDDR7 offers double the bandwidth over GDDR6, reaching up to 192 GB/s per device, and is poised to meet the escalating demand for more memory bandwidth in graphics, gaming, compute, networking and AI applications.

JESD239 GDDR7 is the first JEDEC standard DRAM to use the Pulse Amplitude Modulation (PAM) interface for high frequency operations. Its PAM3 interface improves the signal to noise ratio (SNR) for high frequency operation while enhancing energy efficiency. By using 3 levels (+1, 0, -1) to transmit 3 bits over 2-cycles versus the traditional NRZ (non-return-to-zero) interface transmitting 2 bits over 2-cycles, PAM3 offers higher data transmission rate per cycle resulting in improved performance.

V-COLOR Intros EXPO OC RDIMM Memory Octo-kits for AMD Threadripper 7000 WRX90 Workstations

V-COLOR today introduced a series of overclocking memory RDIMM kits for workstations powered by AMD Ryzen Threadripper 7000WX processors on the WRX90 platform that features 8-channel DDR5 memory. The kits include 8 RDIMMs, with densities ranging between 16 GB per RDIMM (128 GB per kit), to 96 GB per RDIMM (768 GB per kit); and comes in speeds ranging between DDR5-5600 and DDR5-7200. The best part? These modules feature AMD EXPO profiles, which should make enabling their advertised speeds as easy as a couple of clicks in the motherboard's UEFI setup program.

An EXPO profile not just applies the kit's memory speed, timings, and voltages, but also several sub-timings and settings that are specific to the AMD platform, which are not found on Intel. V-COLOR has tested its overclocking RDIMMs on popular AMD WRX90 chipset motherboards, namely the ASRock WRX90 WS EVO, ASUS PRO WS WRX90E-SAGE SE, and certain unreleased WRX90 workstation motherboards by Supermicro. Although the RDIMMs lack heatspreaders for the DRAM chips, V-COLOR is including what it calls "micro heatsinks" for the PMIC and RCDs. The RCD in particular is crucial to get Threadrippers to operate at speeds such as DDR5-7200. The kits should be available starting today, with all models available from mid-March. The company didn't reveal pricing.

Samsung Develops Industry-First 36GB HBM3E 12H DRAM

Samsung Electronics, a world leader in advanced memory technology, today announced that it has developed HBM3E 12H, the industry's first 12-stack HBM3E DRAM and the highest-capacity HBM product to date. Samsung's HBM3E 12H provides an all-time high bandwidth of up to 1,280 gigabytes per second (GB/s) and an industry-leading capacity of 36 gigabytes (GB). In comparison to the 8-stack HBM3 8H, both aspects have improved by more than 50%.

"The industry's AI service providers are increasingly requiring HBM with higher capacity, and our new HBM3E 12H product has been designed to answer that need," said Yongcheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "This new memory solution forms part of our drive toward developing core technologies for high-stack HBM and providing technological leadership for the high-capacity HBM market in the AI era."

Solid State Disks Introduces SCSIFlash-Fast Product Range

Solid State Disks Ltd. (SSDL), a leading manufacturer of solid-state-drives (SSDs) and a value-added reseller (VAR) of latest-technology Flash and DRAM solutions, has launched SCSIFlash-Fast, a swap-in upgrade/replacement for electromechanical hard disk drives (HDDs) that use the SCSI interface. Initially available with 68- and 80-pin connectors and write speeds of up to 80 MB/s, SCSIFlash-Fast uses proven SCSI drive architecture and industrial CFast or M.2 SSD memory (with storage capacities ranging from 2 GB to 1 TB). The drive features configurable hardware, allowing the OEMs of (or those responsible for maintaining) legacy systems to replace or upgrade obsolete HDDs that were made in the 1980s, 1990s and early 2000s, and improve system reliability and security.

James Hilken, SSDL's Sales & Marketing Director, says: "There are several computer-based systems in use within aerospace, defense, manufacturing, medical, telecommunications and other sectors that were designed decades ago and were fitted with then state-of-the-art SCSI hard disk drives. With their moving parts, these long-obsolete drives are increasingly failing. Our SCSIFlash-Fast drive is a highly reliable swap-in replacement for virtually any SCSI hard disk drive that's more than 20 years old." SCSIFlash-Fast is configured to order and can replicate the exact behavior of the SCSI HDD it replaces, meaning no modifications need to be made to the host system; which in many cases must not be modified (i.e. its functionality has been certified) or it is simply not cost-effective to do so. With SSDL's SCSIFlash-Fast, the SCSI version is set to that of the host system (SASI, SCSI-1, SCSI-2 or Ultra3) and the disk sector size is set to 256, 512, 768, 1024, 2048 or 4096. Other configurations can also be applied, including the preloading of data.

ZOTAC Updates its MEK Hero Line of Gaming Desktops with RTX 40 SUPER Graphics

ZOTAC USA Inc., a leading manufacturer of innovative gaming hardware solutions, is thrilled to announce the market launch of three new MEK HERO pre-built Gaming PCs: the MEK HERO A7647S, A7647STI, and A7748S built using the latest ZOTAC GAMING GeForce RTX 4070 SUPER Twin Edge OC, ZOTAC GAMING GeForce RTX 4070 Ti SUPER Trinity OC White Edition, and ZOTAC GAMING GeForce RTX 4080 SUPER Trinity Black Edition Graphic Cards. These are teamed up with AMD Ryzen 5 7600 and Ryzen 7 7700X processors, offering stunning 4K graphics for unprecedented gaming performance with AI-accelerated DLSS 3.5 and Real-Time Ray Tracing, bringing gamers an immersive gaming experience. This new series of MEK HERO offers 3 years of warranty coverage, specifically for the included ZOTAC GAMING GeForce RTX 40 SUPER Series Graphics Card component.

The MEK HERO A7647S, A7647STI, and A7748S Gaming PCs feature 16 GB DDR5 system memory and NVMe M.2 SSD storage, elevating PC gaming and the visual content creation process to new heights. Each MEK HERO Gaming PC undergoes meticulous assembly and hand-testing in the United States, ensuring the highest build quality. Crafted for optimal performance, durability, and an unmatched experience, MEK HERO Gaming PCs with GeForce RTX 40 SUPER Series Graphics Cards are tailored to meet the demands of hardcore gamers and content creators alike.

AI's Rocketing Demand to Drive Server DRAM—2024 Predictions Show a 17.3% Annual Increase in Content per Box, Outpacing Other Applications

In 2024, the tech industry remains steadfastly focused on AI, with the continued rollout of advanced AI chips leading to significant enhancements in processing speeds. TrendForce posits that this advancement is set to drive growth in both DRAM and NAND Flash across various AI applications, including smartphones, servers, and notebooks. The server sector is expected to see the most significant growth, with content per box for server DRAM projected to rise by 17.3% annually, while enterprise SSDs are forecast to increase by 13.2%. The market penetration rate for AI smartphones and AI PCs is expected to experience noticeable growth in 2025 and is anticipated to further drive the average content per box upward.

Looking first at smartphones, despite chipmakers focusing on improving processing performance, the absence of new AI functionalities has somewhat constrained the impact of AI. Memory prices plummeted in 2023 due to oversupply, making lower-priced options attractive and leading to a 17.5% increase in average DRAM capacity and a 19.2% increase in NAND Flash capacity per smartphone. However, with no new applications expected in 2024, the growth rate in content per box for both DRAM and NAND Flash in smartphones is set to slow down, estimated at 14.1% and 9.3%, respectively.

Samsung to Also Showcase 280-layer 3D QLC NAND Flash, 32 Gbit DDR5-8000 Memory Chips at IEEE-SSCC

In addition to the 37 Gbps GDDR7 memory, Samsung Electronics prepares to showcase several other memory innovations at the 2024 IEEE-SSCC as compiled by VideoCardz. To begin with, the company is showcasing a new 280-layer 3D QLC NAND flash memory in the 1 Tb density, enabling next generation of mainstream SSDs and smartphone storage. This chip offers an areal density of 28.5 Gb/mm², and a speed of 3.2 GB/s. To put this into perspective, the fastest 3D NAND flash types powering the current crop of flagship NVMe SSDs rely on 2.4 GB/s of I/O data rates.

Next up, is a new generation DDR5 memory chip offers data rates of DDR5-8000 with a density of 32 Gbit (4 GB). This chip uses a symmetric-mosaic DRAM cell architecture, and is built on a 5th generation 10 nm class foundry node Samsung optimized for DRAM products. What's impressive about this chip is that it will allow PC memory vendors to build 32 GB and 48 GB DIMMs in single-rank configuration with DDR5-8000 speeds; as well as 64 GB and 96 GB DIMMs in dual-rank configuration (impressive, provided your platform can play well with DDR5-8000 in dual-rank).

SK hynix Reports Financial Results for 2023, 4Q23

SK hynix Inc. announced today that it recorded an operating profit of 346 billion won in the fourth quarter of last year amid a recovery of the memory chip market, marking the first quarter of profit following four straight quarters of losses. The company posted revenues of 11.31 trillion won, operating profit of 346 billion won (operating profit margin at 3%), and net loss of 1.38 trillion won (net profit margin at negative 12%) for the three months ended December 31, 2023. (Based on K-IFRS)

SK hynix said that the overall memory market conditions improved in the last quarter of 2023 with demand for AI server and mobile applications increasing and average selling price (ASP) rising. "We recorded the first quarterly profit in a year following efforts to focus on profitability," it said. The financial results of the last quarter helped narrow the operating loss for the entire year to 7.73 trillion won (operating profit margin at negative 24%) and net loss to 9.14 trillion won (with net profit margin at negative 28%). The revenues were 32.77 trillion won.

Higher DRAM and NAND Prices this Year, if Suppliers can Control Output

TrendForce's latest analysis reveals that the downswing of DRAM contract prices, which had lasted for eight consecutive quarters since 4Q21, was finally reversed in 4Q23. Likewise, NAND Flash rebounded in 3Q23 after four quarters of decline. The persistence of this rally in memory prices during 2024 will largely hinge on suppliers' ongoing and effective control over their capacity utilization rates.

According to TrendForce Senior Research Vice President, Avril Wu, the first quarter of this year is already shaping up to be a season of growth, with TrendForce confirming its initial projections: a hike of around 13-18% QoQ for DRAM contract prices and a hike of 18-23% for NAND Flash contract prices. Despite a generally conservative outlook for overall market demand in 2Q24, suppliers in both DRAM and NAND Flash markets have begun raising their capacity utilization rates since the end of 4Q23. Furthermore, NAND Flash buyers are anticipated to complete their inventory restocking in advance in 1Q24. Due to the rise in capacity utilization rates and earlier restocking efforts, leading to a more moderated QoQ price increase of 3-8% for both DRAM and NAND Flash contract prices for 2Q24.

Worldwide Semiconductor Revenue Declined 11% in 2023, Intel Reclaims No. 1 Spot

Worldwide semiconductor revenue in 2023 totaled $533 billion, a decrease of 11.1% from 2022, according to preliminary results by Gartner, Inc.

"While the cyclicality in the semiconductor industry was present again in 2023, the market suffered a difficult year with memory revenue recording one of its worst declines in history," said Alan Priestley, VP Analyst at Gartner. "The underperforming market also negatively impacted several semiconductor vendors. Only 9 of the top 25 semiconductor vendors posted revenue growth in 2023, with 10 experiencing double-digit declines."

The combined semiconductor revenue of the top 25 semiconductor vendors declined 14.1% in 2023, accounting for 74.4% of the market, down from 77.2% in 2022.

SK Hynix Throws a Jab: CAMM is Coming to Desktop PCs

In a surprising turn of events, SK Hynix has hinted at the possibility of the Compression Attached Memory Module (CAMM) standard, initially designed for laptops, being introduced to desktop PCs. This revelation came from a comment made by an SK Hynix representative at the CES 2024 in Las Vegas for the Korean tech media ITSubIssub. According to the SK Hynix representative, the first implementation is underway, but there are no specific details. CAMM, an innovative memory standard developed by Dell in 2022, was certified to replace SO-DIMM as the official standard for laptop memory. However, the transition to desktop PCs could significantly disrupt the desktop memory market. The CAMM modules, unlike the vertical DRAM sticks currently in use, are horizontal and are screwed into a socket. This design change would necessitate a complete overhaul of the desktop motherboard layout.

The thin, flat design of the CAMM modules could also limit the number that can be installed on an ATX board. However, the desktop version of the standard CAMM2 was announced by JEDEC just a month ago. It is designed for DDR5 memory, but it is expected to become mainstream with the introduction of DDR6 around 2025. While CAMM allows for higher speeds and densities for mobile memory, its advantages for desktops over traditional memory sticks are yet to be fully understood. Although low-power CAMM modules could offer energy savings, this is typically more relevant for mobile devices than desktops. As we move towards DDR6 and DDR7, more information about CAMM for desktops will be needed to understand its potential benefits. JEDEC's official words on the new standard indicate that "DDR5 CAMM2s are intended for performance notebooks and mainstream desktops, while LPDDR5/5X CAMM2s target a broader range of notebooks and certain server market segments." So, we can expect to see CAMM2 in both desktops and some server applications.

Microsoft Sets 16 GB RAM as Minimum-Requirement for Copilot and Windows AI Features

Microsoft has reportedly set 16 GB as the minimum system requirement for AI PCs, a TrendForce market research report finds. To say that Microsoft has a pivotal role to play in PC hardware specs is an understatement. This year sees the introduction of the first "AI PCs," or PCs with on-device AI acceleration for several new features native to Windows 11 23H2, mainly Microsoft Copilot. From the looks of it, Copilot is receiving the highest corporate attention from Microsoft, as the company looks to integrate the AI chatbot that automates and generates work, into the mainstream PC. In fact, Microsoft is even pushing for a dedicated Copilot button on PC keyboards along the lines of the key that brings up the Start menu. The company's biggest move with Copilot will be the 2024 introduction of Copilot Pro, an AI assistant integrated with Office and 365, which the company plans to sell on a subscription basis alone.

Besides cloud-based acceleration, Microsoft's various AI features will rely on some basic hardware specs for local acceleration. One of them of course is the NPU, with Intel's AI Boost and AMD's Ryzen AI being introduced with their latest mobile processors. The other requirement will be memory. AI acceleration is a highly memory sensitive operation, and LLMs require a sizable amount of fast frequent-access memory. So Microsoft arrived at 16 GB as the bare minimum amount of memory for not just native acceleration, but also cloud-based Copilot AI features to work. This should see the notebooks of 2024 set 16 GB as their baseline memory specs; and for commercial notebooks to scale up to 32 GB or even 64 GB, depending on organizational requirements. The development bodes particularly well for the DRAM industry.

DRAM Contract Prices Projected to Increase 13-18% in 1Q24 as Price Surge Continues

TrendForce reports that the DRAM contract prices are estimated to increase by approximately 13-18% in 1Q24 with mobile DRAM leading the surge. It appears that due to the unclear demand outlook for the entire year of 2024, manufacturers believe that sustained production cuts are necessary to maintain the supply-demand balance in the memory industry.

PC DRAM: The market is buzzing with unfilled DDR5 orders, while savvy buyers brace for a continued surge in DDR4 prices, keeping procurement engines running. This trend, however, is shadowed by a gradual industry pivot toward DDR5, casting uncertainty over the expansion of DDR4 bit procurement volumes. Despite this, both DDR4 and DDR5 prices have yet to hit the target set by manufacturers, and buyers seem ready to ride the wave of price hikes into 1Q24. This sets the stage for an estimated 10-15% in PC DRAM contract prices, with DDR5 poised to take the lead over DDR4 in this pricing rally.

Report: Global Semiconductor Capacity Projected to Reach Record High 30 Million Wafers Per Month in 2024

Global semiconductor capacity is expected to increase 6.4% in 2024 to top the 30 million *wafers per month (wpm) mark for the first time after rising 5.5% to 29.6 wpm in 2023, SEMI announced today in its latest quarterly World Fab Forecast report.

The 2024 growth will be driven by capacity increases in leading-edge logic and foundry, applications including generative AI and high-performance computing (HPC), and the recovery in end-demand for chips. The capacity expansion slowed in 2023 due to softening semiconductor market demand and the resulting inventory correction.

Rambus Advances Data Center Server Performance with Industry-First Gen4 DDR5 RCD

Rambus Inc., a premier chip and silicon IP provider making data faster and safer, today announced the availability of its state-of-the-art Gen 4 DDR5 Registering Clock Driver (RCD) which began sampling to the major DDR5 memory module (RDIMM) manufacturers in the fourth quarter of 2023. The Rambus Gen 4 RCD boosts the data rate to 7200 MT/s, setting a new benchmark for performance and enabling a 50% increase in memory bandwidth over today's 4800 MT/s DDR5 module solutions. It supports the rapid pace of server main memory performance improvements to meet the demands of generative AI and other advanced data center workloads.

"With memory being an essential enabler of server performance, the need for greater memory bandwidth continues its meteoric rise driven by demanding workloads like generative AI," said Sean Fan, chief operating officer at Rambus. "The Rambus Gen 4 DDR5 RCD is the latest demonstration of our commitment to providing leadership products ahead of the market need to support our customers' current and planned server platforms."

Apple Wants to Store LLMs on Flash Memory to Bring AI to Smartphones and Laptops

Apple has been experimenting with Large Language Models (LLMs) that power most of today's AI applications. The company wants these LLMs to serve the users best and deliver them efficiently, which is a difficult task as they require a lot of resources, including compute and memory. Traditionally, LLMs have required AI accelerators in combination with large DRAM capacity to store model weights. However, Apple has published a paper that aims to bring LLMs to devices with limited memory capacity. By storing LLMs on NAND flash memory (regular storage), the method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding optimization in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Instead of storing the model weights on DRAM, Apple wants to utilize flash memory to store weights and only pull them on-demand to DRAM once it is needed.

Two principal techniques are introduced within this flash memory-informed framework: "windowing" and "row-column bundling." These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to native loading approaches on CPU and GPU, respectively. Integrating sparsity awareness, context-adaptive loading, and a hardware-oriented design pave the way for practical inference of LLMs on devices with limited memory, such as SoCs with 8/16/32 GB of available DRAM. Especially with DRAM prices outweighing NAND Flash, setups such as smartphone configurations could easily store and inference LLMs with multi-billion parameters, even if the DRAM available isn't sufficient. For a more technical deep dive, read the paper on arXiv here.

Phison Predicts 2024: Security is Paramount, PCIe 5.0 NAND Flash Infrastructure Imminent as AI Requires More Balanced AI Data Ecosystem

Phison Electronics Corp., a global leader in NAND flash controller and storage solutions, today announced the company's predictions for 2024 trends in NAND flash infrastructure deployment. The company predicts that rapid proliferation of artificial intelligence (AI) technologies will continue apace, with PCIe 5.0-based infrastructure providing high-performance, sustainable support for AI workload consistency as adoption rapidly expands. PCIe 5.0 NAND flash solutions will be at the core of a well-balanced hardware ecosystem, with private AI deployments such as on-premise large language models (LLMs) driving significant growth in both everyday AI and the infrastructure required to support it.

"We are moving past initial excitement over AI toward wider everyday deployment of the technology. In these configurations, high-quality AI output must be achieved by infrastructure designed to be secure, while also being affordable. The organizations that leverage AI to boost productivity will be incredibly successful," said Sebastien Jean, CTO, Phison US. "Building on the widespread proliferation of AI applications, infrastructure providers will be responsible for making certain that AI models do not run up against the limitations of memory - and NAND flash will become central to how we configure data center architectures to support today's developing AI market while laying the foundation for success in our fast-evolving digital future."

Mobile DRAM and eMMC/UFS Prices to Surge 18-23% in 1Q24 as Smartphone Brands Continue Stockpiling

TrendForce predicts a significant rise in mobile DRAM and NAND Flash (eMMC/UFS) prices for the first quarter of 2024, with an expected seasonal increase of 18-23%. This surge could be further amplified in a market dominated by a few major players or if brand clients resort to panic buying under pressure.

Observations for 1Q24 indicate steady production planning by Chinese smartphone OEMs. A clear rise in memory prices is driving buyers to actively increase their purchasing efforts as they aim to establish secure and competitively priced inventory levels.

JEDEC Publishes New CAMM2 Memory Module Standard

JEDEC Solid State Technology Association, the global leader in the development of standards for the microelectronics industry, today announced the publication of JESD318: Compression Attached Memory Module (CAMM2) Common Standard. This groundbreaking standard defines the electrical and mechanical requirements for both Double Data Rate, Synchronous DRAM Compression-Attached Memory Modules (DDR5 SDRAM CAMM2s) and Low Power Double Data Rate, Synchronous DRAM Compression-Attached Memory Modules (LPDDR5/5X SDRAM CAMM2s) in a single, comprehensive document. JESD318 CAMM2 is available for download from the JEDEC website.

DDR5 and LPDDR5/5X CAMM2s cater to distinct use cases. DDR5 CAMM2s are intended for performance notebooks and mainstream desktops, while LPDDR5/5X CAMM2s target a broader range of notebooks and certain server market segments.
Return to Keyword Browsing
Nov 23rd, 2024 06:06 EST change timezone

New Forum Posts

Popular Reviews

Controversial News Posts