NVIDIA's intrepid CEO, Jensen Huang, has spent a fair chunk of January travelling around China—news outlets believe that Team Green's leader has conducted business meetings with very important clients in the region. Insiders proposed that his low-profile business trip included visits to NVIDIA operations in Shenzhen, Shanghai and Beijing. The latest updates allege that a stopover in Taiwan was also planned, following the conclusion of Mainland activities. Photos from an NVIDIA Chinese new year celebratory event have been spreading across the internet lately—many were surprised to see Huang appear on-stage in Shanghai and quickly dispense with his trademark black leather jacket. He swapped into a colorful "Year of the Wood Dragon" sleeveless shirt for a traditional dance routine.

It was not all fun and games during Huang's first trip to China in four years—inside sources have informed the Wall Street Journey about growing unrest within the nation's top ranked Cloud AI tech firms. Anonymous informants allege that leadership, at Alibaba Group and Tencent, are not happy with NVIDIA's selection of compromised enterprise GPUs—it is posited that NVIDIA's President has spent time convincing key clients to not adopt natively-developed solutions (unaffected by US Sanctions). The short hop over to Taiwan is reported not to be for R&R purposes—insiders had Huang's visiting key supply partners; TSMC and Wistron. Industry experts think that these meetings are linked to NVIDIA's upcoming "Blackwell" B100 AI GPU, and "supercharged" H200 "Hopper" accelerator. It is too early for the rumor mill to start speculation about nerfed versions of NVIDIA's 2024 enterprise products reaching Chinese shores, but Jensen Huang is seemingly ready to hold diplomatic talks with all sides.

LaminiAI appears to be one of AMD's first customers to receive a bulk order of Instinct MI300X GPUs—late last week, Sharon Zhou (CEO and co-founder) posted about the "next batch of LaminiAI LLM Pods" up and running with Team Red's cutting-edge CDNA 3 series accelerators inside. Her short post on social media stated: "rocm-smi...like freshly baked bread, 8x MI300X is online—if you're building on open LLMs and you're blocked on compute, lmk. Everyone should have access to this wizard technology called LLMs."

An attached screenshot of a ROCm System Management Interface (ROCm SMI) session showcases an individual Pod configuration sporting eight Instinct MI300X GPUs. According to official blog entries, LaminiAI has utilized bog-standard MI300 accelerators since 2023, so it is not surprising to see their partnership continue to grow with AMD. Industry predictions have the Instinct MI300X and MI300A models placed as great alternatives to NVIDIA's dominant H100 "Hopper" series—AMD stock is climbing due to encouraging financial analyst estimations.

According to the recent listing on Goofish, discovered by VideoCardz, Chinese companies have begun selling mobile versions of NVIDIA's latest RTX 40-series GPUs as desktop graphics cards. Initially designed for gaming laptops, the GeForce RTX 4080M and RTX 4090M are now being marketed in China as more affordable alternatives to their official desktop counterparts. This development is no surprise to industry observers who recall similar adaptations with the RTX 20 and 30 series. These companies are leveraging the lower cost of mobile GPUs, combined with budget cooling solutions and simpler PCB designs, to offer more affordable desktop GPU options. The mobile GPUs, which are capped at a power consumption of 175 Watts, are being repurposed without official sanction, with NVIDIA seemingly disregarding this practice. Despite the lack of official endorsement, these modified GPUs are finding their way into the market, providing gamers a cost-effective alternative to the more expensive desktop versions.

While not officially supported by NVIDIA, these cards utilize the mobile GPU dies paired with custom cooling solutions and PCBs to work in desktop PCs. According to reports, the RTX 4080M desktop variant offers 7424 CUDA cores and 12 GB GDDR6 memory, representing a 24% reduction in cores and 4 GB less memory versus the desktop RTX 4080. The desktop RTX 4090M is even more cut-down, with 9728 cores and 16 GB memory—a 40% drop in cores and 8 GB less memory than the flagship RTX 4090 desktop card. Pricing falls between $420 and $560 for the RTX 4080M and exceeds that of even the desktop RTX 4090 for the 4090M variant. Performance and longevity still need to be determined for these unofficial cards. While they present a cheaper RTX 40-series option for Chinese gamers, the reduced specifications come with tradeoffs. Still, their availability indicates the ongoing demand for next-gen GPUs and the lengths some vendors go to to meet that demand.

ASRock showcased customized Radeon RX 7600 XT 16 GB GPU offerings at CES 2024—only a couple days after AMD's official unveiling of its expanded lower mid-range RDNA 3 line. ASRock was among a select few Team Red board partners with finalized units (based on Navi 33 XT) on display—it seems that the Taiwanese manufacturer is preparing for a retail launch of its Radeon RX 7600 XT Steel Legend 16 GB OC and Challenger 16 GB OC graphics card models. ASRock's website has been updated with product pages for the latest Radeon RX 7000-series entries, but press material for an imminent product launch has not been published (at the time of writing).

ASRock's mid-tier triple-fan Steel Legend and entry-level dual-fan Challenger designs are a familiar sight across the company's Radeon RX 7000 and 6000 product lines—last September, customized Radeon RX 7800 XT and Radeon RX 7700 XT models were unveiled as sporting these shrouds, along with higher-end Phantom Gaming OC options. A slightly overclocked Radeon RX 7600 XT GPU is not expected to be a heat producing monster, so expensive cooling solutions are not a necessity for a cost-conscious audience—likely targeting a decent level of 1080p gaming performance. The ASRock Radeon RX 7600 XT Challenger 16 GB OC model is expected to launch at an MSRP of $329 (AMD's official guide SEP), while the fancier Steel Legend OC is believed to be only marginally more expensive.

Mark Zuckerberg has shared some interesting insights about Meta's AI infrastructure buildout, which is on track to include an astonishing number of NVIDIA H100 Tensor GPUs. In the post on Instagram, Meta's CEO has noted the following: "We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs." That means that the company will enhance its AI infrastructure with 350,000 H100 GPUs on top of the existing GPUs, which is equivalent to 250,000 H100 in terms of computing power, for a total of 600,000 H100-equivalent GPUs.

The raw number of GPUs installed comes at a steep price. With the average selling price of H100 GPU nearing 30,000 US dollars, Meta's investment will settle the company back around $10.5 billion. Other GPUs should be in the infrastructure, but most will comprise the NVIDIA Hopper family. Additionally, Meta is currently training the LLama 3 AI model, which will be much more capable than the existing LLama 2 family and will include better reasoning, coding, and math-solving capabilities. These models will be open-source. Later down the pipeline, as the artificial general intelligence (AGI) comes into play, Zuckerberg has noted that "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit." So, expect to see these models in the GitHub repositories in the future.

GALAX is known to go overboard with its top flight graphic card models—the introduction of GeForce RTX 40 SUPER models has further lengthened the manufacturer's naming conventions. A Hall of Fame (HOF) OC LAB Master Edition card based on NVIDIA's freshly launched GeForce RTX 4070 SUPER GPU has been introduced via a Galaxy BBS blog post. As befits such a fancily named card, GALAX has rolled out the red carpet with a very special cooling solution that is designed to temper a substantial (211 MHz) overclock over Team Green's reference settings. Its 2685 MHz spec sits atop the customized GeForce RTX 4070 SUPER GPU pile—VideoCardz notes that another high-end option—GIGABYTE's AORUS RTX 4070 SUPER MASTER model—trails by a 30 MHz margin.

The GALAX GeForce RTX 4070 SUPER Hall of Fame OC LAB Master Edition graphics card sports an almost all-white design, even its PCB is outfitted in a pale hue. This Ada Lovelace AD104-350-A1-driven flagship is specced with a 250 W TGP as standard, user-adjustable up to 320 W. A three-pin to sixteen-pin adapter is supplied by default; ensuring that more than enough juice is supplied. GALAX states that the card features a 12+3 phase design, coupled with a power section is controlled by an XDPE10281 PWM. The Chinese manufacturer hints that more white Master Edition SUPER models are incoming (see below). We hope to see further announcements, and full product pages uploaded to their web site(s), but GALAX is unlikely to sell these top flight cards outside of their native market.

At CES, AMD didn't give away too many technical details of its upcoming Ryzen 8000G-series APUs, but details are starting to trickle out and it's not all good news. As has been known for some time, AMD is using two different chips to make the Ryzen 8000G APUs and they're known as the Phoenix 1 and Phoenix 2, where the Phoenix 2 parts feature Zen 4c cores, which are not present in the Phoenix 1 APUs. This in and of itself shouldn't be a huge issue, although the Zen 4c CPU cores can be slightly slower in some tasks based on testing of AMD's EPYC server parts.

However, PCGamesN noticed that Gigabyte has posted the full specs for the B650E Aorus Elite X AX Ice motherboard and it looks like there's a much bigger difference between the Phoenix 1 and Phoenix 2 based APUs. Namely, the Phoenix 2 APUs have fewer PCIe lanes and as such are limited to two PCIe 4.0 lanes for the secondary NVMe slot. As if this wasn't bad enough, the Phoenix 2 APUs only have four PCIe 4.0 lanes for add-in GPUs, whereas the Phoenix 1 APUs have eight. This is very likely to lead to reduced performance if a higher-end GPU is used with such an APU. Note that this will vary depending on the motherboard design, but many B650/B650E boards feature a similar design with regards to the PCIe lanes coming from the CPU socket. Luckily, it's easy to avoid this issue, as the Ryzen 5 8600G and the Ryzen 7 8700G are both Phoenix 1 designs, whereas the Ryzen 5 8500G is the only Phoenix 2 design available in retail, as the Ryzen 3 8300G is an OEM only part.

AEWIN is glad to announce our latest High-Performance Network Appliance powered by Intel latest 5th Gen Xeon Scalable Processors, SCB-1942 Series. It is a series of flagship products powered by dual Intel Emerald Rapids CPUs, having up to 128 CPU cores (64 cores per CPU) for the extreme computing power pursued in the market. SCB-1942 series has multiple SKU with various PCIe slots options for great expandability to fulfill customer's solutions.

The SCB-1942A is a 2U, 2-socket network computing platform having 16x memory socket of DDR5 up to 5600 MHz, and 8x PCIe 5.0 expansion slots for AEWIN wide coverage NIC cards with 1G/10/25/40/100G copper/fiber interfaces or other Accelerators & NVMe SSDs for flexible functionality enhancement. The SCB-1942A provides the flexibility to change the 2x PCIe slots to 1x PCIe x16 slot for standard PCIe form factor which can install off-the-shelf add-on card for additional function required. It can support 400G NIC card installed such as Mellanox PCIe 5.0 NIC. In addition, the SCB-1942 series support 10 SATA which make it also suitable for various kinds of storage applications.

At CES 2024 in Las Vegas, case maker InWin displayed several innovative new models for custom PC builds. The company continues its reputation for eye-catching designs with the debut of the adjustable flatpack POC One case and highly modular Mini Mod-II and Mod-III chassis. The POC One features interlocking panels for easy snap-together assembly, an advancement over the original April-launched POC line. It includes repositionable top or side handles for portable or horizontal setups. Using a mix of aluminium, other metals, and acrylic, the POC One initially comes in black/orange or blue/silver styles. Despite its minimalist flatpack form, it fits full-size components like 335 mm long triple-slot GPUs, 140 mm tall CPU coolers, and standard ATX power supplies. Three PCIe slots and an included riser allow vertical graphics card mounting.

Intel released the latest version of its Arc GPU Graphics drivers. Version 101.5122 WHQL comes with support for the 14th Gen Core HX and Desktop 65 W series processors with their Intel UHD 770/730 series integrated graphics based on the Xe-LP architecture. The drivers also add optimization for "Prince of Persia: The Lost Crown." The company hasn't fixed any issues with this particular driver release, but identified a handful new issues to fix with future releases. Grab the drivers from the link below.

DOWNLOAD: Intel Arc GPU Graphics Drivers 101.5122 WHQL

At CES 2024, Razer has updated its Blade laptop family spanning across various sizes, and even got a chance to present a "world's first" feature in a gaming product. The star of the Razer booth is the company's flagship Blade 16 laptop, which now supports 240 Hz refresh rate in its 16-inch OLED display format. Being the first to get there, Razer offers a high refresh rate at 2560 x 1600 QHD+ resolution. In addition to 0.2 ms response time and DCI-P3 100% color gamut, the display had VESA ClearMR 11000 and DisplayHDR True Black 500 certifications. At the center of the laptop is the 14th Gen Intel Core i9-14900HX processor, paired with up to NVIDIA GeForce RTX 4090 GPU with 175 Watt TGP. Pricing starts at $2999 for lower-end configurations and is available now.

Acer today announced the Acer Nitro 17 (AN17-72) gaming laptop, unlocking improved performance, immersive experiences, plus essential features for playing and multi-tasking on the go. The device is powered by the latest Intel Core 14th gen processors, featuring Intel's Performance Hybrid Architecture with improved power and core frequencies to manage workloads efficiently. At the same time, NVIDIA GeForce RTX 40 Series GPUs with DLSS 3.5 technology. NVIDIA RTX Laptop GPUs are packed with specialized AI Tensor Cores enabling unmatched AI performance in creative apps, ultra-efficient productivity, blistering fast gaming, and more. Built with Microsoft Copilot and the next wave of computing, the Nitro laptop provides access to Copilot in Windows via a dedicated Copilot key, making it even easier to harness the power of AI to assist with productivity and creativity tasks.

Users can feast their eyes on immersive visuals and pristine colors on the device's 17-inch QHD display with a fast 165 Hz[1] refresh rate and support for the NVIDIA Advanced Optimus feature. The Nitro 17 is further supported by an advanced cooling system, AI-assisted communication features, and NitroSense software for full device control. To top it all off, the Windows 11 gaming laptop is shipped with one month of Xbox Game Pass Ultimate, providing access to a library of hundreds of high-quality games to be explored with friends on PC, console, or cloud.

GIGABYTE Technology, a leading global brand in the computer industry, debuts AORUS and GIGABYTE Gaming AI gaming laptops at the 2024 Consumer Electronics Show (CES) in the United States, marking its entry into the new battleground of AI PCs. Today, the company launches the latest AI gaming laptops: G6X, G6, and G5, featuring up to the 13th gen. Intel Core HX series processors and NVIDIA RTX 40 Laptop GPUs. All three models the highest configuration come equipped with a Intel Core i7 processor and NVIDIA RTX 4060 Laptop GPU, delivering robust AI generative content computing power.

The G6X, featuring the Intel Core i7-13650HX processor and NVIDIA GeForce RTX 4060 Laptop GPU, excels at running diverse gaming titles and professional applications for creative content creation. It offers a quicker computing experience for AI generative content creation, enabling real-time and seamless realization of creative content. The laptops integrate with Microsoft's AI assistant, Copilot, assisting users in easier task creation and completion, thereby reducing daily workload and unlocking limitless productivity. In terms of audio, the entire series incorporates Dolby Atmos technology, delivering an unprecedented sense of depth, clarity, and detailed soundscapes. This not only enables gamers to accurately locate sounds and make precise strikes but also provides an unparalleled immersive experience.

ASUS Republic of Gamers (ROG) today announced the 2024 Zephyrus G14 and Zephyrus G16, the latest in an illustrious lineup of supremely powerful thin-and-light gaming laptops. These machines feature a new CNC-machined aluminium chassis, a customizable Slash Lighting array, and a brand-new Platinum White colorway, while cutting-edge AI accelerated silicon from Intel, AMD, and NVIDIA stand ready to push gamers and creators to new heights of performance. Both the Zephyrus G14 and G16 come equipped with the ROG Nebula Display, stunningly color-accurate OLED panels that are also G-SYNC capable for incredible gaming experiences. Ultra-efficient cooling technology, including tri-fan technology, liquid metal, and vapor chambers on select models enable the Zephyrus G14 and G16 to breathe easily despite their ultra-portable designs.

Brand-new chassis design
The 2024 Zephyrus G14 and Zephyrus G16 have been completely redesigned inside and out. Both machines boast all-new and all-aluminium CNC-machined chassis for the perfect mix of weight reduction, structural rigidity, and increased chassis space. This allows for an edge-to-edge keyboard design, as well as the inclusion of larger and louder speakers with superior bass response down to 100 Hz. The speakers are 25% larger than the previous generation, with a 47% volume increase for more immersive audio experiences than ever before. The Zephyrus G14 and G16 also come with larger individual keycaps and a larger touchpad, for superior typing, precision scrolling, and fluid gaming. Both the 2024 Zephyrus G14 and Zephyrus G16 ship with three months of Xbox Game Pass Ultimate, providing access to a library of hundreds of great games.

PNY announced today the arrival of the new VERTO GeForce RTX 4080 SUPER 16GB, RTX 4070 Ti SUPER 16GB, and RTX 4070 SUPER 12GB graphics cards to its lineup of NVIDIA GeForce RTX GPUs. The latest generation of RTX, GeForce RTX SUPER 40-series graphics cards are blazingly fast, offering gamers and creators an unparalleled boost in performance, neural rendering, and many more cutting-edge platform features. They are fueled by the revolutionary NVIDIA Ada Lovelace architecture; a major advancement in GPU technology which empowers accelerated content production techniques, amazing AI capabilities, and hyper-realistic gaming experiences.

The new GeForce RTX SUPER GPUs are the ultimate way to experience AI on PCs. Specialized AI Tensor Cores deliver up to 836 AI TOPS to deliver transformative capabilities for AI in gaming, creating and everyday productivity. PC gamers demand the very best in visual quality, and AI-powered NVIDIA Deep Learning Super Sampling (DLSS) Super Resolution, Frame Generation and Ray Reconstruction combine with ray tracing to offer stunning worlds. With DLSS, seven out of eight pixels can be AI-generated, accelerating full ray tracing by up to 4x with better image quality.

GIGABYTE TECHNOLOGY Co. Ltd, a leading manufacturer of premium gaming hardware, today launches a new graphics card powered by AMD RDNA 3 architecture. The GIGABYTE AMD Radeon RX 7600 XT GAMING OC 16G graphics card comes with the top-of-the-line WINDFORCE cooling system from GIGABYTE. It delivers unmatched performance, stunning visual effects, and exceptional efficiency, perfect for smooth 1080p gaming and streaming experience.

The GIGABYTE WINDFORCE cooling system is tailored for gamers, boasting three unique blade fans with alternate spinning, composite copper heat pipes in direct contact with the GPU, 3D active fans and screen cooling. The Alternate Spinning technology rotates the central fan in the opposite direction of the side fans, directing airflow in the same direction and doubling air pressure while reducing turbulence. This design effectively dissipates heat from both the top and the bottom of the graphics card, resulting in improved overall cooling performance.

Acer today announced the extension of its SpatialLabs stereoscopic 3D lineup to the Aspire line of laptops and Predator gaming monitors.

The new Aspire 3D 15 SpatialLabs Edition laptop delivers captivating 3D content for entertainment and creation on its 15.6-inch UHD display; It also comes with a suite of AI-powered SpatialLabs applications for 3D viewing and content creation, without the need for specialized glasses, delighting users when watching their favorite content and empowering developers to see their designs in their real 3D forms. With Microsoft Copilot in Windows 11, users can experience upscaled creativity and productivity with AI-powered task assistance, while Acer's suite of AI-supported solutions in Acer PurifiedView and PurifiedVoice elevate conference calls on the 3D laptop.

The Video Electronics Standards Association (VESA) today announced that it has published an update to its Adaptive-Sync Display Compliance Test Specification (Adaptive-Sync Display CTS), which is the first publicly open standard for front-of-screen performance of variable refresh rate displays. Adaptive-Sync Display version 1.1a provides updated testing procedures and logo support for an emerging category of displays that can operate at different maximum refresh rates when resolution is reduced. This optional "Dual Mode" testing and logo support allows display OEMs with qualifying hardware to certify their products at two different sets of resolution and refresh rate (for example, 4K/144 Hz and 1080p/280 Hz).

Adaptive-Sync Display v1.1a also includes an update that allows display OEMs to achieve a higher AdaptiveSync Display refresh rate certification for displays that support an "overclocked" or faster mode option that is not enabled by default in the factory configuration. In such cases, the overclocked mode must support Adaptive-Sync-enabled GPUs in a non-proprietary manner, and the display must pass all of the rigorous Adaptive-Sync Display compliance tests in both its factory default mode, and completely retested a second time in the overclocking mode. Both the dual mode and overclocking changes to the Adaptive-Sync Display CTS v1.1a only apply to the VESA Certified AdaptiveSync Display logo program; they do not apply to the VESA Certified MediaSync Display logo program.To date, more than 100 products have been certified to the Adaptive-Sync Display standard. A complete list of Adaptive-Sync Display certified products can be found at https://www.adaptivesync.org/certified-products/.

According to the latest round of reports, AMD has decided not to include China in the initial global launch of its upcoming Radeon RX 7600 XT graphics card. The RX 7600 XT, featuring 16 GB of memory and based on AMD's next-generation RDNA 3 architecture, was expected to launch soon at a price of around $300. However, the company is currently re-evaluating its Chinese GPU launch strategy due to the runaway success of its existing Radeon RX 6750 Golden Rabbit Edition (GRE) series in the region. The RX 6750 GRE cards with 10 GB and 12 GB configurations retail between $269-$289 in China, offering exceptional value compared to rival NVIDIA RTX models. AMD seems hesitant to risk undercutting sales of its popular RX 6750 GPUs by launching the newer 7600 XT.

While the RX 7600 XT promises more raw performance thanks to advanced RDNA 3 architecture, 6750 GRE, with its RDNA 2 design, seemingly remains efficient enough for most Chinese mainstream gamers. With the RX 6750 GRE still selling strongly in China, AMD has postponed the RX 7600 XT introduction for this key market. Final launch timelines for the 7600 XT in China and globally remain unconfirmed by AMD at time of writing. The company appears to be treading cautiously amidst the shifting competitive landscape.

During the recent IEDM conference, TSMC previewed its process roadmap for delivering next-generation chip packages packing over one trillion transistors by 2030. This aligns with similar long-term visions from Intel. Such enormous transistor counts will come through advanced 3D packaging of multiple chipsets. But TSMC also aims to push monolithic chip complexity higher, ultimately enabling 200 billion transistor designs on a single die. This requires steady enhancement of TSMC's planned N2, N2P, N1.4, and N1 nodes, which are slated to arrive between now and the end of the decade. While multi-chipset architectures are currently gaining favor, TSMC asserts both packaging density and raw transistor density must scale up in tandem. Some perspective on the magnitude of TSMC's goals include NVIDIA's 80 billion transistor GH100 GPU—among today's largest chips, excluding wafer-scale designs from Cerebras.

Yet TSMC's roadmap calls for more than doubling that, first with over 100 billion transistor monolithic designs, then eventually 200 billion. Of course, yields become more challenging as die sizes grow, which is where advanced packaging of smaller chiplets becomes crucial. Multi-chip module offerings like AMD's MI300X and Intel's Ponte Vecchio already integrate dozens of tiles, with PVC having 47 tiles. TSMC envisions this expansion to chip packages housing more than a trillion transistors via its CoWoS, InFO, 3D stacking, and many other technologies. While the scaling cadence has recently slowed, TSMC remains confident in achieving both packaging and process breakthroughs to meet future density demands. The foundry's continuous investment ensures progress in unlocking next-generation semiconductor capabilities. But physics ultimately dictates timelines, no matter how aggressive the roadmap.

In the era of high-tech development and the ever-increasing demand for data processing power, data centers are consuming more energy and generating excess heat. As a global leader in thermal solutions, SUNON is at the forefront, offering a diverse range of cutting-edge liquid cooling solutions tailored to advanced data centers equipped with high-capacity CPU and GPU computing for AI, edge, and cloud servers.

SUNON's liquid cooling design services are ideally suited for modern data centers, generative AI computing, and high-performance computing (HPC) applications. These solutions are meticulously customized to fit the cooling space and server density of each data center. With their compact yet comprehensive design, they guarantee exceptional cooling efficiency and reliability, ultimately contributing to a significant reduction in a client's total cost of ownership (TCO) in the long term. In the pursuit of net-zero emissions standards, SUNON's liquid cooling solutions play a pivotal role in enhancing corporate sustainability. They o ff er a win-win scenario for clients seeking to transition toward greener and more digitalized operations.

MemryX Inc. is announcing the availability of production level silicon of its cutting-edge AI Accelerator (MX3). MemryX is a pioneering startup specializing in accelerating artificial intelligence (AI) processing for edge devices. In less than 30 days after receiving production silicon from TSMC, MemryX will publicly showcase the ability to efficiently run hundreds of unaltered AI models at the 2024 Consumer Electronics Show (CES) in Las Vegas from Jan 9 through Jan 12.

Apple has been experimenting with Large Language Models (LLMs) that power most of today's AI applications. The company wants these LLMs to serve the users best and deliver them efficiently, which is a difficult task as they require a lot of resources, including compute and memory. Traditionally, LLMs have required AI accelerators in combination with large DRAM capacity to store model weights. However, Apple has published a paper that aims to bring LLMs to devices with limited memory capacity. By storing LLMs on NAND flash memory (regular storage), the method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding optimization in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Instead of storing the model weights on DRAM, Apple wants to utilize flash memory to store weights and only pull them on-demand to DRAM once it is needed.

Two principal techniques are introduced within this flash memory-informed framework: "windowing" and "row-column bundling." These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to native loading approaches on CPU and GPU, respectively. Integrating sparsity awareness, context-adaptive loading, and a hardware-oriented design pave the way for practical inference of LLMs on devices with limited memory, such as SoCs with 8/16/32 GB of available DRAM. Especially with DRAM prices outweighing NAND Flash, setups such as smartphone configurations could easily store and inference LLMs with multi-billion parameters, even if the DRAM available isn't sufficient. For a more technical deep dive, read the paper on arXiv here.

Phison Electronics Corp., a global leader in NAND flash controller and storage solutions, today announced the company's predictions for 2024 trends in NAND flash infrastructure deployment. The company predicts that rapid proliferation of artificial intelligence (AI) technologies will continue apace, with PCIe 5.0-based infrastructure providing high-performance, sustainable support for AI workload consistency as adoption rapidly expands. PCIe 5.0 NAND flash solutions will be at the core of a well-balanced hardware ecosystem, with private AI deployments such as on-premise large language models (LLMs) driving significant growth in both everyday AI and the infrastructure required to support it.

"We are moving past initial excitement over AI toward wider everyday deployment of the technology. In these configurations, high-quality AI output must be achieved by infrastructure designed to be secure, while also being affordable. The organizations that leverage AI to boost productivity will be incredibly successful," said Sebastien Jean, CTO, Phison US. "Building on the widespread proliferation of AI applications, infrastructure providers will be responsible for making certain that AI models do not run up against the limitations of memory - and NAND flash will become central to how we configure data center architectures to support today's developing AI market while laying the foundation for success in our fast-evolving digital future."

Chinese company Sipeed has introduced the Lichee Pocket 4A, one of the first handheld gaming devices based on the RISC-V open-source instruction set architecture (ISA). Sipeed positions the device as a retro gaming platform capable of running simple titles via software rendering or GPU acceleration. At its core is Alibaba's T-Head TH1520 processor featuring four 2.50 GHz Xuantie C910 RISC-V general-purpose CPU cores and an unnamed Imagination GPU. The chip was originally aimed at laptop designs. Memory options include 8 GB or 16 GB LPDDR4X RAM and 32 GB or 128 GB of storage. The Lichee Pocket 4A has a 7-inch 1280x800 LCD touchscreen, Wi-Fi/Bluetooth connectivity, and an array of wired ports like USB and Ethernet. It weighs under 500 grams. The device can run Android or Linux distributions like Debian, Ubuntu, and others.

As an early RISC-V gaming entrant, performance expectations should be modest—the focus is retro gaming and small indie titles, not modern AAA games. Specific gaming capabilities remain to be fully tested. However, the release helps showcase RISC-V's potential for consumer electronics and competitive positioning against proprietary ISAs like ARM. Pricing is still undefined, but another Sipeed handheld console retails for around $250 currently. Reception from enthusiasts and developers will demonstrate whether there's a viable market for RISC-V gaming devices. Success could encourage additional hardware experimentation efforts across emerging open architectures. With a 6000 mAh battery, battery life should be decent. Other specifications can be seen in the table below, and the pre-order link is here.