NVIDIA is possibly launching a more cost-effective variant of its GeForce RTX 4070 graphics card featuring GDDR6 memory, replacing the GDDR6X that it originally launched with. The new SKU would be better differentiated from the RTX 4070 SUPER. When NVIDIA refreshed its RTX 40-series "Ada" product stack in January, it discontinued the RTX 4070 Ti and RTX 4080, which had been replaced in the lineup by the RTX 4070 Ti SUPER and RTX 4080 SUPER, but at the time it didn't tinker with the RTX 4070, which continued to sell at a roughly $50 lower price than the RTX 4070 SUPER. This new SKU could be an attempt by NVIDIA to get further down below the $500-mark in pricing.

The RTX 4070 originally launched with 21 Gbps GDDR6X memory. This new variant sees the memory replaced with 20 Gbps conventional GDDR6. The JEDEC standard GDDR6 chips could be cheaper than GDDR6X, and could very well be the same GDDR6 chip models AMD uses in some of its higher Radeon RX 7000 series SKUs. This, however, comes with a 4.75% drop in memory bandwidth, which NVIDIA probably overcomes with increasing the GPU clocks a touch. The ASIC code for this SKU is AD104-251, compared to the AD104-250 of the original RTX 4070. The core configuration is otherwise unchanged—you get 5,888 CUDA cores across 46 streaming multiprocessors. Galax has a card based on this SKU ready.

Avnet ASIC, a division of Avnet Silica, an Avnet company, today announced that it has launched its new ultra-low-power design services for TSMC's cutting-edge 4 nm and below process technologies. These services are designed to enable customers to achieve exceptional power efficiency and performance in their high-performance applications, such as blockchain and AI edge computing. TSMC is the world's leading silicon foundry and Avnet ASIC division is a leading provider of ASIC and SoC full turnkey solutions.

The new design services leverage a comprehensive approach to address the challenges of operating at extreme low-voltage conditions in the 4 nm and below nodes. This includes recharacterizing standard cells for lower voltages, performing early RTL exploration to optimize power, performance, and area (PPA) tradeoffs, implementing an optimized clock tree, and utilizing transistor-level simulations to enhance the power optimization process.

According to The Information, OpenAI is reportedly in talks with Broadcom about developing a custom AI accelerator to power OpenAI's growing demand for high-performance solutions. Broadcom is a fabless chip designer known for a wide range of silicon solutions spanning from networking, PCIe, SSD controllers, and PHYs all the way up to custom ASICs. The latter part is what OpenAI wants to focus on, but all the aforementioned IP developed by Broadcom is of use in a data center. Suppose OpenAI decides to use Broadcom's solutions. In that case, the fabless silicon designer offers a complete vertical stack of products for inter-system communication using various protocols such as PCIe, system-to-system communication using Ethernet networking with Broadcom Tomahawk 6 and future revisions, alongside storage solutions and many other complimentary elements of a data center.

As a company skilled in making various IPs, it also makes ASIC solutions for other companies and has assisted Google in the making of its Tensor Processing Unit (TPU), which is now in its sixth generation. Google TPUs are massively successful as Google deploys millions of them and provides AI solutions to billions of users across the globe. Now, OpenAI wants to be part of the AI chip game, and Broadcom could come to the rescue with its already-established AI success and various other data center componentry to help make a custom AI accelerator to power OpenAI's infrastructure needed for the next generation of AI models. With each new AI model released by OpenAI, compute demand spikes by several orders of magnitude, and having an AI accelerator that exactly matches their need will help the company move faster and run even bigger AI models.

Tenstorrent is launching their next generation Wormhole chip featuring PCIe cards and workstations designed for developers who are interested in scalability for multi-chip development using Tenstorrent's powerful open-source software stacks.

These Wormhole-based cards and systems are now available for immediate order on tenstorrent.com:

• Wormhole n150, powered by a single processor
• Wormhole n300, powered by two processors
• TT-LoudBox, a developer workstation powered by four Wormhole n300s (eight processors)

TrendForce's latest industry report on AI servers reveals that high demand for advanced AI servers from major CSPs and brand clients is expected to continue in 2024. Meanwhile, TSMC, SK hynix, Samsung, and Micron's gradual production expansion has significantly eased shortages in 2Q24. Consequently, the lead time for NVIDIA's flagship H100 solution has decreased from the previous 40-50 weeks to less than 16 weeks.

TrendForce estimates that AI server shipments in the second quarter will increase by nearly 20% QoQ, and has revised the annual shipment forecast up to 1.67 million units—marking a 41.5% YoY growth.

ByteDance, TikTok's parent company, is reportedly working with American chip designer Broadcom to develop a cutting-edge AI processor. This collaboration could secure a stable supply of high-performance chips for ByteDance, according to Reuters. Sources claim the joint project involves a 5 nm Application-Specific Integrated Circuit (ASIC), designed to comply with U.S. export regulations. TSMC is slated to manufacture the chip, though production is not expected to begin this year.

This partnership marks a significant development in U.S.-China tech relations, as no public announcements of such collaborations on advanced chips have been made since Washington implemented stricter export controls in 2022. For ByteDance, this move could reduce procurement costs and ensure a steady chip supply, crucial for powering its array of popular apps, including TikTok and the ChatGPT-like AI chatbot "Doubao." The company has already invested heavily in AI chips, reportedly spending $2 billion on NVIDIA processors in 2023.

A new startup emerged out of stealth mode today to power the next generation of generative AI. Etched is a company that makes an application-specific integrated circuit (ASIC) to process "Transformers." The transformer is an architecture for designing deep learning models developed by Google and is now the powerhouse behind models like OpenAI's GPT-4o in ChatGPT, Antrophic Claude, Google Gemini, and Meta's Llama family. Etched wanted to create an ASIC for processing only the transformer models, making a chip called Sohu. The claim is Sohu outperforms NVIDIA's latest and greatest by an entire order of magnitude. Where a server configuration with eight NVIDIA H100 GPU clusters pushes Llama-3 70B models at 25,000 tokens per second, and the latest eight B200 "Blackwell" GPU cluster pushes 43,000 tokens/s, the eight Sohu clusters manage to output 500,000 tokens per second.

Why is this important? Not only does the ASIC outperform Hopper by 20x and Blackwell by 10x, but it also serves so many tokens per second that it enables an entirely new fleet of AI applications requiring real-time output. The Sohu architecture is so efficient that 90% of the FLOPS can be used, while traditional GPUs boast a 30-40% FLOP utilization rate. This translates into inefficiency and waste of power, which Etched hopes to solve by building an accelerator dedicated to power transformers (the "T" in GPT) at massive scales. Given that the frontier model development costs more than one billion US dollars, and hardware costs are measured in tens of billions of US Dollars, having an accelerator dedicated to powering a specific application can help advance AI faster. AI researchers often say that "scale is all you need" (resembling the legendary "attention is all you need" paper), and Etched wants to build on that.

D-Central Technologies proudly announces the launch of its latest innovations in Bitcoin mining hardware, the Bitaxe Supra v401 and the NerdAxe Ultra. These cutting-edge devices are set to revolutionize the Bitcoin mining landscape with their advanced features and unparalleled performance.

"The latest Bitaxe Supra v401 and NerdAxe Ultra set a new standard in Bitcoin mining accessibility", Jonathan Bertrand, CEO, D-Central Technologies

Today at Microsoft Build, AMD (NASDAQ: AMD) showcased its latest end-to-end compute and software capabilities for Microsoft customers and developers. By using AMD solutions such as AMD Instinct MI300X accelerators, ROCm open software, Ryzen AI processors and software, and Alveo MA35D media accelerators, Microsoft is able to provide a powerful suite of tools for AI-based deployments across numerous markets. The new Microsoft Azure ND MI300X virtual machines (VMs) are now generally available, giving customers like Hugging Face, access to impressive performance and efficiency for their most demanding AI workloads.

"The AMD Instinct MI300X and ROCm software stack is powering the Azure OpenAI Chat GPT 3.5 and 4 services, which are some of the world's most demanding AI workloads," said Victor Peng, president, AMD. "With the general availability of the new VMs from Azure, AI customers have broader access to MI300X to deliver high-performance and efficient solutions for AI applications."

Phison is arguably the most popular brand for SSD controllers in the client segment, but is turning more of attention to the vast enterprise segment. The company had been making first-party enterprise SSDs under its main marquee, but decided that the lineup needed its own brand that enterprise customers could better discern from the controller ASIC main brand. We hence have Pascari and Imagin. Pascari is an entire product family of fully built enterprise SSDs from Phison. The company's existing first-party drives under the main brand will probably migrate to the Pascari catalog. Imagin, on the other hand, is a design service for large cloud and data-center customers, so they could develop bespoke tiered storage solutions at scale.

The Pascari line of enterprise SSDs are designed completely in-house by Phison, feature their latest controllers, firmware, PCB, PMIC, and on-device power-failure protection on select products. The third-party components here are the NAND flash and DRAM chips, which have both been thoroughly evaluated by Phison for the best performance, endurance, and reliability, at their enterprise SSD design facility in Broomfield, Colorado. Phison already had a constellation of industry partners and suppliers to go around with, and the company's drives even power space missions; but the Pascari brand better differentiates the fully-built SSD lineup from the ASIC make. Pascari makes its debut with the X200 series high-performance SSDs for high-access heat data. The drive leverages Phison's latest PCIe Gen 5 controller technology, the most optimized memory components, and availability in all contemporary server storage form-factors.

During the Vision 2024 event, Intel announced its latest Gaudi 3 AI accelerator, promising significant improvements over its predecessor. Intel claims the Gaudi 3 offers up to 70% improvement in training performance, 50% better inference, and 40% better efficiency than Nvidia's H100 processors. The new AI accelerator is presented as a PCIe Gen 5 dual-slot add-in card with a 600 W TDP or an OAM module with 900 W. The PCIe card has the same peak 1,835 TeraFLOPS of FP8 performance as the OAM module despite a 300 W lower TDP. The PCIe version works as a group of four per system, while the OAM HL-325L modules can be run in an eight-accelerator configuration per server. This likely will result in a lower sustained performance, given the lower TDP, but it confirms that the same silicon is used, just finetuned with a lower frequency. Built on TSMC's N5 5 nm node, the AI accelerator features 64 Tensor Cores, delivering double the FP8 and quadruple FP16 performance over the previous generation Gaudi 2.

The Gaudi 3 AI chip comes with 128 GB of HBM2E with 3.7 TB/s of bandwidth and 24 200 Gbps Ethernet NICs, with dual 400 Gbps NICs used for scale-out. All of that is laid out on 10 tiles that make up the Gaudi 3 accelerator, which you can see pictured below. There is 96 MB of SRAM split between two compute tiles, which acts as a low-level cache that bridges data communication between Tensor Cores and HBM memory. Intel also announced support for the new performance-boosting standardized MXFP4 data format and is developing an AI NIC ASIC for Ultra Ethernet Consortium-compliant networking. The Gaudi 3 supports clusters of up to 8192 cards, coming from 1024 nodes comprised of systems with eight accelerators. It is on track for volume production in Q3, offering a cost-effective alternative to NVIDIA accelerators with the additional promise of a more open ecosystem. More information and a deeper dive can be found in the Gaudi 3 Whitepaper.

Samsung debuted its Mach-1 generation of AI processors during a recent shareholder meeting—the South Korean megacorp anticipates an early 2025 launch window. Their application-specific integrated circuit (ASIC) design is expected to "excel in edge computing applications," with a focus on low power and efficiency-oriented operating environments. Naver Corporation was a key NVIDIA high-end AI customer in South Korea (and Japan), but the leading search platform firm and creator of HyperCLOVA X LLM (reportedly) deliberated on an adoption alternative hardware last October. The Korea Economic Daily believes that Naver's relationship with Samsung is set to grow, courtesy of a proposed $752 million investment: "the world's top memory chipmaker, will supply its next-generation Mach-1 artificial intelligence chips to Naver Corp. by the end of this year."

Reports from last December indicated that the two companies were deep into the process of co-designing power-efficient AI accelerators—Naver's main goal is to finalize a product that will offer eight times more energy efficiency than NVIDIA's H100 AI accelerator. Naver's alleged bulk order—of roughly 150,000 to 200,000 Samsung Mach-1 AI chips—appears to be a stopgap. Industry insiders reckon that Samsung's first-gen AI accelerator is much cheaper when compared to NVIDIA H100 GPU price points—a per-unit figure of $3756 is mentioned in the KED Global article. Samsung is speculated to be shopping its fledgling AI tech to Microsoft and Meta.

Tenstorrent and MosChip Technologies announced today that they are partnering on design for Tenstorrent's cutting-edge RISC-V solutions. In selecting MosChip Technologies, Tenstorrent stands to strongly advance both its own and its customers' development of RISC-V solutions as they work together on Physical Design, DFT, Verification, and RTL Design services.

"MosChip Technologies is special in that they have unparalleled tape out expertise in design services, with more than 200 multi-million gate ASICs under their belt", said David Bennett, CCO of Tenstorrent. "Partnering with MosChip enables us to design the strongest RISC-V solution we can to serve ourselves, our partners, and our customers alike."

MICLEDI Microdisplays today announced a first closing of its Series A funding round with participation from imec.xpand, PMV, imec, KBC and SFPIM demonstrating strong support for the company's value proposition and commercial and technological progress achieved in the seed round. Series A follows a significant seed round award and additional non-dilutive funding in the form of grants and other vehicles from VLAIO. This brings the company's total funding to date to nearly $30 million.

"The company's achievements during this seed round have been astounding," said Sean Lord, CEO of MICLEDI. "Our door is open to engagements with some of the world's largest and most innovative electronic product manufacturing companies, most of whom are working on their own internal development projects for augmented reality (AR) displays in such diverse use cases as smart-wearable devices and automotive HUDs. This level of total funding to date is almost unheard of for a four-year-old startup."

TrendForce underscores that the primary momentum for server shipments this year remains with American CSPs. However, due to persistently high inflation and elevated corporate financing costs curtailing capital expenditures, overall demand has not yet returned to pre-pandemic growth levels. Global server shipments are estimated to reach approximately. 13.654 million units in 2024, an increase of about 2.05% YoY. Meanwhile, the market continues to focus on the deployment of AI servers, with their shipment share estimated at around 12.1%.

Foxconn is expected to see the highest growth rate, with an estimated annual increase of about 5-7%. This growth includes significant orders such as Dell's 16G platform, AWS Graviton 3 and 4, Google Genoa, and Microsoft Gen9. In terms of AI server orders, Foxconn has made notable inroads with Oracle and has also secured some AWS ASIC orders.

In 2024, the tech industry remains steadfastly focused on AI, with the continued rollout of advanced AI chips leading to significant enhancements in processing speeds. TrendForce posits that this advancement is set to drive growth in both DRAM and NAND Flash across various AI applications, including smartphones, servers, and notebooks. The server sector is expected to see the most significant growth, with content per box for server DRAM projected to rise by 17.3% annually, while enterprise SSDs are forecast to increase by 13.2%. The market penetration rate for AI smartphones and AI PCs is expected to experience noticeable growth in 2025 and is anticipated to further drive the average content per box upward.

Looking first at smartphones, despite chipmakers focusing on improving processing performance, the absence of new AI functionalities has somewhat constrained the impact of AI. Memory prices plummeted in 2023 due to oversupply, making lower-priced options attractive and leading to a 17.5% increase in average DRAM capacity and a 19.2% increase in NAND Flash capacity per smartphone. However, with no new applications expected in 2024, the growth rate in content per box for both DRAM and NAND Flash in smartphones is set to slow down, estimated at 14.1% and 9.3%, respectively.

Faraday Technology Corporation, a Taiwanese silicon IP designer, has announced plans to develop a new 64-core system-on-chip (SoC) utilizing Intel's most advanced 18A process technology. The Arm-based SoC will integrate Arm Neoverse compute subsystems (CSS) to deliver high performance and efficiency for data centers, infrastructure edge, and 5G networks. This collaboration brings together Faraday, Arm, and Intel Foundry Services. Faraday will leverage its ASIC design and IP solutions expertise to build the SoC. Arm will provide the Neoverse compute subsystem IP to enable scalable computing. Intel Foundry Services will manufacture the chip using its cutting-edge 18A process, which delivers one of the best-in-class transistor performance.

The new 64-core SoC will be a key component of Faraday's upcoming SoC evaluation platform. This platform aims to accelerate customer development of data center servers, high-performance computing ASICs, and custom SoCs. The platform will also incorporate interface IPs from the Arm Total Design ecosystem for complete implementation and verification. Both Arm and Intel Foundry Services expressed excitement about working with Faraday on this advanced Arm-based custom silicon project. "We're thrilled to see industry leaders like Faraday and Intel on the cutting edge of Arm-based custom silicon development," said an Arm spokesperson. Intel SVP Stuart Pann said, "We are pleased to work with Faraday in the development of the SoC based on Arm Neoverse CSS utilizing our most competitive Intel 18A process technology." The collaboration represents Faraday's strategic focus on leading-edge technologies to meet evolving application requirements. With its extensive silicon IP portfolio and design capabilities, Faraday wants to deliver innovative solutions and break into next-generation computing design.

Neuchips, a leading AI Application-Specific Integrated Circuits (ASIC) solutions provider, will demo its revolutionary Raptor Gen AI accelerator chip (previously named N3000) and Evo PCIe accelerator card LLM solutions at CES 2024. Raptor, the new chip solution, enables enterprises to deploy large language models (LLMs) inference at a fraction of the cost of existing solutions.

"We are thrilled to unveil our Raptor chip and Evo card to the industry at CES 2024," said Ken Lau, CEO of Neuchips. "Neuchips' solutions represent a massive leap in price to performance for natural language processing. With Neuchips, any organisation can now access the power of LLMs for a wide range of AI applications."

TrendForce reports that 3Q23 has been a historic quarter for the world's leading IC design houses as total revenue soared 17.8% to reach a record-breaking US$44.7 billion. This remarkable growth is fueled by a robust season of stockpiling for smartphones and laptops, combined with a rapid acceleration in the shipment of generative AI chips and components. NVIDIA, capitalizing on the AI boom, emerged as the top performer in revenue and market share. Notably, analog IC supplier Cirrus Logic overtook US PMIC manufacturer MPS to snatch the tenth spot, driven by strong demand for smartphone stockpiling.

NVIDIA's revenue soared 45.7% to US$16.5 billion in the third quarter, bolstered by sustained demand for generative AI and LLMs. Its data center business—accounting for nearly 80% of its revenue—was a key driver in this exceptional growth.

Huawei's subsidiary HiSilicon has made significant strides in the independent R&D of AI chips, launching the next-gen Ascend 910B. These chips are utilized not only in Huawei's public cloud infrastructure but also sold to other Chinese companies. This year, Baidu ordered over a thousand Ascend 910B chips from Huawei to build approximately 200 AI servers. Additionally, in August, Chinese company iFlytek, in partnership with Huawei, released the "Gemini Star Program," a hardware and software integrated device for exclusive enterprise LLMs, equipped with the Ascend 910B AI acceleration chip, according to TrendForce's research.

TrendForce conjectures that the next-generation Ascend 910B chip is likely manufactured using SMIC's N+2 process. However, the production faces two potential risks. Firstly, as Huawei recently focused on expanding its smartphone business, the N+2 process capacity at SMIC is almost entirely allocated to Huawei's smartphone products, potentially limiting future capacity for AI chips. Secondly, SMIC remains on the Entity List, possibly restricting access to advanced process equipment.

Zero ASIC, a semiconductor startup, came out of stealth today to announce early access to its one-of-a-kind ChipMaker platform, demonstrating a number of world firsts:

• 3D chiplet composability enabling billions of new silicon products
• Fully automated no-code chiplet-based chip design
• Zero install interactive RTL-based chip emulation
• Roadmap to 100X reduction in chip development costs

"Custom Application Specific Integrated Circuits (ASICs) offer 10-100X cost and energy advantage over commercial off the shelf (COTS) devices, but the enormous development cost makes ASICs non-viable for most applications," said Andreas Olofsson, CEO and founder of Zero ASIC. "To build the next wave of world changing silicon devices, we need to reduce the barrier to ASICs by orders of magnitude. Our mission at Zero ASIC is to make ordering an ASIC as easy as ordering catalog parts from an electronics distributor."

Phison Electronics, a global leader in NAND controllers and storage solutions, announced today that the company has expanded its portfolio of PCIe 5.0 high-speed transmission solutions with PCIe 5.0, CXL 2.0 compatible redriver and retimer data signal conditioning IC products. Leveraging the company's deep expertise in PCIe engineering, Phison is the only signal conditioners provider that offers the widest portfolio of multi-channel PCIe 5.0 redriver and retimer solutions and PCIe 5.0 storage solutions designed specifically to meet the data infrastructure demands of artificial intelligence and machine learning (AI+ML), edge computing, high-performance computing, and other data-intensive, next-gen applications. At the 2023 Open Compute Project Global Summit, the Phison team is showcasing its expansive PCIe 5.0 portfolio, demonstrating the redriver and retimer technologies alongside other enterprise NAND flash, illustrating a holistic vision for a PCIe 5.0 data ecosystem to address the most demanding applications of the AI-everywhere era.

"Phison has focused industry-leading R&D efforts on developing in-house, chip-to-chip communication technologies since the introduction of the PCIe 3.0 protocol, with PCIe 4.0 and PCIe 5.0 solutions now in mass production, and PCIe 6.0 solutions now in the design phase," said Michael Wu, President & General Manager, Phison US. "Phison's accumulated experience in high-speed signaling enables our team to deliver retimer and redriver design solutions that are optimized for top signal integration, low power usage, and high temperature endurance, to deliver interface speeds for the most challenging compute environments."

Avicena, a privately held company headquartered in Sunnyvale, CA, is demonstrating its LightBundle multi-Tbps chip-to-chip interconnect technology at the European Conference for Optical Communications (ECOC) 2023 in Glasgow, Scotland (https://www.ecocexhibition.com/). Avicena's microLED-based LightBundle architecture breaks new ground by unlocking the performance of processors, memory and sensors, removing key bandwidth and proximity constraints while simultaneously offering class leading energy efficiency.

"As generative AI continues to evolve, the role of high bandwidth-density, low-power and low latency interconnects between xPUs and HBM modules cannot be overstated", says Chris Pfistner, VP Sales & Marketing of Avicena. "Avicena's innovative LightBundle interconnects have the potential to fundamentally change the way processors connect to each other and to memory because their inherent parallelism is well-matched to the internal wide and slow bus architecture within ICs. With a roadmap to multi-terabit per second capacity and sub-pJ/bit efficiency these interconnects are poised to enable the next era of AI innovation, paving the way for even more capable models and a wide range of AI applications that will shape the future."

NVIDIA's latest financial report for FY2Q24 reveals that its data center business reached US$10.32 billion—a QoQ growth of 141% and YoY increase of 171%. The company remains optimistic about its future growth. TrendForce believes that the primary driver behind NVIDIA's robust revenue growth stems from its data center's AI server-related solutions. Key products include AI-accelerated GPUs and AI server HGX reference architecture, which serve as the foundational AI infrastructure for large data centers.

TrendForce further anticipates that NVIDIA will integrate its software and hardware resources. Utilizing a refined approach, NVIDIA will align its high-end, mid-tier, and entry-level GPU AI accelerator chips with various ODMs and OEMs, establishing a collaborative system certification model. Beyond accelerating the deployment of CSP cloud AI server infrastructures, NVIDIA is also partnering with entities like VMware on solutions including the Private AI Foundation. This strategy extends NVIDIA's reach into the edge enterprise AI server market, underpinning steady growth in its data center business for the next two years.

Semiconductor industry research firm TechInsights said it has found that Samsung's 3 nm GAA (gate-all-around) process has been incorporated into the crypto miner ASIC (Whatsminer M56S++) from a Chinese manufacturer, MicroBT. In a Disruptive Technology Event Brief exclusively provided to DIGITIMES Asia, TechInsights points out that the significance of this development lies in the commercial utilization of GAA technology, which facilitates the scaling of transistors to 2 nm and beyond. "This development is crucial because it has the potential to enhance performance, improve energy efficiency, keep up with Moore's Law, and enable advanced applications," said TechInsights, identifying the MicroBT ASIC chip the first commercialized product using GAA technology in the industry.

But this would also reveal that Samsung is the foundry for MicroBT, using the 3 nm GAA process. DIGITIMES Research semiconductor analyst Eric Chen pointed out that Samsung indeed has started producing chips using the 3 nm GAA process, but the capacity is still small. "Getting revenues from shipment can be defined as 'commercialization', but ASIC is a relatively simple kind of chip to produce, in terms of architecture."