MSI, a leading global provider of high-performance server solutions, proudly announced it is showcasing new AI servers powered by the NVIDIA MGX platform—designed to address the increasing demand for scalable, energy-efficient AI workloads in modern data centers—at the OCP Global Summit 2024, booth A6. This collaboration highlights MSI's continued commitment to advancing server solutions, focusing on cutting-edge AI acceleration and high-performance computing (HPC).

The NVIDIA MGX platform offers a flexible architecture that enables MSI to deliver purpose-built solutions optimized for AI, HPC, and LLMs. By leveraging this platform, MSI's AI server solutions provide exceptional scalability, efficiency, and enhanced GPU density—key factors in meeting the growing computational demands of AI workloads. Tapping into MSI's engineering expertise and NVIDIA's advanced AI technologies, these AI servers based on the MGX architecture deliver unparalleled compute power, positioning data centers to maximize performance and power efficiency while paving the way for the future of AI-driven infrastructure.

At Lenovo Tech World 2024, we announced new Supercomputing servers for HPC and AI workloads. These new water-cooled servers use the latest processor and accelerator technology from Intel and NVIDIA.

ThinkSystem SC750 V4
Engineered for large-scale cloud infrastructures and High Performance Computing (HPC), the Lenovo ThinkSystem SC750 V4 Neptune excels in intensive simulations and complex modeling. It's designed to handle technical computing, grid deployments, and analytics workloads in various fields such as research, life sciences, energy, engineering, and financial simulation.

Yesterday, Arm has announced significant progress in its Total Design initiative. The program, launched a year ago, aims to accelerate the development of custom silicon for data centers by fostering collaboration among industry partners. The ecosystem has now grown to include nearly 30 participating companies, with recent additions such as Alcor Micro, Egis, PUF Security, and SEMIFIVE. A notable development is a partnership between Arm, Samsung Foundry, ADTechnology, and Rebellions to create an AI CPU chiplet platform. This collaboration aims to deliver a solution for cloud, HPC, and AI/ML workloads, combining Rebellions' AI accelerator with ADTechnology's compute chiplet, implemented using Samsung Foundry's 2 nm Gate-All-Around (GAA) FET technology. The platform is expected to offer significant efficiency gains for generative AI workloads, with estimates suggesting a 2-3x improvement over the standard CPU design for LLMs like Llama3.1 with 405 billion parameters.

Arm's approach emphasizes the importance of CPU compute in supporting the complete AI stack, including data pre-processing, orchestration, and advanced techniques like Retrieval-augmented Generation (RAG). The company's Compute Subsystems (CSS) are designed to address these requirements, providing a foundation for partners to build diverse chiplet solutions. Several companies, including Alcor Micro and Alphawave, have already announced plans to develop CSS-powered chiplets for various AI and high-performance computing applications. The initiative also focuses on software readiness, ensuring that major frameworks and operating systems are compatible with Arm-based systems. Recent efforts include the introduction of Arm Kleidi technology, which optimizes CPU-based inference for open-source projects like PyTorch and Llama.cpp. Notably, as Google claims, most AI workloads are being inferenced on CPUs, so creating the most efficient and most performant CPUs for AI makes a lot of sense.

The EuroHPC Joint Undertaking (EuroHPC JU) has signed a purchase agreement with IQM Quantum Computers (IQM), a global leader in designing, building, and selling superconducting quantum computers. Under the agreement, IQM will deliver two advanced Radiance quantum systems of 54 qubits and 150 qubits in the second half of 2025 and by the end of 2026, respectively.

The two distinct systems, featuring high-quality qubits and industry-leading fidelities will play a pivotal role in executing quantum algorithms across a range of application domains.

MSI, a leading global provider of high-performance server solutions, is excited to showcase its comprehensive lineup of motherboards and servers based on the OCP Modular Hardware System (DC-MHS) architecture at the OCP Global Summit from October 15-17 at booth A6. These cutting-edge solutions represent a breakthrough in server designs, enabling flexible deployments for cloud and high-density data centers. Featured innovations include CXL memory expansion servers and AI-optimized servers, demonstrating MSI's leadership in pushing the boundaries of AI performance and computing power.

DC-MHS Series Motherboards and Servers: Enabling Flexible Deployment in Data Centers
"The rapidly evolving IT landscape requires cloud service providers, large-scale data center operators, and enterprises to handle expanding workloads and future growth with more flexible and powerful infrastructure. MSI's new rage of DC-MHS-based solutions provides the needed flexibility and efficiency for modern data center environments," said Danny Hsu, General Manager of Enterprise Platform Solutions.

Supermicro, Inc., a Total IT Solution Provider for AI, Cloud, Storage, and 5G/Edge, is launching a new optimized storage system for high performance AI training, inference and HPC workloads. This JBOF (Just a Bunch of Flash) system utilizes up to four NVIDIA BlueField-3 data processing units (DPUs) in a 2U form factor to run software-defined storage workloads. Each BlueField-3 DPU features 400 Gb Ethernet or InfiniBand networking and hardware acceleration for high computation storage and networking workloads such as encryption, compression and erasure coding, as well as AI storage expansion. The state-of-the-art, dual port JBOF architecture enables active-active clustering ensuring high availability for scale up mission critical storage applications as well as scale-out storage such as object storage and parallel file systems.

"Supermicro's new high performance JBOF Storage System is designed using our Building Block approach which enables support for either E3.S or U.2 form-factor SSDs and the latest PCIe Gen 5 connectivity for the SSDs and the DPU networking and storage platform," said Charles Liang, president and CEO of Supermicro. "Supermicro's system design supports 24 or 36 SSD's enabling up to 1.105PB of raw capacity using 30.71 TB SSDs. Our balanced network and storage I/O design can saturate the full 400 Gb/s BlueField-3 line-rate realizing more than 250 GB/s bandwidth of the Gen 5 SSDs."

Flex today announced new reference platforms for liquid-cooled servers, rack, and power products that will enable customers to sustainably accelerate data center growth. These innovations build on Flex's ability to address technical challenges associated with power, heat generation, and scale to support artificial intelligence (AI) and high-performance computing (HPC) workloads.

"Flex delivers integrated data center IT and power infrastructure solutions that address the growing power and compute demands in the AI era," said Michael Hartung, president and chief commercial officer, Flex. "We are expanding our unique portfolio of advanced manufacturing capabilities, innovative products, and lifecycle services, enabling customers to deploy IT and power infrastructure at scale and drive AI data center expansion."

In close cooperation with hyperscalers and server OEMs, Rittal has developed a modular cooling distribution unit (CDU) that delivers a cooling capacity of over 1 MW. It will be the centerpiece exhibit at Rittal's booth A24 at 2024 OCP Global Summit. The CDU uses direct liquid cooling based on water - and is thus an example for new IT infrastructure technologies that are enablers for AI applications.

New technology, familiar handling?
"To put the technology into practice, it is not enough to simply provide the cooling capacity and integrate the solution into the facility - which also still poses challenges," says Lars Platzhoff, Head of Rittal's Business Unit Cooling Solutions: "Despite the new technology, the solutions must remain manageable by the data center team as part of the usual service. At best, this should be taken into account already at the design stage."

AMD has unveiled its latest EPYC processors, codenamed "Turin," featuring Zen 5 and Zen 5C dense cores. Phoronix's thorough testing reveals remarkable advancements in performance, efficiency, and value. The new lineup includes the EPYC 9575F (64-core), EPYC 9755 (128-core), and EPYC 9965 (192-core) models, all showing impressive capabilities across various server and HPC workloads. In benchmarks, a dual-socket configuration of the 128-core EPYC 9755 Turin outperformed Intel's dual Xeon "Granite Rapids" 6980P setup with MRDIMM-8800 by 40% in the geometric mean of all tests. Surprisingly, even a single EPYC 9755 or EPYC 9965 matched the dual Xeon 6980P in expanded tests with regular DDR5-6400. Within AMD's lineup, the EPYC 9755 showed a 1.55x performance increase over its predecessor, the 96-core EPYC 9654 "Genoa". The EPYC 9965 surpassed the dual EPYC 9754 "Bergamo" by 45%.

These gains come with improved efficiency. While power consumption increased moderately, performance improvements resulted in better overall efficiency. For example, the EPYC 9965 used 32% more power than the EPYC 9654 but delivered 1.55x the performance. Power consumption remains competitive: the EPYC 9965 averaged 275 Watts (peak 461 Watts), the EPYC 9755 averaged 324 Watts (peak 500 Watts), while Intel's Xeon 6980P averaged 322 Watts (peak 547 Watts). AMD's pricing strategy adds to the appeal. The 192-core model is priced at $14,813, compared to Intel's 128-core CPU at $17,800. This competitive pricing, combined with superior performance per dollar and watt, has resonated with hyperscalers. Estimates suggest 50-60% of hyperscale deployments now use AMD processors.

MiTAC Computing Technology Corporation, an industry-leading server platform design manufacturer and a subsidiary of MiTAC Holdings Corporation (TSE:3706), today announced the launch of its new high-performance servers, featuring the latest AMD EPYC 9005 Series CPUs and AMD Instinct MI325X accelerators.

"AMD is the trusted data center solutions provider of choice for leading enterprises worldwide, whether they are enabling corporate AI initiatives, building large-scale cloud deployments, or hosting critical business applications on-premises," said Ravi Kuppuswamy, senior vice president, Server Business Unit, AMD. "Our latest 5th Gen AMD EPYC CPUs provide the performance, flexibility and reliability - with compatibility across the x86 data center ecosystem - to deliver tailored solutions that meet the diverse demands of the modern data center."

ASRock Rack Inc., a leading innovative server company, announced upgrades to its extensive lineup to support AMD EPYC 9005 Series processors. Among these updates is the introduction of the new 6U8M-TURIN2 GPU server. This advanced platform features AMD Instinct MI325X accelerators, specifically optimized for intensive enterprise AI applications, and will be showcased at AMD Advancing AI 2024.

ASRock Rack Introduce GPU Servers Powered by AMD EPYC 9005 series processors
AMD today revealed the 5th Generation AMD EPYC processors, offering a wide range of core counts (up to 192 cores), frequencies (up to 5 GHz), and expansive cache capacities. Select high-frequency processors, such as the AMD EPYC 9575F, are optimized for use as host CPUs in GPU-enabled systems. Additionally, the just launched AMD Instinct MI325X accelerators feature substantial HBM3E memory and 6 TB/s of memory bandwidth, enabling quick access and efficient handling of large datasets and complex computations.

MSI, a leading global provider of high-performance server solutions, today introduced its latest AMD EPYC 9005 Series CPU-based server boards and platforms, engineered to tackle the most demanding data center workloads with leadership performance and efficiency.

Featuring AMD EPYC 9005 Series processors with up to 192 cores and 384 threads, MSI's new server platforms deliver breakthrough compute power, unparalleled density, and exceptional energy efficiency, making them ideal for handling AI-enabled, cloud-native, and business-critical workloads in modern data centers.

AMD (NASDAQ: AMD) today announced the availability of the 5th Gen AMD EPYC processors, formerly codenamed "Turin," the world's best server CPU for enterprise, AI and cloud. Using the "Zen 5" core architecture, compatible with the broadly deployed SP5 platform and offering a broad range of core counts spanning from 8 to 192, the AMD EPYC 9005 Series processors extend the record-breaking performance and energy efficiency of the previous generations with the top of stack 192 core CPU delivering up to 2.7X the performance compared to the competition.

New to the AMD EPYC 9005 Series CPUs is the 64 core AMD EPYC 9575F, tailor made for GPU powered AI solutions that need the ultimate in host CPU capabilities. Boosting up to 5 GHz, compared to the 3.8 GHz processor of the competition, it provides up to 28% faster processing needed to keep GPUs fed with data for demanding AI workloads.

Astera Labs, Inc, a global leader in semiconductor-based connectivity solutions for AI and cloud infrastructure, today announced a new portfolio of fabric switches, including the industry's first PCIe 6 switch, built from the ground up for demanding AI workloads in accelerated computing platforms deployed at cloud-scale. The Scorpio Smart Fabric Switch portfolio is optimized for AI dataflows to deliver maximum predictable performance per watt, high reliability, easy cloud-scale deployment, reduced time-to-market, and lower total cost of ownership.

The Scorpio Smart Fabric Switch portfolio features two application-specific product lines with a multi-generational roadmap:

• Scorpio P-Series for GPU-to-CPU/NIC/SSD PCIe 6 connectivity- architected to support mixed traffic head-node connectivity across a diverse ecosystem of PCIe hosts and endpoints.
• Scorpio X-Series for back-end GPU clustering-architected to deliver the highest back-end GPU-to-GPU bandwidth with platform-specific customization.

After Apple, we just learned that AMD is the next company in line for US-based manufacturing in the TSMC Arizona facility. Industry analyst Tim Culpan reports that TSMC's Fab 21 in Arizona will soon be producing AMD's high-performance computing (HPC) processors, with tape out and manufacturing expected to commence on TSMC's 5 nm node next year. This move comes after previously reported Apple's A16 SoC production, which is already in progress at the facility and could see shipments before the end of this year, significantly ahead of the initially projected early 2025 schedule. The production of AMD's HPC chips in Arizona marks a crucial step towards establishing an AI-hardware supply chain operating entirely on American soil, which is expected to further expand with Intel Foundry and Samsung Texas facility.

Making HPC processors domestically serves as a significant milestone in reducing dependence on overseas semiconductor manufacturing and strengthening the US's position in the global chip industry. Adding to the momentum, TSMC and Amkor recently announced a collaboration on advanced packaging technologies, including Integrated Fan-Out (InFO) and Chip-on-Wafer-on-Substrate (CoWoS), which are vital for high-performance AI chips. However, as Amkor facilities are yet to be built, these chips are going to be shipped back to Taiwan for packaging before being integrated into the final product. Once the Amkor facility is up and running, Arizona will become the birthplace of fully manufactured and packaged silicon chips.

Supermicro, Inc., a Total IT Solution Provider for Cloud, AI/ML, Storage, and 5G/Edge, is announcing a complete liquid cooling solution that includes powerful Coolant Distribution Units (CDUs), cold plates, Coolant Distribution Manifolds (CDMs), cooling towers and end to end management software. This complete solution reduces ongoing power costs and Day 0 hardware acquisition and data center cooling infrastructure costs. The entire end-to-end data center scale liquid cooling solution is available directly from Supermicro.

"Supermicro continues to innovate, delivering full data center plug-and-play rack scale liquid cooling solutions," said Charles Liang, CEO and president of Supermicro. "Our complete liquid cooling solutions, including SuperCloud Composer for the entire life-cycle management of all components, are now cooling massive, state-of-the-art AI factories, reducing costs and improving performance. The combination of Supermicro deployment experience and delivering innovative technology is resulting in data center operators coming to Supermicro to meet their technical and financial goals for both the construction of greenfield sites and the modernization of existing data centers. Since Supermicro supplies all the components, the time to deployment and online are measured in weeks, not months."

Fujitsu Limited and Supermicro, Inc. (NASDAQ: SMCI), today announced they will collaborate to establish a long-term strategic engagement in technology and business, to develop and market a platform with Fujitsu's future Arm-based "FUJITSU-MONAKA" processor that is designed for high-performance and energy efficiency and targeted for release in 2027. In addition, the two companies will also collaborate on developing liquid-cooled systems for HPC, Gen AI, and next-generation green data centers.

"Supermicro is excited to collaborate with Fujitsu to deliver state-of-the-art servers and solutions that are high performance, power efficient, and cost-optimized," said Charles Liang, president and CEO of Supermicro. "These systems will be optimized to support a broad range of workloads in AI, HPC, cloud and edge environments. The two companies will focus on green IT designs with energy-saving architectures, such as liquid cooling rack scale PnP, to minimize technology's environmental impact."

NVIDIA has reportedly discontinued its dual-rack GB200 NVL36x2 GPU model, opting to focus on the single-rack GB200 NVL72 and NVL36 models. This shift, revealed by industry analyst Ming-Chi Kuo, aims to simplify NVIDIA's offerings in the AI and HPC markets. The decision was influenced by major clients like Microsoft, who prefer the NVL72's improved space efficiency and potential for enhanced inference performance. While both models perform similarly in AI large language model (LLM) training, the NVL72 is expected to excel in non-parallelizable inference tasks. As a reminder, the NVL72 features 36 Grace CPUs, delivering 2,592 Arm Neoverse V2 cores with 17 TB LPDDR5X memory with 18.4 TB/s aggregate bandwidth. Additionally, it includes 72 Blackwell GB200 SXM GPUs that have a massive 13.5 TB of HBM3e combined, running at 576 TB/s aggregate bandwidth.

However, this shift presents significant challenges. The NVL72's power consumption of around 120kW far exceeds typical data center capabilities, potentially limiting its immediate widespread adoption. The discontinuation of the NVL36x2 has also sparked concerns about NVIDIA's execution capabilities and may disrupt the supply chain for assembly and cooling solutions. Despite these hurdles, industry experts view this as a pragmatic approach to product planning in the dynamic AI landscape. While some customers may be disappointed by the dual-rack model's cancellation, NVIDIA's long-term outlook in the AI technology market remains strong. The company continues to work with clients and listen to their needs, to position itself as a leader in high-performance computing solutions.

The zettascale era is officially on the map, as Japan has announced plans to develop a successor to its renowned Fugaku supercomputer. The Ministry of Education, Culture, Sports, Science and Technology (MEXT) has set its sights on creating a machine capable of unprecedented processing power, aiming for 50 ExaFLOPS of peak AI performance with zettascale capabilities. The ambitious "Fugaku Next" project, slated to begin development next year, will be headed by RIKEN, one of Japan's leading research institutions, in collaboration with tech giant Fujitsu. With a target completion date of 2030, the new supercomputer aims to surpass current technological boundaries, potentially becoming the world's fastest once again. MEXT's vision for the "Fugaku Next" includes groundbreaking specifications for each computational node.

The ministry anticipates peak performance of several hundred FP64 TFLOPS for double-precision computations, around 50 FP16 PFLOPS for AI-oriented half-precision calculations, and approximately 100 PFLOPS for AI-oriented 8-bit precision calculations. These figures represent a major leap from Fugaku's current capabilities. The project's initial funding is set at ¥4.2 billion ($29.06 million) for the first year, with total government investment expected to exceed ¥110 billion ($761 million). While the specific architecture remains undecided, MEXT suggests the use of CPUs with special-purpose accelerators or a CPU-GPU combination. The semiconductor node of choice will likely be a 1 nm node or even more advanced nodes available at the time, with advanced packaging also used. The supercomputer will also feature an advanced storage system to handle traditional HPC and AI workloads efficiently. We already have an insight into Monaka, Fujitsu's upcoming CPU design with 150 Armv9 cores. However, Fugaku Next will be powered by the Monaka Next design, which will likely be much more capable.

Demonstrating the depth and breadth of its technologies at Hot Chips 2024, Intel showcased advancements across AI use cases - from the data center, cloud and network to the edge and PC - while covering the industry's most advanced and first-ever fully integrated optical compute interconnect (OCI) chiplet for high-speed AI data processing. The company also unveiled new details about the Intel Xeon 6 SoC (code-named Granite Rapids-D), scheduled to launch during the first half of 2025.

"Across consumer and enterprise AI usages, Intel continuously delivers the platforms, systems and technologies necessary to redefine what's possible. As AI workloads intensify, Intel's broad industry experience enables us to understand what our customers need to drive innovation, creativity and ideal business outcomes. While more performant silicon and increased platform bandwidth are essential, Intel also knows that every workload has unique challenges: A system designed for the data center can no longer simply be repurposed for the edge. With proven expertise in systems architecture across the compute continuum, Intel is well-positioned to power the next generation of AI innovation." -Pere Monclus, chief technology officer, Network and Edge Group at Intel.

ASUS today announced its ambitious All in AI initiative, marking a significant leap into the server market with a complete AI infrastructure solution, designed to meet the evolving demands of AI-driven applications from edge, inference and generative AI the new, unparalleled wave of AI supercomputing. ASUS has proven its expertise lies in striking the perfect balance between hardware and software, including infrastructure and cluster architecture design, server installation, testing, onboarding, remote management and cloud services - positioning the ASUS brand and AI server solutions to lead the way in driving innovation and enabling the widespread adoption of AI across industries.

Meeting diverse AI needs
In partnership with NVIDIA, Intel and AMD, ASUS offer comprehensive AI-infrastructure solutions with robust software platforms and services, from entry-level AI servers and machine-learning solutions to full racks and data centers for large-scale supercomputing. At the forefront is the ESC AI POD with NVIDIA GB200 NVL72, a cutting-edge rack designed to accelerate trillion-token LLM training and real-time inference operations. Complemented by the latest NVIDIA Blackwell GPUs, NVIDIA Grace CPUs and 5th Gen NVIDIA NVLink technology, ASUS servers ensure unparalleled computing power and efficiency.

Tachyum today announced the final build of its Prodigy FPGA emulation system in advance of chip production and general availability next year. As part of the announcement, the company is also ending its purchase program for prototype systems that was previously offered to commercial and federal customers.

These last hardware FPGA prototype units will ensure Tachyum hits its extreme-reliability test targets of more than 10 quadrillion cycles prior to tape-out and before the first Prodigy chips hit the market. Tachyum's software emulation system - and access to it - is expanding with additional availability of open-source software ported ahead of Prodigy's upstreaming.

KIOXIA America, Inc. will showcase a prototype broadband SSD with an optical interface for next-generation data centers at the FMS: the Future of Memory and Storage conference, being held in Santa Clara, California from August 6 to August 8. By replacing the electrical wiring interface with an optical interface, this SSD technology allows for greater physical distance between the compute and storage devices, slims down wiring and delivers high flexibility to data center system designs and applications—all while maintaining energy efficiency and high signal quality.

By adopting an optical interface, it becomes possible to aggregate individual components that make up systems, such as SSDs and CPUs, and seamlessly interconnect them. This furthers the evolution of a disaggregated computing system that can efficiently utilize resources according to a specific workload. Additionally, with its high signal integrity, the optical interface may enhance high-performance compute (HPC) environments, such as in supercomputers, cloud based-HPC and even applications in outer space.

According to sources close to Bloomberg, Intel plans to cut 10,000 jobs from its global workforce. The news comes amid heavy pressure on the semiconductor giant, which has been on a steady decline over the years, while other industry rivals like AMD and NVIDIA have been rising and taking market share in various areas from Intel. It is reported that Intel currently has 110,000 employees globally, and reducing the workforce by 10,000 would net Intel around 100,000 global employees left. These figures exclude employees from spun-out units like Altera FPGA company, which is under Intel's ownership. Intel's aim to reduce its workforce is expected to come with a significant cost benefit to the company, with projected savings of $10 billion by 2025.

The news isn't yet official, but it is expected to see the light of the day as soon as this week. As Intel's CEO Pat Gelsinger invests heavily into the fab construction and development of next-generation products, there have been a few notes that Intel would have to overcome some challenges shortly to reach its long-term goals like more advanced silicon manufacturing facilities and new products for AI/HPC and client sector. One of those short-term measures is reducing the workforce to cut down expenses. Intel has reduced its workforce before. In 2022, the company announced reduced spending in non-critical areas and reducing the workforce, and in 2023, cut the workforce by 5% to 124,800 employees last year, only to be left with 110,000 employees in 2024.

Cornelis Networks, a leading independent provider of intelligent, high-performance networking solutions, today announced the appointment of Lisa Spelman as its new chief executive officer (CEO), effective August 15. Spelman joins Cornelis from Intel Corporation, where she held executive leadership roles for more than two decades, including leading the company's core data center business. Spelman will succeed Philip Murphy, who will assume the role of president and chief operating officer (COO).

"Cornelis is unique in having the products, roadmap, and talent to help customers address this issue. I look forward to joining the team to bring their innovations to even more organizations around the globe."