Inference software enables shift to NVIDIA A100 Tensor Core GPUs, delivering 7x throughput for the search giant. Jiusheng Chen's team just got accelerated. They're delivering personalized ads to users of Microsoft Bing with 7x throughput at reduced cost, thanks to NVIDIA Triton Inference Server running on NVIDIA A100 Tensor Core GPUs. It's an amazing achievement for the principal software engineering manager and his crew.

Tuning a Complex System
Bing's ad service uses hundreds of models that are constantly evolving. Each must respond to a request within as little as 10 milliseconds, about 10x faster than the blink of an eye. The latest speedup got its start with two innovations the team delivered to make AI models run faster: Bang and EL-Attention. Together, they apply sophisticated techniques to do more work in less time with less computer memory. Model training was based on Azure Machine Learning for efficiency.

Technology is creating amazing opportunities for our industry and the world, and Windows 11 is at the center of innovation with new experiences that transform how we interact with the Windows PC and each other. With Windows 11, we're seeing record engagement, our highest customer satisfaction over any previous version of Windows in the U.S., and accelerated growth in commercial deployments. In fact, we recently shared as part of our latest earnings report that over 90% of Fortune 500 companies are currently trialing or have already deployed Windows 11.

In February, we brought the new AI-powered Bing to the taskbar and other features to improve the way people get things done on their PCs. Today, we are providing a look at new features and enhancements for Windows 11 that will start to become available tomorrow, May 24, that focus on business needs, such as security and IT management, and new benefits that enhance professional and personal usability. For developers building on Windows, this week we are announcing new innovation at Microsoft Build.

NVIDIA today announced that it is integrating its NVIDIA AI Enterprise software into Microsoft's Azure Machine Learning to help enterprises accelerate their AI initiatives. The integration will create a secure, enterprise-ready platform that enables Azure customers worldwide to quickly build, deploy and manage customized applications using the more than 100 NVIDIA AI frameworks and tools that come fully supported in NVIDIA AI Enterprise, the software layer of NVIDIA's AI platform.

"With the coming wave of generative AI applications, enterprises are seeking secure accelerated tools and services that drive innovation," said Manuvir Das, vice president of enterprise computing at NVIDIA. "The combination of NVIDIA AI Enterprise software and Azure Machine Learning will help enterprises speed up their AI initiatives with a straight, efficient path from development to production."

Ampere Computing today announced a new AmpereOne Family of processors with up to 192 single threaded Ampere cores - the highest core count in the industry. This is the first product from Ampere based on the company's new custom core, built from the ground up and leveraging the company's internal IP. CEO Renée James, who founded Ampere Computing to offer a modern alternative to the industry with processors designed specifically for both efficiency and performance in the Cloud, said there was a fundamental shift happening that required a new approach.

"Every few decades of compute there has emerged a driving application or use of performance that sets a new bar of what is required of performance," James said. "The current driving uses are AI and connected everything combined with our continued use and desire for streaming media. We cannot continue to use power as a proxy for performance in the data center. At Ampere, we design our products to maximize performance at a sustainable power, so we can continue to drive the future of the industry."

Intel and SAP SE today announced a strategic collaboration to deliver more powerful and sustainable SAP software landscapes in the cloud. Designed to help customers derive greater scalability, agility and consolidation of existing SAP software landscapes, the collaboration deepens Intel's focus on delivering extremely powerful and secure instances for SAP, powered by 4th Gen Intel Xeon Scalable processors.

Using SAP Application Performance Standard benchmarks, Intel's 4th Gen Xeon processors enable significantly higher performance numbers when compared to previous generations of Xeon processors, and these impressive results will be passed along to SAP customers around the globe. Additionally, Intel enables current virtual machine (VM) sizes up to 24 TB with a goal to ramp up to VM sizes of 32 TB with the RISE with SAP solution.

Microsoft Corp. today announced the following results for the quarter ended March 31, 2023, as compared to the corresponding period of last fiscal year:

• Revenue was $52.9 billion and increased 7% (up 10% in constant currency)
• Operating income was $22.4 billion and increased 10% (up 15% in constant currency)
• Net income was $18.3 billion and increased 9% (up 14% in constant currency)
• Diluted earnings per share was $2.45 and increased 10% (up 14% in constant currency)

"The world's most advanced AI models are coming together with the world's most universal user interface - natural language - to create a new era of computing," said Satya Nadella, chairman and chief executive officer of Microsoft. "Across the Microsoft Cloud, we are the platform of choice to help customers get the most value out of their digital spend and innovate for this next generation of AI."

According to The Information, Microsoft has been working on creating custom processors for processing AI with a project codenamed Athena. Based on TSMC's 5 nm process, these chips are designed to accelerate AI workloads and scale to hundreds or even thousands of chips. With the boom of Large Language Models (LLMs) that require billions of parameters, training them requires a rapid increase of computational power to a point where companies purchase hundreds of thousands of GPUs from the likes of NVIDIA. However, creating custom processors is a familiar feat for a company like Microsoft. Hyperscalers like AWS, Google, and Meta are already invested in the creation of processors for AI training, and Microsoft is just joining as well.

While we don't have much information about these processors, we know that Microsoft started the project in 2019, and today these processors are in the hands of select employees of Microsoft and OpenAI that work with AI projects and need computational horsepower. Interestingly, some projections assume that if Microsoft could match NVIDIA's GPU performance, the cost would only be a third of NVIDIA's offerings. However, it is challenging to predict that until more information is provided. Microsoft plans to make these chips more widely available as early as next year; however, there is no specific information on when and how, but Azure cloud customers would be the most logical place to start.

Western Digital has declared that its My Cloud online service has been compromised by a group of hackers late last month: "On March 26, 2023, Western Digital identified a network security incident involving Western Digital's systems. In connection with the ongoing incident, an unauthorized third party gained access to a number of the Company's systems. Upon discovery of the incident, the Company implemented incident response efforts and initiated an investigation with the assistance of leading outside security and forensic experts. This investigation is in its early stages and Western Digital is coordinating with law enforcement authorities."

The statement, issued on April 4, continues: "The Company is implementing proactive measures to secure its business operations including taking systems and services offline and will continue taking additional steps as appropriate. As part of its remediation efforts, Western Digital is actively working to restore impacted infrastructure and services. Based on the investigation to date, the Company believes the unauthorized party obtained certain data from its systems and is working to understand the nature and scope of that data. While Western Digital is focused on remediating this security incident, it has caused and may continue to cause disruption to parts of the Company's business operations."

Qualcomm Technologies, Inc. today announced the world's first integrated 5G IoT processors that are designed to support four major operating systems, in addition to two new robotics platforms, and an accelerator program for IoT ecosystem partners. These new innovations will empower manufacturers participating in the rapidly expanding world of devices at the connected intelligent edge.

The need for connected, intelligent, and autonomous devices is growing rapidly, and it is expected to hit $116 billion by 2030 according to Precedence Research. Businesses attempting to compete in this fast-moving economy need a reliable source of control and connectivity technology for their IoT and robotic devices. Qualcomm Technologies, which has shipped over 350 million dedicated IoT chipsets, is uniquely capable of providing manufacturers with the platforms needed to address this expanding segment.

Microsoft Azure announced their new ND H100 v5 virtual machine which packs Intel's Sapphire Rapids Xeon Scalable processors with NVIDIA's Hopper H100 GPUs, as well as NVIDIA's Quantum-2 CX7 interconnect. Inside each physical machine sits eight H100s—presumably the SXM5 variant packing a whopping 132 SMs and 528 4th generation tensor cores—interconnected by NVLink 4.0 which ties them all together with 3.6 TB/s bisectional bandwidth. Outside each local machine is a network of thousands more H100s connected together with 400 GB/s Quantum-2 CX7 InfiniBand, which Microsoft says allows 3.2 Tb/s per VM for on-demand scaling to accelerate the largest AI training workloads.

Generative AI solutions like ChatGPT have accelerated demand for multi-ExaOP cloud services that can handle the large training sets and utilize the latest development tools. Azure's new ND H100 v5 VMs offer that capability to organizations of any size, whether you're a smaller startup or a larger company looking to implement large-scale AI training deployments. While Microsoft is not making any direct claims for performance, NVIDIA has advertised H100 as running up to 30x faster than the preceding Ampere architecture that is currently offered with the ND A100 v4 VMs.

According to TrendForce's latest survey of the server market, many cloud service providers (CSPs) have begun large-scale investments in the kinds of equipment that support artificial intelligence (AI) technologies. This development is in response to the emergence of new applications such as self-driving cars, artificial intelligence of things (AIoT), and edge computing since 2018. TrendForce estimates that in 2022, AI servers that are equipped with general-purpose GPUs (GPGPUs) accounted for almost 1% of annual global server shipments. Moving into 2023, shipments of AI servers are projected to grow by 8% YoY thanks to ChatBot and similar applications generating demand across AI-related fields. Furthermore, shipments of AI servers are forecasted to increase at a CAGR of 10.8% from 2022 to 2026.

NVIDIA has recently reported its Q4 earnings, and the earnings call following the report contains exciting details about the company and its plans to open up to new possibilities. NVIDIA's CEO Jensen Huang has stated that the company is on track to become an AI-as-a-Service (AIaaS) provider, which technically makes it a cloud service provider (CSP). "Today, I want to share with you the next level of our business model to help put AI within reach of every enterprise customer. We are partnering with major service -- cloud service providers to offer NVIDIA AI cloud services, offered directly by NVIDIA and through our network of go-to-market partners, and hosted within the world's largest clouds." Said Mr. Huang, adding that "NVIDIA AI as a service offers enterprises easy access to the world's most advanced AI platform, while remaining close to the storage, networking, security and cloud services offered by the world's most advanced clouds. Customers can engage NVIDIA AI cloud services at the AI supercomputer, acceleration library software, or pretrained AI model layers."

In addition to enrolling other CSPs into the race, NVIDIA is also going to offer DGX machines on demand in the cloud. Using select CSPs, you can get access to an entire DGX and harness the computing power for AI research purposes. Mr. Huang noted "NVIDIA DGX is an AI supercomputer, and the blueprint of AI factories being built around the world. AI supercomputers are hard and time-consuming to build. Today, we are announcing the NVIDIA DGX Cloud, the fastest and easiest way to have your own DGX AI supercomputer, just open your browser. NVIDIA DGX Cloud is already available through Oracle Cloud Infrastructure and Microsoft Azure, Google GCP, and others on the way."

Microsoft's Windows Server 2008 and 2008 R2 customers still represent a large group, as Microsoft has announced an additional year of Extended Security Updates (ESU) with a caveat. Only available for Microsoft Azure customers, the ESU program will allow Windows Server 2008 and R2 users on Azure cloud to get security updates until January 9, 2024. By no means is this not a free program, and Microsoft will bill this extensively as it is available internationally. Many customers are forced to join the ESU program for their Windows Server 2008 and R2 systems, as upgrading the OS to the latest version is not always possible without significant downtime or a hardware update.

The following customer base has legibility to the fourth year of the ESU program:

• Windows Server 2008 R2 Service Pack 1 (SP1)
• Windows Server 2008 Service Pack 2 (SP2)
• Windows Embedded POSReady 7
• Windows Embedded Standard 7
• All Azure virtual machines (VMs) running Windows Server 2008 R2 and Windows Server 2008 operating systems on Azure, Azure Stack, Azure VMWare Solutions, or Azure Nutanix Solution.

Today, we are announcing the third phase of our long-term partnership with OpenAI through a multiyear, multibillion dollar investment to accelerate AI breakthroughs to ensure these benefits are broadly shared with the world.

This agreement follows our previous investments in 2019 and 2021. It extends our ongoing collaboration across AI supercomputing and research and enables each of us to independently commercialize the resulting advanced AI technologies.

IonQ, Inc. (NYSE: IONQ), an industry leader in quantum computing, today announced plans to open the first known dedicated quantum computing manufacturing facility in the U.S., located in the suburbs of Seattle, Washington. The new facility will house IonQ's growing R&D and manufacturing teams, as they develop systems to meet continued customer demand. With public support from U.S. Senator Patty Murray (D-WA) - an early proponent of the CHIPS and Science Act - and Congresswoman Suzan DelBene, US representative from Washington's 1st congressional district,today's announcement is part of IonQ's broader intent to invest $1 billion through expansion in the Pacific Northwest over the next 10 years.

"IonQ making the decision to open the first ever quantum computing manufacturing facility in the country right here in Bothell is a very big deal—and it's great news for Washington state," said Senator Murray. "Opening this facility will absolutely help ensure Washington state continues to be a leader in innovation and cutting-edge technologies—but it also means jobs that will be an investment in our families and their futures. These are the kinds of investments that happen when we pass legislation like the CHIPS and Science Act to invest in American manufacturing and build the economy of the future right here at home."

Amidst the global economic downturns, Microsoft is reportedly joining other tech giants in reducing the amount of the company's working staff. According to Sky News, citing its sources, Microsoft will lay off as many as 5% of its workers. The company's massive team of over 220,000 employees will affect a large group estimated to be close to 11,000 people. In addition, Sky News' Wall Street Analyst source suggests that the people familiar with the matter would not be surprised if the reported figure is higher. If finalized, the decision is expected to be made official by Microsoft's chairman and CEO, Satya Nadella, on January 24.

The company's current market capitalization is $1.79 trillion, making it one of the world's most valuable companies. However, more than the investment in expanding Azure cloud services is needed to offset the stagnating consumer segment where Microsoft dominates with its Windows and Office services, so the company is forced to cut a part of its workforce. We don't have exact details on which segment is getting the highest deduction in staff; however, we expect to hear more at the company's Q4 2022 results call on January 24.

Today, we shared our vision for the next era of the Windows PC, where the PC and the cloud intersect and tap into innovative AI technology that unlocks new experiences. So that each of us can participate, be seen, heard and express our creativity.

For nearly 40 years, the Windows PC has held a place at the center of our lives. It's contributed to new levels of productivity, kept us all connected, and unlocked our creativity and potential through innovations we couldn't have imagined when we first began this journey. Just think about how far we've come in how people interact with it. From the very first text-based keyboard input to the precision of point and click with the mouse, up to today, where touch, voice, pen and gestures all help people use the Windows PC more naturally and intuitively. From its inception, Surface has been a catalyst for that change.

AAEON, an industry leading developer of edge computing platforms, is happy to announce that it has joined the Azure Certified Device program, ensuring customers get IoT solutions up and running quickly with hardware and software that has been pre-tested and verified to work with Azure IoT. This certification assures that AAEON's new NVIDIA Jetson AGX Orin -powered BOXER-8641AI has been tested for functionality and interoperability, ensuring that it has used the Microsoft reference configuration to become one of the market's first Azure-certified NVIDIA Jetson Orin devices.

This announcement illustrates AAEON's continued innovation in the edge AI space, as the BOXER-8641AI has been validated to work with Azure and the latest versions of IoT Edge. Such a certification gives AAEON customers confidence that when choosing the BOXER-8641AI for their application, it is fully IoT Edge compatible.

AMD announced that the AMD Pensando Distributed Services Card, powered by the industry's most advanced data processing unit (DPU)1, will be one of the first DPU solutions to support VMware vSphere 8 available from leading server vendors including Dell Technologies, HPE and Lenovo.

As data center applications grow in scale and sophistication, the resulting workloads increase the demand on infrastructure services as well as crucial CPU resources. VMware vSphere 8 aims to reimagine IT infrastructure as a composable architecture with a goal of offloading infrastructure workloads such as networking, storage, and security from the CPU by leveraging the new vSphere Distributed Services Engine, freeing up valuable CPU cycles to be used for business functions and revenue generating applications.

Microsoft is announcing the general availability of the latest Azure Virtual Machines featuring the Ampere Altra Arm-based processor. The new virtual machines will be generally available on September 1, and customers can now launch them in 10 Azure regions and multiple availability zones around the world. In addition, the Arm-based virtual machines can be included in Kubernetes clusters managed using Azure Kubernetes Service (AKS). This ability has been in preview and will be generally available over the coming weeks in all the regions that offer the new virtual machines.

Earlier this year, we launched the preview of the new general-purpose Dpsv5 and Dplsv5 and memory optimized Epsv5 Azure Virtual Machine series, built on the Ampere Altra processor. These new virtual machines have been engineered to efficiently run scale-out, cloud-native workloads. Since then, hundreds of customers have tested and experienced firsthand the excellent price-performance that the Arm architecture can provide for web and application servers, open-source databases, microservices, Java and.NET applications, gaming, media servers, and more. Starting today, all Azure customers can deploy these new virtual machines using the Azure portal, SDKs, API, PowerShell, and the command-line interface (CLI).

Microsoft Corp. today announced the following results for the quarter ended June 30, 2022, as compared to the corresponding period of last fiscal year:

• Revenue was $51.9 billion and increased 12% (up 16% in constant currency)
• Operating income was $20.5 billion and increased 8% (up 14% in constant currency)
• Net income was $16.7 billion and increased 2% (up 7% in constant currency)
• Diluted earnings per share was $2.23 and increased 3% (up 8% in constant currency)

"We see real opportunity to help every customer in every industry use digital technology to overcome today's challenges and emerge stronger," said Satya Nadella, chairman and chief executive officer of Microsoft. "No company is better positioned than Microsoft to help organizations deliver on their digital imperative - so they can do more with less."

Hyperscalers such as Microsoft, Google, Amazon, etc., all run their cloud divisions with a specific goal. To provide their hardware to someone else in a form called instance and have the user pay for it by the hour. However, instances are usually bound by a specific CPU and memory configuration, which you can not configure yourself. But instead, you can only choose from the few available options that are listed. For example, when selecting one virtual CPU core, you get two GB of RAM and can go as high as you want with CPU cores. However, the available RAM will also double, even though you might not need it. When renting an instance, the allocated CPU cores and memory are yours until the instance is turned off.

And it is precisely this that hyperscalers are dealing with. Many instances don't fully utilize their DRAM, making the whole data center usage inefficient. Microsoft Azure, one of the largest cloud providers, measured that 50% of all VMs never touch 50% of their rented memory. This makes memory stranded in a rented VM, making it unusable for anything else.

At Azure, we find that a major contributor to DRAM inefficiency is platform-level memory stranding. Memory stranding occurs when a server's cores are fully rented to virtual machines (VMs), but unrented memory remains. With the cores exhausted, the remaining memory is unrentable on its own, and is thus stranded. Surprisingly, we find that up to 25% of DRAM may become stranded at any given moment.

The recent semiconductor shortage has put an unprecedented amount of focus on the industry. Both commercial and government entities have come to recognize the lack of advanced node semiconductor manufacturing capabilities onshore in the United States. Intel Foundry Services (IFS) entry into the commercial foundry space is poised to change all that. As part of IFS Accelerator program, Intel recently announced their new IFS Cloud Alliance program, with Microsoft Azure as one of the inaugural members.

This is the latest chapter in a partnership between Intel and Microsoft that stretches back decades all the way back to the early days of the personal computer. In the last few years, Intel and Microsoft have collaborated on advancing semiconductor design on the cloud by working together to bring out EDA centric cloud compute such as the FX series on Azure, working with EDA vendors to enhance their software to better take advantage of the elasticity of the Azure cloud, as well as collaborating on a secure cloud-based semiconductor development platform for the US Department of Defense RAMP and RAMP-C programs.

Today, Microsoft launches Azure Virtual Machines (VM) based on the Ampere Altra Cloud Native Processor. This marks an important milestone as developers can now take advantage of these modern high-performance VMs for their existing and greenfield applications. The Ampere Altra processor family leads in performance across a range of broadly deployed cloud workloads and is now making available the Arm architecture on Azure.

Industry leading performance and the most sustainable solution
Cloud users who have pushed the limits of legacy x86 architectures now have a high-performance compute alternative that scales up in a linear fashion and delivers predictable performance even at full utilization. For example, Ampere Altra VMs outperform equivalently sized Intel and AMD instances from the same generation by 39% and 47%, respectively.* In addition to being the high-performance choice, Ampere Altra processors are extremely power efficient, directly reducing users' overall carbon footprint.

IonQ, a leader in quantum computing, today announced that it had signed an agreement with Microsoft to bring IonQ Aria to the Azure Quantum platform. The partnership will add IonQ Aria, the company's latest quantum system, to the cloud platform which already features IonQ's prior generation of systems among the lineup of available hardware. IonQ Aria is IonQ's most advanced commercially available quantum computer. Featuring 20 Algorithmic Qubits (#AQ), it is also the industry's most powerful quantum computer based on standard application-oriented industry benchmarks. Through this partnership, anyone with an internet connection will be able to harness IonQ Aria's abilities, furthering the democratization of quantum computing.

"We're excited to bring IonQ Aria's leading capabilities to more customers through Microsoft Azure and our Expanded Beta program," said IonQ President and CEO Peter Chapman. "We believe the future of quantum computing relies on getting the power of today's systems into the hands of as many people as possible, and building on our existing partnership with Microsoft is an important step along that path."