NVIDIA's AI hardware dominance is challenged as the world's leading AI lab, OpenAI, taps into Google TPU hardware, showing significant efforts to move away from single-vendor solutions. In June 2025, OpenAI began leasing Google Cloud's Tensor Processing Units to handle ChatGPT's increasing inference workload. This is the first time OpenAI has relied on non-NVIDIA chips in large-scale production. Until recently, NVIDIA GPUs powered both model training and inference for OpenAI's products. Training large language models on those cards remains costly, but it was a periodic process. Inference, by contrast, runs continuously and carries its own substantial expense. ChatGPT now serves more than 100 million daily active users, including 25 million paid subscribers. Inference operations account for nearly half of OpenAI's estimated $40 billion annual compute budget. Google's TPUs, like v6e "Trillium" provide a more cost-effective solution for steady-state inference, as they are designed specifically for high throughput and low latency.

Beyond cost savings, this decision reflects OpenAI's desire to reduce reliance on any single vendor. Microsoft Azure has been its primary cloud provider since early investments and collaborations. However, GPU supply shortages and price fluctuations exposed a weakness in relying too heavily on a single source. By adding Google Cloud to its infrastructure mix, OpenAI gains greater flexibility, avoids vendor lock-in, and can scale more smoothly during usage peaks. For Google, winning OpenAI as a TPU customer offers strong validation for its in-house chip development. TPUs were once reserved almost exclusively for internal projects such as powering the Gemini model. Now, they are attracting leading organizations like Apple and Anthropic. Note that, beyond v6e inference, Google also designs TPUs for training (yet-to-be-announced v6p), which means companies can scale their entire training runs on Google's infrastructure on demand.

Our Stargate project, an unprecedented investment in America's AI infrastructure announced in January with President Trump and our partners Oracle and SoftBank, is now underway with our first supercomputing campus in Abilene, Texas, and more sites to come. We've heard from many countries asking for help in building out similar AI infrastructure—that they want their own Stargates and similar projects. It's clear to everyone now that this kind of infrastructure is going to be the backbone of future economic growth and national development. Technological innovation has always driven growth by helping people do more than they otherwise could—AI will scale human ingenuity itself and drive more prosperity by scaling our freedoms to learn, think, create and produce all at once.

We want to help these countries, and in the process, spread democratic AI, which means the development, use and deployment of AI that protects and incorporates long-standing democratic principles. Examples of this include the freedom for people to choose how they work with and direct AI, the prevention of government use of AI to amass control, and a free market that ensures free competition. All these things contribute to broad distribution of the benefits of AI, discourage the concentration of power, and help advance our mission. Likewise, we believe that partnering closely with the US government is the best way to advance democratic AI.

Razer, the leading global lifestyle brand for gamers, today announced the launch of its new line of productivity focused wireless mice, Razer Pro Click V2 Vertical Edition, and Razer Pro Click V2. Designed for users who want to integrate gaming products into their work setup, the new Razer Pro Click V2 mice emphasize ergonomic comfort, offering all-day support with gaming precision.

Ergonomic excellence for all-day comfort
Built with ergonomics in mind, Razer Pro Click V2 Vertical Edition is the brand's first wireless vertical ergonomic mouse. It features a 71.7º angle that mimics a natural handshake grip, reducing strain during prolonged use. The extended thumb rest keeps the hand relaxed as the base support elevates the wrist for smoother movements.

AI is rapidly reshaping what's possible on a PC—whether for real-time image generation or voice-controlled workflows. As AI capabilities grow, so does their complexity. Tapping into the power of AI can entail navigating a maze of system settings, software and hardware configurations. Enabling users to explore how on-device AI can simplify and enhance the PC experience, Project G-Assist—an AI assistant that helps tune, control and optimize GeForce RTX systems—is now available as an experimental feature in the NVIDIA app. Developers can try out AI-powered voice and text commands for tasks like monitoring performance, adjusting settings and interacting with supporting peripherals. Users can even summon other AIs powered by GeForce RTX AI PCs.

And it doesn't stop there. For those looking to expand Project G-Assist capabilities in creative ways, the AI supports custom plug-ins. With the new ChatGPT-based G-Assist Plug-In Builder, developers and enthusiasts can create and customize G-Assist's functionality, adding new commands, connecting external tools and building AI workflows tailored to specific needs. With the plug-in builder, users can generate properly formatted code with AI, then integrate the code into G-Assist—enabling quick, AI-assisted functionality that responds to text and voice commands.

Yesterday, OpenAI unveiled its new GPT-4.1 model family, which includes GPT-4.1, GPT-4.1 mini, and GPT-4.1 nano. These models promise significantly improved coding, instruction following, and long-context understanding compared to earlier versions like GPT-4o. All three support up to one million tokens of context, so they can handle much larger documents, codebases, or even hour-long video transcripts. In coding tests, GPT-4.1 scored 54.6% on the SWE-bench Verified benchmark, a jump of 21.4 points over GPT-4o. That makes it one of the strongest models for real-world software engineering tasks. It's better at exploring code repositories, creating patches that actually compile and pass tests, and following diff formats without extra edits. Areas like instruction following got a boost, too. On Scale's MultiChallenge benchmark, GPT-4.1 scored 38.3%, up 10.5 points from GPT-4o. That means it can stick to multi-step prompts more reliably and format its responses the way you ask.

The models' long-context skills stand out on the Video-MME benchmark, where GPT-4.1 scored 72.0% in the "long, no subtitles" category, 6.7 points higher than GPT-4o. Thanks to its one-million-token window, GPT-4.1 can pull together information spread across huge inputs, whether it's scattered text in a document or key moments in a video. OpenAI says these gains come from close work with developers, tuning the models for tasks that matter while cutting costs and latency. GPT-4.1 mini cuts cost 83% and nearly half the response time compared to GPT-4o. GPT-4.1 nano is even faster and cheaper, making it ideal for classification or autocompletion. All three models are available now through OpenAI's API. They won't appear directly in ChatGPT initially, though many of their improvements have already made it into the latest GPT-4o chatbot. Developers using the GPT-4.5 Preview should plan to switch over by July 14, 2025, when that version will be retired. While o3-mini remains the most powerful thinking model, the non-thinking GPT-4.1 family is closing the gap to deliver faster response times.

Generative AI is unlocking new capabilities for PCs and workstations, including game assistants, enhanced content-creation and productivity tools and more. NVIDIA NIM microservices, available now, and AI Blueprints, in the coming weeks, accelerate AI development and improve its accessibility. Announced at the CES trade show in January, NVIDIA NIM provides prepackaged, state-of-the-art AI models optimized for the NVIDIA RTX platform, including the NVIDIA GeForce RTX 50 Series and, now, the new NVIDIA Blackwell RTX PRO GPUs. The microservices are easy to download and run. They span the top modalities for PC development and are compatible with top ecosystem applications and tools.

The experimental System Assistant feature of Project G-Assist was also released today. Project G-Assist showcases how AI assistants can enhance apps and games. The System Assistant allows users to run real-time diagnostics, get recommendations on performance optimizations, or control system software and peripherals - all via simple voice or text commands. Developers and enthusiasts can extend its capabilities with a simple plug-in architecture and new plug-in builder.

Ask a typical IT professional today whether they're leveraging AI, and there's a good chance they'll say yes-after all, they have reputations to protect! Kidding aside, many will report that their teams may use Web-based tools like ChatGPT or even have internal chatbots that serve their employee base on their intranet, but for that not much AI is really being implemented at the infrastructure level. As it turns out, the true answer is a bit different. AI tools and techniques have embedded themselves firmly into standard enterprise workloads and are a more common, everyday phenomena than even many IT people may realize. Assembly line operations now include computer vision-powered inspections. Supply chains use AI for demand forecasting making business move faster and of course, AI note-taking and meeting summary is embedded on virtually all the variants of collaboration and meeting software.

Increasingly, critical enterprise software tools incorporate built-in recommendation systems, virtual agents or some other form of AI-enabled assistance. AI is truly becoming a pervasive, complementary tool for everyday business. At the same time, today's enterprises are navigating a hybrid landscape where traditional, mission-critical workloads coexist with innovative AI-driven tasks. This "mixed enterprise and AI" workload environment calls for infrastructure that can handle both types of processing seamlessly. Robust, general-purpose CPUs like the AMD EPYC processors are designed to be powerful and secure and flexible to address this need. They handle everyday tasks—running databases, web servers, ERP systems—and offer strong security features crucial for enterprise operations augmented with AI workloads. In essence, modern enterprise infrastructure is about creating a balanced ecosystem. AMD EPYC CPUs play a pivotal role in creating this balance, delivering high performance, efficiency, and security features that underpin both traditional enterprise workloads and advanced AI operations.

Apple today introduced the faster, more powerful iPad Air with the M3 chip and built for Apple Intelligence. iPad Air with M3 brings Apple's advanced graphics architecture to iPad Air for the first time—taking its incredible combination of power-efficient performance and portability to a new level. iPad Air with M3 is nearly 2x faster compared to iPad Air with M1, and up to 3.5x faster than iPad Air with A14 Bionic. Users will feel the speed of M3 in everything they do, from creating engaging content faster than ever to playing demanding, graphics-intensive games. Available in two sizes and four gorgeous finishes that users love, the 11-inch iPad Air is super portable while on the go, and the 13-inch model provides an even larger display for more room to be creative and productive. Designed for iPad Air, the new Magic Keyboard enhances its versatility and delivers more capabilities at a lower price. With iPadOS 18, support for Apple Intelligence, advanced cameras, fast wireless 5G connectivity, and compatibility with Apple Pencil Pro and Apple Pencil (USB-C), the new iPad Air offers an unrivaled experience.

With the same starting price of just $599 for the 11-inch model and $799 for the 13-inch model, the new iPad Air is a fantastic value. And for education, the 11-inch iPad Air starts at just $549, and the 13-inch model starts at just $749. Customers can pre-order the new iPad Air with M3 and Magic Keyboard for iPad Air starting today, with availability beginning Wednesday, March 12.

This afternoon at Mobile World Congress 2025 (MWC), Newnal Inc. will unveil another global breakthrough in personal blockchain and AI innovation, the Newnal personal AI operating system (OS). Built on a web3 foundation, Newnal's blockchain-based OS can be embedded and accessed via any device. Newnal gives people the power to design, drive and protect the experiences and opportunities they care about the most.

"With Newnal ai, each of us can now create another 'I'," said founder and Chief Creative Officer YT Kim, "The last generation of technology underserved humanity. Web3 ai Newnal gives the power of creativity and experience back to people, allowing each of us to enhance our lives in ways that we could only have imagined in the past. It's AI focused on our hearts."

The $3,499 Apple Vision Pro AR/VR headset is certainly not the kind of device that Apple expects to be mass-adopted. Since its launch in 2023, the luxurious gadget is yet to witness a single hardware refresh. The gadget's aging M2 SoC paired with 16 GB of memory is almost three years old, but despite that, respected analyst Mark Gurman expects the device to soon sport Apple Intelligence features alongside the upcoming VisionOS 2.4 update that is scheduled for April. Considering that a hardware update is not expected to hit the scene for at least the rest of the year, a major software update is welcome news for the select few of us who have invested in the product.

The fact that the Vision Pro will support Apple Intelligence features is actually not much of a shocker, since Macs as old as the M1-generation also boast support for Apple's AI features. As such, the Vision Pro will soon receive support for Apple's writing tools with ChatGPT-integration, genmoji, image playground, as well as a bunch of other features with debatable efficacy. Moreover, the report also adds that the device will also get a new app for spatial experiences - which was undoubtedly one of the key selling points of the device. A 'revamped' guest mode is also on the cards, which will allow buyers to share the device with their family and friends - a factor that Apple considers important in order to drive sales. That said, the Vision Pro had abysmal sales to begin with, which indicates that a lower-end variant with a reasonable price tag might be the only way to make even the most ardent of Apple disciples to hop on the VR/AR hype train.

Today, OpenAI has announced a new ChatGPT feature called "Deep Research," which is capable of performing complex, multi-step research processes entirely on its own. Using so-called agents, which are autonomous bots working on top of the AI model, this feature searches the web and curates all needed information. This agentic behavior was trained on real-world browser usage, accompanied by Python code execution. Deep search, like OpenAI's o1 and o3 models, uses reinforcement learning, which steps back to "think" and creates a chain of thought before delivering users an answer to their question. Depending on the topic, deep research can take 5 to 30 minutes to search the web, crawl through data, and compile it in a reader-friendly manner.

Regarding benchmarks of its performance, OpenAI put out a lot of interesting comparisons and evaluations. Compared to all previous models, deep research gives these models additional context to help AI with more information. Thus, in evaluation benchmarks like Humanity's Last exam, deep research scored 26.6%, whereas o1 and o3-mini scored 9.1 and 13%, respectively. Other evaluations showed a modest improvement, while concrete comparisons were made in UX, business, and medical research. Turning the deep research feature on delivered more information every time, and you can see it for yourself here.

A few days ago, we reported on the US reading a massive 500 billion US Dollar package called "Stargate Project" to build AI infrastructure on American soil. However, China is also planning to stay close behind, or even overlap the US in some areas, with a one trillion Yuan "AI Industry Development Action Plan". Translating into around 138 billion US Dollars at the time of writing, the Chinese AI plan is similar to the US Stargate project: develop AI infrastructure through data center expansion and AI accelerator scale-up. Unlike the Stargate project, led by private initiatives and OpenAI at the helm, the Chinese AI Industry Development Action Plan is an entirely state-sponsored initiative that will fund firms like Baidu, ByteDance, Alibaba, and DeepSeek with additional AI accelerators (or sanction-abiding GPUs) to create more advanced AI systems.

Over the past few days, DeepSeek, a branch of a Chinese hedge fund, has not only open-sourced its R1 reasoning model but made it entirely free to use for everyone. This has challenged moat of Western competitors like OpenAI, pushing its CEO Sam Altman to offer an O3-mini reasoning model for up to 100 queries per day for the ChatGPT Plus user tier. Not only did DeepSeek provide a model equally intelligent to OpenAI's best, but it also offered it completely free. This has stirred up the tech community quite a bit and showed that Chinese companies are not much behind Western competitors. With this AI action plan, the Chinese government wants to push its domestic AI makers even further ahead and allow them to overtake cutting-edge model development potentially. Of course, getting GPUs for these projects remains an intricate task, but with export control loopholes and domestic AI accelerator development, the AI arms race is picking up quite a bit.

NVIDIA today announced NVIDIA Cosmos, a platform comprising state-of-the-art generative world foundation models, advanced tokenizers, guardrails and an accelerated video processing pipeline built to advance the development of physical AI systems such as autonomous vehicles (AVs) and robots.

Physical AI models are costly to develop, and require vast amounts of real-world data and testing. Cosmos world foundation models, or WFMs, offer developers an easy way to generate massive amounts of photoreal, physics-based synthetic data to train and evaluate their existing models. Developers can also build custom models by fine-tuning Cosmos WFMs. Cosmos models will be available under an open model license to accelerate the work of the robotics and AV community. Developers can preview the first models on the NVIDIA API catalog, or download the family of models and fine-tuning framework from the NVIDIA NGC catalog or Hugging Face.

Microsoft has taken another significant step in its AI integration strategy by introducing "Terminal Chat," an AI assistant now available in Windows Terminal. This latest feature brings conversational AI capabilities directly to the command-line interface, marking a notable advancement in making terminal operations more accessible to users of all skill levels. The new feature, currently available in Windows Terminal (Canary), leverages various AI services, including ChatGPT, GitHub Copilot, and Azure OpenAI, to provide interactive assistance for command-line operations. What sets Terminal Chat apart is its context-aware functionality, which automatically recognizes the specific shell environment being used—whether it's PowerShell, Command Prompt, WSL Ubuntu, or Azure Cloud Shell—and tailors its responses accordingly.

Users can interact with Terminal Chat through a dedicated interface within Windows Terminal, where they can ask questions, troubleshoot errors, and request guidance on specific commands. The system provides shell-specific suggestions, automatically adjusting its recommendations based on whether a user is working in Windows PowerShell, Linux, or other environments. For example, when asked about creating a directory, Terminal Chat will suggest "New-Item -ItemType Directory" for PowerShell users while providing "mkdir" as the appropriate command for Linux environments. This intelligent adaptation helps bridge the knowledge gap between different command-line interfaces. Below are some examples courtesy of Windows Latest and their testing:

According to a recent Reuters report, OpenAI is continuing with its moves in the custom silicon space, expanding beyond its reported talks with Broadcom to include a broader strategy involving multiple industry leaders. Broadcom is a fabless chip designer known for a wide range of silicon solutions spanning from networking, PCIe, SSD controllers, and PHYs all the way up to custom ASICs. The company behind ChatGPT is actively working with both Broadcom and TSMC to develop its first proprietary AI chip, specifically focused on inference operations. Getting a custom chip to do training runs is a bit more complex task, and OpenAI leaves that to its current partners until the company figures out all details. Even with an inference chip, the scale at which OpenAI works and serves its models makes financial sense for the company to develop custom solutions tailored to its infrastructure needs.

This time, the initiative represents a more concrete and nuanced approach than previously understood. Rather than just exploratory discussions, OpenAI has assembled a dedicated chip team of approximately 20 people, led by former Google TPU engineers Thomas Norrie and Richard Ho. The company has secured manufacturing capacity with TSMC, targeting a 2026 timeline for its first custom-designed chip. While Broadcom's involvement leverages its expertise in helping companies optimize chip designs for manufacturing and manage data movement between chips—crucial for AI systems running thousands of processors in parallel—OpenAI is simultaneously diversifying its compute strategy. This includes adding AMD's Instinct MI300X chips to its infrastructure alongside its existing NVIDIA deployments. Similarly, Meta has the same approach, where it now trains its models on NVIDIA GPUs and serves them to the public (inferencing) using AMD Instinct MI300X.

A few days ago, we reported on AMD's newest expansion plans for Serbia. The company opened two new engineering design centers with offices in Belgrade and Nis. We were invited to join the opening ceremony and got an exclusive interview with one of AMD's top executives, Andrej Zdravkovic, who is the senior vice president and Chief Software Officer. Previously, we reported on AMD's transition to become a software company. The company has recently tripled its software engineering workforce and is moving some of its best people to support these teams. AMD's plan is spread over a three to five-year timeframe to improve its software ecosystem, accelerating hardware development to launch new products more frequently and to react to changes in software demand. AMD found that to help these expansion efforts, opening new design centers in Serbia would be very advantageous.

We sat down with Andrej Zdravkovic to discuss the purpose of AMD's establishment in Serbia and the future of some products. Zdravkovic is actually an engineer from Serbia, where he completed his Bachelor's and Master's degrees in electrical engineering from Belgrade University. In 1998, Zdravkovic joined ATI and quickly rose through the ranks, eventually becoming a senior director. During his decade-long tenure, Zdravkovic witnessed a significant industry shift as AMD acquired ATI in 2006. After a brief stint at another company, Zdravkovic returned to AMD in 2015, bringing with him a wealth of experience and a unique perspective on the evolution of the graphics and computing industry.Here is the full interview:

A new startup emerged out of stealth mode today to power the next generation of generative AI. Etched is a company that makes an application-specific integrated circuit (ASIC) to process "Transformers." The transformer is an architecture for designing deep learning models developed by Google and is now the powerhouse behind models like OpenAI's GPT-4o in ChatGPT, Anthropic Claude, Google Gemini, and Meta's Llama family. Etched wanted to create an ASIC for processing only the transformer models, making a chip called Sohu. The claim is Sohu outperforms NVIDIA's latest and greatest by an entire order of magnitude. Where a server configuration with eight NVIDIA H100 GPU clusters pushes Llama-3 70B models at 25,000 tokens per second, and the latest eight B200 "Blackwell" GPU cluster pushes 43,000 tokens/s, the eight Sohu clusters manage to output 500,000 tokens per second.

Why is this important? Not only does the ASIC outperform Hopper by 20x and Blackwell by 10x, but it also serves so many tokens per second that it enables an entirely new fleet of AI applications requiring real-time output. The Sohu architecture is so efficient that 90% of the FLOPS can be used, while traditional GPUs boast a 30-40% FLOP utilization rate. This translates into inefficiency and waste of power, which Etched hopes to solve by building an accelerator dedicated to power transformers (the "T" in GPT) at massive scales. Given that the frontier model development costs more than one billion US dollars, and hardware costs are measured in tens of billions of US Dollars, having an accelerator dedicated to powering a specific application can help advance AI faster. AI researchers often say that "scale is all you need" (resembling the legendary "attention is all you need" paper), and Etched wants to build on that.

Apple today previewed macOS Sequoia, the next version of the world's most advanced desktop operating system, bringing entirely new ways of working and transformative intelligence features to Mac. macOS Sequoia is full of exciting new capabilities, including iPhone Mirroring, which expands Continuity by enabling full access to and control of iPhone directly from macOS. Safari gets another big update with the new Highlights feature for effortless information discovery on webpages while browsing. The new Passwords app makes it even easier to access passwords and organize credentials all in one place. Gaming advances with a more immersive experience, as well as a breadth of new titles, including Assassin's Creed Shadows, Frostpunk 2, and more.

macOS Sequoia also introduces Apple Intelligence, the personal intelligence system for Mac, iPhone, and iPad that combines the power of generative models with personal context to deliver intelligence that's incredibly useful and relevant. Built with privacy from the ground up, Apple Intelligence is deeply integrated into macOS Sequoia, iOS 18, and iPadOS 18. It understands and creates language and images, takes action across apps, and draws from personal context, simplifying and accelerating everyday tasks. Taking full advantage of the power of Apple Silicon and the Neural Engine, Apple Intelligence will be supported by every Mac with an M-series chip.

Imagine you're playing a game, you're stuck somewhere, don't know how to craft something, can't manage your inventory, or need to get out of some tricky puzzle. You normally pause and minimize your game, pull up a web-browser, and sift through dozens of search results to find the right Wiki or community thread that solves your problem. Wouldn't it be great, if you could pull up an in-game chat window and interact with an AI chat assistant (not unlike ChatGPT), except, the assistant is situationally aware of your entire game-state? That is NVIDIA Project G-Assist.

NVIDIA is training an AI on the vast ocean of information from the Internet on the countless games there are. You will soon be able to pull up an AI chat assistant, and will be able to either give it a text or natural voice input on how you want it to help you in the situation of the game you're in, and it will be able to guide you. G-Assist will not play the game for you, just give you precise and concise answers based on your game state (something no Google search or game wiki can). Besides assisting your gameplay, G-Assist will also help you with performance optimization and system tuning, the way the NVIDIA App and GeForce Experience already do.

At OpenAI's spring update, a lot of eyes were fixed on the company, which spurred the AI boom with the ChatGPT application. Now being almost a must-have app for consumers and prosumers alike, ChatGPT is a de-facto application for the latest AI innovation, backed by researchers and scientists from OpenAI. Today, OpenAI announced a new model called GPT-4o (Omni), which hopes to bring advanced intelligence, improved overall capabilities, and real-time voice interaction with users. Now, the ChatGPT application wants to become like a personal assistant that actively communicates with users and provides much broader capabilities. OpenAI claims that it can respond to audio inputs as quickly as 232 milliseconds, with an average of 320 milliseconds, similar to human response time in conversations.

However, OpenAI states that it wants ChatGPT's latest GPT-4o model to be available to the free, Plus, and Team paid subscribers, where paid subscribers get 5x higher usage and early access to the model. Interestingly, the GPT-4o model is much improved across a variety of standard benchmarks like MMLU, Math, HumanEval, GPQA, and others, where it now surpasses almost all models except Claude 3 Opus in MGSM. It now understands more than 50 languages and can do real time translation. In addition to the new model, OpenAI announced that they are launching a desktop ChatGPT app, which can act as a personal assistant and see what is happening on the screen, but it is only allowed by user command. This is supposed to bring a much more refined user experience and enable users to use AI as a third person to help understand the screen's content. Initially only available on macOS, we are waiting for OpenAI to launch the Windows ChatGPT application so everyone can also experience the new technology.

To bring cutting-edge artificial intelligence capabilities to its flagship product, Apple is said to be finalizing a deal with OpenAI to integrate the ChatGPT technology into the upcoming iOS 18 for iPhones. According to Bloomberg, multiple sources report that after months of negotiations, the two tech giants are putting the finishing touches on a partnership that would be an important moment for consumer AI. However, OpenAI may not be Apple's only AI ally. The company has also reportedly been in talks with Google over licensing the Gemini chatbot, though no known agreement has been reached yet. The rare team-up between the fiercely competitive firms underscores the intense focus on AI integration across the industry.

Apple's strategic moves are a clear indication of its recognition of the transformative potential of advanced AI capabilities for the iPhone experience. The integration of OpenAI's language model could empower Siri to understand and respond to complex voice queries with deep contextual awareness. This could revolutionize the way Apple's customers interact with devices, offering hope for a more intuitive and advanced iPhone experience. Potential Gemini integration opens up another realm of possibilities around Google's image and multimodal AI capabilities. Future iPhones may be able to analyze and describe visual scenes, annotate images, generate custom imagery from natural language prompts, and even synthesize audio using AI vocals - all within a conversational interface. As the AI arms race intensifies, Apple wants to position itself at the forefront through these partnerships.

NVIDIA today unveiled ChatRTX, the AI assistant that runs locally on your machine, and which is accelerated by your GeForce RTX GPU. NVIDIA had originally launched this as "Chat with RTX" back in February 2024, back then this was regarded more as a public tech demo. We reviewed the application in our feature article. The ChatRTX rebranding is probably aimed at making the name sound more like ChatGPT, which is what the application aims to be—except it runs completely on your machine, and is exhaustively customizable. The most obvious advantage of a locally-run AI assistant is privacy—you are interacting with an assistant that processes your prompt locally, and accelerated by your GPU; the second is that you're not held back by performance bottlenecks by cloud-based assistants.

ChatRTX is a major update over the Chat with RTX tech-demo from February. To begin with, the application has several stability refinements from Chat with RTX, which felt a little rough on the edges. NVIDIA has significantly updated the LLMs included with the application, including Mistral 7B INT4, and Llama 2 7B INT4. Support is also added for additional LLMs, including Gemma, a local LLM trained by Google, based on the same technology used to make Google's flagship Gemini model. ChatRTX now also supports ChatGLM3, for both English and Chinese prompts. Perhaps the biggest upgrade ChatRTX is its ability to recognize images on your machine, as it incorporates CLIP (contrastive language-image pre-training) from OpenAI. CLIP is an LLM that recognizes what it's seeing in image collections. Using this feature, you can interact with your image library without the need for metadata. ChatRTX doesn't just take text input—you can speak to it. It now accepts natural voice input, as it integrates the Whisper speech-to-text NLI model.DOWNLOAD: NVIDIA ChatRTX

AI workloads are split into two different categories: training and inference. While training requires large computing and memory capacity, access speeds are not a significant contributor; inference is another story. With inference, the AI model must run extremely fast to serve the end-user with as many tokens (words) as possible, hence giving the user answers to their prompts faster. An AI chip startup, Groq, which was in stealth mode for a long time, has been making major moves in providing ultra-fast inference speeds using its Language Processing Unit (LPU) designed for large language models (LLMs) like GPT, Llama, and Mistral LLMs. The Groq LPU is a single-core unit based on the Tensor-Streaming Processor (TSP) architecture which achieves 750 TOPS at INT8 and 188 TeraFLOPS at FP16, with 320x320 fused dot product matrix multiplication, in addition to 5,120 Vector ALUs.

Having massive concurrency with 80 TB/s of bandwidth, the Groq LPU has 230 MB capacity of local SRAM. All of this is working together to provide Groq with a fantastic performance, making waves over the past few days on the internet. Serving the Mixtral 8x7B model at 480 tokens per second, the Groq LPU is providing one of the leading inference numbers in the industry. In models like Llama 2 70B with 4096 token context length, Groq can serve 300 tokens/s, while in smaller Llama 2 7B with 2048 tokens of context, Groq LPU can output 750 tokens/s. According to the LLMPerf Leaderboard, the Groq LPU is beating the GPU-based cloud providers at inferencing LLMs Llama in configurations of anywhere from 7 to 70 billion parameters. In token throughput (output) and time to first token (latency), Groq is leading the pack, achieving the highest throughput and second lowest latency.

OpenAI CEO, Sam Altman, had a turbulent winter 2023 career moment, but appears to be going all in with his company's future interests. A Bloomberg report suggests that the tech visionary has initiated a major fundraising initiative for the construction of OpenAI-specific semiconductor production plants. The AI evangelist reckons that his industry will become prevalent enough to demand a dedicated network of manufacturing facilities—the U.S. based artificial intelligence (AI) research organization is (reportedly) exploring custom artificial intelligence chip designs. Proprietary AI-focused GPUs and accelerators are not novelties at this stage in time—many top tech companies rely on NVIDIA solutions, but are keen to deploy custom-built hardware in the near future.

OpenAI's popular ChatGPT system is reliant on NVIDIA H100 and A100 GPUs, but tailor-made alternatives seem to be the desired route for Altman & Co. The "on their own terms" pathway seemingly skips an expected/traditional chip manufacturing process—the big foundries could struggle to keep up with demand for AI-oriented silicon. G42 (an Abu Dhabi-based AI development holding company) and SoftBank Group are mentioned as prime investment partners in OpenAI's fledgling scheme—Bloomberg proposes that Altman's team is negotiating a $8 to 10 billion deal with top brass at G42. OpenAI's planned creation of its own foundry network is certainly a lofty and costly goal—the report does not specify whether existing facilities will be purchased and overhauled, or new plants being constructed entirely from scratch.

Microsoft today announced the introduction of a new Copilot key devoted to its AI assistant on Windows PC keyboards. The key will provide instant access to Microsoft's conversational Copilot feature, offering a ChatGPT-style AI bot right from a button press. The Copilot key represents the first significant Windows keyboard change in nearly 30 years since the addition of the Windows key itself in the 90s. Microsoft sees it as similarly transformative - making AI an integrated part of devices. The company expects broad adoption from PC manufacturers starting this spring. The Copilot key will likely substitute keys like menu or Office on standard layouts. While currently just launching Copilot, Microsoft could also enable combo presses in the future.

The physical keyboard button helps make AI feel native rather than an add-on, as Microsoft aggressively pushes Copilot into Windows 11 and Edge. The company declared its aim to make 2024 the "year of the AI PC", with Copilot as the entry point. Microsoft envisions AI eventually becoming seamlessly woven into computing through system, silicon, and hardware advances. The Copilot key may appear minor, but it signals that profound change is on the horizon. However, users will only embrace the vision if Copilot proves consistently beneficial rather than gimmicky. Microsoft is betting that injecting AI deeper into PCs will provide usefulness, justifying the disruption. With major OS and hardware partners already committed to adopting the Copilot key, Microsoft's AI-first computer vision is materializing rapidly. The button press that invokes Copilot may soon feel as natural as hitting the Windows key or spacebar. As we await the reported launch of Windows 12, we can expect deeper integration with Copilot to appear.