New York-based security firm Trail of Bits has identified a security vulnerability with various GPU models, which include AMD, Qualcomm, and Apple. This vulnerability, named LeftoverLocals, could potentially allow attackers to steal large amounts of data from a GPU's memory. Mainstream client-GPUs form a sizable chunk of the hardware accelerating AI and LLMs, as they cost a fraction of purpose-built data-center GPUs, and are available in the retail market. Unlike CPUs, which have undergone extensive hardening against data leaks, GPUs were primarily designed for graphics acceleration and lack similar data privacy architecture. To our knowledge, none of the client GPUs use virtualization with their graphics memory. Graphics acceleration in general is a very memory sensitive application, and requires SIMD units to have bare-metal access to memory, with as little latency as possible.

First the good news—for this vulnerability to be exploited, it requires the attacker to have access to the target device with the vulnerable GPU (i.e. cut through OS-level security). The attack could break down data silos on modern computers and servers, allowing unauthorized access to GPU memory. The potential data breach could include queries, responses generated by LLMs, and the weights driving the response. The researchers tested 11 chips from seven GPU makers and found the vulnerability in GPUs from Apple, AMD, and Qualcomm. While NVIDIA, Intel, and Arm first-party GPUs did not show evidence of the vulnerability, Apple, Qualcomm, and AMD confirmed to wired that their GPUs are affected, and that they're working on a security response. Apple has released fixes for its latest M3 and A17 processors, but older devices with previous generations of Apple silicon remain vulnerable. Qualcomm is providing security updates, and AMD plans to offer mitigations through driver updates in March 2024.