In April 2021, the Vulkan Working Group at Khronos released a set of provisional extensions, collectively referred to as Vulkan Video which provide seamless encoding and decoding of video streams using a variety of video coding standards. The December 2022 release of Vulkan 1.3.238 saw the finalization of the extensions to decode H.264 and H.265, and today, with the release of Vulkan 1.3.274, Khronos has finalized their counterpart: the extensions to enable encoding of H.264 and H.265 video streams. Leveraging the Vulkan framework, they provide a standardized, seamless, low-overhead, and highly controllable way to produce H.264 and H.265 video via hardware accelerators, with applications ranging from real-time, low-latency streaming to offline server-scale transcoding.

Incorporating industry feedback, the extensions saw many improvements since their introduction, from a bidirectional interface (overrides) to help with coding and exposing advanced hardware capabilities, to rate control configuration parameters and an interface to aid with quality vs. performance trade-offs. This feedback also prompted the release of the first video maintenance extension. In addition, given the high industry demand for AV1 codec support, an AV1 decode extension release is imminent, with an AV1 encode extension development also underway. Figure 1 depicts Vulkan Video extensions along with their status and relations.

In April 2021, the Vulkan Working Group at Khronos released a set of provisional extensions, collectively called "Vulkan Video," for seamlessly integrating hardware-accelerated video compression and decompression into the Vulkan API. Today, Khronos is releasing finalized extensions that incorporate industry feedback and expose core and decode Vulkan Video functionality to provide fully accelerated H.264 and H.265 decode.

Khronos will release an ongoing series of Vulkan Video extensions to enable additional codecs and accelerated encode as well as decode. This blog is a general overview of the Vulkan Video architecture and also provides details about the finalized extensions and links to important resources to help you create your first Vulkan Video applications.