Framework architecture

The video framework enables guests running in virtual machines (VMs) to access the VPU in the hypervisor host system.

The framework provides a virtual device (vdev), virtio-video-decoder, that offers the functionality of a physical VPU to client applications so they can decode video. This vdev is based on draft version 3 of the virtio-video specification, which is a proposed amendment to the existing VirtIO 1.1 specifications. (Although this draft version supports video encoding, this isn't supported in this virtualization frameworks release; only decoding is.)

VPU sharing

The host manages all interactions between guests and its VPU:

• The hypervisor host runs a service, vcodec-*, to provide a vendor-independent means for accessing the VPU through an OpenMAX IL interface. This service communicates with the VPU that's specific to the hardware vendor (OEM). Therefore, you must run the variant designed for your underlying hardware. For instance, you would run vcodec-imx8 on an NXP i.MX8QM board and vcodec-sa8155 on a Qualcomm SA8155P board.
• If a guest requires access to the host's VPU, the VM (i.e., qvm process) that hosts the guest must include the video decoder vdev (virtio-video-decoder). To interact with the vdev, the guest runs a video decoder driver that complies with the virtio-video message standard.
• To communicate with each other, the guest driver and the vdev use virtqueues, which are the standard mechanism for bulk data transport on VirtIO devices.
• Guest applications continue to use their existing media codec interface. The video decoding requests go through the driver and the vdev, and are handled by the vcodec-* service. In Android guests, the v4l2_codec2 layer receives these requests from applications and translates the requests into a format that the virtio-video driver can use.

  Only certain video decoding formats are supported by our vdev. This limitation is due to which decoders are implemented in hardware on the host VPU, but also due to the formats supported by the guest's decoder subsystem (e.g., v4l2_codec2 on Android). Unsupported video formats are decoded in software locally on the guest, rather than in hardware on the host. For the list of supported video formats, see the Release Notes.

Note: The video framework does not provide display services. Additional mechanisms are required for playing video that has been decoded. For example, the guest could process frames and then render them using the Shared GPU and Display framework.

The diagram below illustrates the interaction between guests and the host system's VPU, managed by the host through the virtio-video-decoder vdev and vcodec-* service.

Figure 1. Video framework architecture

Note: For more information on device sharing in the QNX Hypervisor, see the section on “Devices” in the “Understanding QNX Virtual Environments” chapter of the QNX Hypervisor User's Guide.