QNX Technical Articles

Date of this edition: February 25th, 2021

The Multimedia for QNX SDP 7.1 is middleware that facilitates the development of applications that can encode and play media content using the mm-renderer or OpenMAX AL interface, and can stream and play content over networks.

You can install this middleware software onto a development host that contains an installation of QNX SDP 7.1. The following host OSs are supported:

• Microsoft Windows 10 Pro 64-bit, or Windows 8.1 Pro 64-bit
• macOS version 10.14, 10.15
• Linux Red Hat Enterprise Linux 7 64-bit, or Ubuntu Desktop 18.04 LTS 64-bit, on x86_64 processors

  Note: QNX SDP isn't supported on Linux on ARM processors

---

Changes to these notes since January 21st, 2021 are highlighted below with this icon:

---

This release lets you encode and/or play audio and video files on the following tested target platforms:

• Generic x86_64
• NXP I.MX8QM
• Renesas R-Car V3H
• VMWare (x86-64)

Throughout this document, you may see reference numbers associated with particular issues, changes, etc. When corresponding with our Technical Support staff about a given issue, please quote the relevant reference number. You might also find the reference numbers useful for tracking issues as they become fixed.

Contents

• What's in this release?
• Supported file formats and audio and video codecs
• Fixed issues
• Descoped items
• Known issues
• Getting started with the documentation
• Technical support

What's in this release?

This release includes the following packages, which you'll find under Available > QNX Software Development Platform in the QNX Software Center.

To get all of the supported multimedia services and utilities, you must install these packages found in Custom Configuration:

• QNX SDP 7.1 Multimedia - Encode and Casting:
  • com.qnx.qnx710.target.mm.encode_cast.group_2.1.0.00170T202012081729S
• QNX SDP 7.1 Multimedia - Playback:
  • com.qnx.qnx710.target.mm.playback.group_2.1.0.00170T202012081729S

---

Installing either of the above packages installs the documentation package, com.qnx.qnx710.target.sf_mm.docs.plugins_0.0.3.00006T202012101054S. This package contains all Multimedia as well as Sensor Framework documents.

---

To support video playback on specific platforms, you can install these packages found in Hardware Support:

• QNX SDP 7.1 Multimedia - Freescale i.MX8 Media CODEC Support:
  • com.qnx.qnx710.target.mm.hw.nxpimx8.group_2.1.0.00170T202012081729S
• QNX SDP 7.1 Multimedia - Intel Apollo Lake Media CODEC Support:
  • com.qnx.qnx710.target.mm.hw.intel.group_2.1.0.00170T202012081729S

For source code examples that use different multimedia services, you can install these packages found in Source Bundles:

• QNX SDP 7.1 Multimedia - Media Renderer API Examples Source Code:
  • com.qnx.qnx710.target.mm.renderer.examples.source_2.1.0.00170T202012081729S
• QNX SDP 7.1 Multimedia - OpenMAX AL Examples Source Code:
  • com.qnx.qnx710.target.mm.openmaxal.examples.source_2.1.0.00170T202012081729S
• QNX SDP 7.1 Multimedia - Streaming and Casting Server API Examples Source Code:
  • com.qnx.qnx710.target.mm.stream.examples.source_2.1.0.00170T202012081729S

To see a list of the contents of a package, right-click it in the QNX Software Center, choose Properties, and then click Package Contents.

Supported file formats and audio and video codecs

Playback file format support

The Multimedia for QNX SDP 7.1 supports the following file containers and associated file extensions for audio/video playback on supported hardware platforms:

Playback audio codec support

The following table lists the supported playback audio codecs (which aren't platform-specific):

Playback video codec support

Video codecs are implemented in hardware, so they're platform-specific. The supported playback video codecs are:

• Generic x86_64 — H.265, H.264, VP-9
• NXP I.MX8QM — H.264

Encoding file format support

The Multimedia for QNX SDP 7.1 supports the following file containers and associated file extensions for audio/video encoding on supported hardware platforms:

Encoding audio codec support

The following table lists the supported encoding audio codecs (which aren't platform-specific):

Encoding video codec support

The H.264 encoding video codec is supported on all hardware platforms supported by Multimedia for QNX SDP 7.1:

• Generic x86_64
• NXP i.MX8QM
• Renesas R-Car V3H

Fixed issues

This release fixes the following issues that were present in previous releases of the multimedia software, including QNX Multimedia Suite 2.0:

• The mm-renderer context parameters for minimum and maximum latency for audio playback are now correctly named in the documentation.

  In the Defining Parameters section of the Multimedia Renderer Developer's Guide, these parameters are spelled properly as minlatency and maxlatency. (Ref# J2169068)

Descoped items

The multimedia services and functionality listed just below are not in scope for this release. If you require the services or functionality below, please contact QNX Engineering Services for support.

• Playback and encoding video codecs not listed above, including VC-1, H.263, Xvid, MPEG-2, MJPEG, and WMV9
• Playback and encoding audio codecs not listed above, including FLAC, ALAC, Vorbis, APE, Opus, and AMR (Narrowband and Wideband)
• Playback and encoding container and file formats not listed above, including AMR-WB (Wideband), AMR-NB (Narrowband), Opus, and H.263
• Some playback features and functionality that existed in previous releases, including:
  • Multi-track audio—switching between audio tracks during playback (e.g., for switching between languages)
  • Subtitle track switching—switching between subtitle track options
  • Audio trickplay—smooth audio trickplay (i.e., playing at non-standard speeds)
  • Streaming playback—media formats such as HTTP/HTTPs, HTTP Live Streaming v3 (HLS), Bluetooth A2DP, and RTP/RTPS
• Subtitles support for the following captioning standards for the specified containers:
  • VOBSUB (vobsub_decoder.so) in MP4/MOV and MKV files
  • SRT (srt_decoder.so) in MKV and Separate Caption File files
  • SMPTE-TT (smptett_decoder.so) in Separate Caption File files
• Multimedia management—support for all media management functionality that existed in the QNX Multimedia Suite 2.0 release, including:
  • Device detection and metadata synchronization to databases
  • Metadata synchronization and management
  • Playlist support for metadata synchronization to databases
  • Additional playlist formats, including .asx, .b4s, .pls, .rmp, and .xspf
• Multimedia device connectivity—support for device connectivity functionality that existed in the QNX Multimedia Suite 2.0 release, including:
  • Apple iOS: iAP1, iAP2 (the latter is required for CarPlay)
  • Android/Windows: MTP
• CarPlay demo—support for a full-screen CarPlay demo
• Android Auto demo—support for a full-screen Android Auto demo

Known issues

This release contains the known issues listed below. Not all items listed are bugs.

• In Adding Multimedia to Target Images, the Playback section lists the file /lib/dll/mmedia/shell_filter.so in Core Binaries. This file is not needed for any supported functionality and therefore does not need to be included in buildfiles and installed on targets. (Ref# J2895663)
• In Adding Multimedia to Target Images, the Encoding and casting section has an inaccurate description and table contents for Core Binaries:
  • The files listed in this table are required for encoding and casting, not metadata management (or synchronization); this latter feature isn't supported in this release.
  • The /lib/dll/mmedia/audio_writer.so and /lib/dll/mmedia/screen_writer.so files are used for playback and not encoding and casting, so they should not be listed here but instead in the Playback section.
  (Ref# J2895663)
• In Adding Multimedia to Target Images, the Encoding and casting section lists the file /lib/dll/mmedia/external_clock.so in Core Binaries. This is an experimental filter that is not needed for any supported functionality and therefore does not need to be included in buildfiles and installed on targets. (Ref# J2895663)
• In Adding Multimedia to Target Images, both the Playback and Encoding and casting sections improperly list certain binaries that are not shipped and therefore won't be on your host system. This also means that they should not be included in any buildfile that would try to install them onto a target system, because that would cause an error. These binaries were used for functionality that is no longer supported in this release. (Ref# J2895663, J2894777)

  This applies to the following files:

  • /lib/dll/mmedia/ape_parser.so, which provides APE file format support, in Feature-based Binaries
  • /lib/dll/mm-cli-mmplaylist.so, which is a command-line tool for the playlist parsing library, in Feature-based Binaries
• The NXP I.MX8QM platform has a known issue in which streaming video doesn't work. This is due to an NV12 tiled to NV12 conversion problem. (Ref# J2894092)
• If you use mm-stream to access only an audio stream through RTSP, the attempted connection fails. This is because the service improperly looks also for a video stream. (Ref# J2893918)
• You can't use mm-stream to stream video that is outputted locally by Screen. For instance, if you use mmrplay to play a video on the local display (by specifying a screen: URL as the output) and then start mm-stream with a configuration that specifies the local host as the peer address, nothing appears on the display. (Ref# J2893562)
• The sample applications that use the OpenMAX AL media engine don't properly log their operations. These applications compile and run successfully but because they use AoSetLogger() to set up logging, which doesn't work with libopenmaxal, they perform only partial logging. (Ref# J2893483)

  Workaround: Modify the code to use openmaxal_set_logger() instead of AoSetLogger().

Getting started with the documentation

After you've installed the QNX Software Center packages onto your development host, you can see the Multimedia documentation in the IDE help. To access this help, launch the IDE and select Help > Help Contents. In the left-side navigation, you'll see a top-level entry entitled QNX Software Development Platform. If you expand this entry, you'll see categories for QNX SDP documentation. You can expand the Multimedia category to see the following guides:

Adding Multimedia to Target Images

Describes the buildfile content needed to add multimedia support to a QNX Neutrino image.

Multimedia Renderer Developer's Guide

Explains how to write multimedia applications that use the mm-renderer service to control playback.

Multimedia Streaming Server Reference

Describes the JSON configuration and C API of the mm-stream server, which delivers media content over IP networks.

Multimedia Test Utilities Guide

Describes how to use the multimedia test utilities to prototype, test, and debug media applications.

QNX Additions to OpenMAX AL

Provides a reference of the QNX-extended OpenMAX AL API, which supports media format conversion, playback, and recording.

Technical support

To obtain technical support for any QNX product, visit the Support area on our website (www.qnx.com). You'll find a wide range of support options, including community forums.