QNX Technical Articles

Date of this edition: December 12, 2018

The QNX Multimedia Suite 2.0 contains the utilities, libraries, and codecs needed to develop applications that can synchronize media metadata from devices, play audio and video content, stream content over networks, and encode, play, and record content using the OpenMAX AL interface.

You can install this software suite onto a development host that contains an installation of QNX SDP 7. The following host OSs are supported:

• Microsoft Windows 10 Professional 64-bit, Windows 8 Professional 64-bit, or Windows 7 Professional 64-bit
• macOS version 10.10, 10.11, or 10.12
• Linux Red Hat Enterprise Linux 7 64-bit, or Ubuntu Desktop 16.04 LTS 64-bit, on x86_64 processors

---

QNX SDP 7 isn't supported on Linux on ARM processors. If you installed an experimental version of this software, uninstall it before installing the production version.

---

This release lets you play audio and video files on the following tested target platforms:

• Intel Apollo Lake
• Intel NUC
• Qualcomm Snapdragon 820
• Renesas R-Car H3 Starter
• Texas Instruments Jacinto 6 (TI J6)

For the most up-to-date version of these release notes, go to the QNX Software Center, right-click an item under the QNX Multimedia Suite category, select Properties, then click General Information. (Individual multimedia items will appear under that category in the Updates, Available, or Installed tab, depending on whether you've installed an earlier or the latest version of them.) The General Information tab contains a link to the latest release notes.

Contents

• What's in this release?
• Fixed issues
• Discontinued issues
• Known issues
• Getting started with the documentation
• Technical support

Throughout this document, you may see reference numbers associated with particular issues, changes, etc. When corresponding with our technical support staff about a given issue, please quote the relevant reference number. You might also find the reference numbers useful for tracking issues as they become fixed.

What's in this release?

This release includes the following packages, which you'll find under Available > QNX Multimedia Suite in the QNX Software Center. To get the full set of multimedia services and documentation, you must install all packages listed here.

In Documentation:

• QNX Multimedia Suite 2.0 - Documentation:
  • com.qnx.qnx700.target.mm.docs.plugins (2.0.381.S201812071608)

---

You should install the documentation package first, to ensure you get the latest version of the documentation right away. If you install any of the four software packages listed just below first, you'll get a slightly out-of-date version of the documentation (which gets automatically installed). You can easily replace this version with the updated version, but you must look for the latest documentation package in the Updates tab instead of the Available tab.

---

In Custom Configuration:

• QNX Multimedia Suite 2.0 - Encode and Casting:
  • com.qnx.qnx700.target.mm.encode_cast.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - Management:
  • com.qnx.qnx700.target.mm.management.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - MTP Support:
  • com.qnx.qnx700.target.mm.mtp.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - Playback:
  • com.qnx.qnx700.target.mm.playback.group (2.0.1557.S201806191908)

In Hardware Support:

• QNX Multimedia Suite 2.0 - Intel Apollo Lake Media CODEC Support:
  • com.qnx.qnx700.target.mm.hw.intel.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - Qualcomm 820A Media CODEC Support:
  • com.qnx.qnx700.target.mm.hw.Qualcomm820.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - Renesas H3 Media CODEC Support:
  • com.qnx.qnx700.target.mm.hw.renesash3.group (2.0.1557.S201806191908)
• QNX Multimedia Suite 2.0 - Texas Instruments AM572X/Jacinto 6 Media CODEC Support:
  • com.qnx.qnx700.target.mm.hw.ti_j6.group (2.0.1557.S201806191908)

To see a list of the contents of a package, right-click it in the QNX Software Center, choose Properties, and then click Package Contents.

Multimedia playback and file format support

The QNX Multimedia Suite supports the following file containers and associated file extensions for audio/video playback on supported hardware platforms:

Audio codec support

The following table provides details about the supported audio codecs (which aren't platform-specific):

Video codec support

Video codecs are implemented in hardware and so, they're platform-specific. The supported video codecs are:

• Intel Apollo Lake and Intel NUC — H.265, H.264, VP-9, VC-1, MPEG-2 video
• Qualcomm 820A — H.265, H.264, MPEG-4 video, MPEG-2 video, H.263, VC-1
• Renesas H3 — H.265, H.264, MPEG-4 video, MPEG-2 video, VC-1
• TI J6 — H.264, H.263, MPEG-4 video, MPEG-2 video, Motion JPEG, VC-1

Subtitle support

When playing videos, you can display subtitles in one of these formats:

• SubRip (SRT) — This is a text-based format in which the subtitles can be read either from a separate .srt file or directly from the video file.

  For the subtitles to be visible, your target system (where you play the video) must contain the right fonts. QNX SDP includes libraries that you can use with Screen to render fonts and configure them. Information about these libraries, including where they're found in the platform installation on the host system, is given in the Font Rendering section of the Screen Developer's Guide.

• TTML / SMPTE-TT — These text-based subtitles are read from a separate .ttml file.

  Again, you need to right fonts on your target system to see the subtitles; for details, see the previously referenced Screen documentation.

• VOBSUB — This is a bitmap-based format in which the subtitles are read from the video file.

Playlist support

This release of the QNX Multimedia Suite supports audio playlists, but not video playlists. You can play the following types of audio playlists:

Fixed issues

This release fixes the following issues that were present in the experimental releases of QNX Multimedia Suite 2.0 or in the previous releases of the multimedia software, notably QNX SDK for Apps and Media 1.X:

• When synchronizing media metadata, flexible file typing is done even when the sync_file_types extended option is used in the sync_start command or mm_sync_start() call.

  For example, suppose you set the ftype attribute to audio for a certain extension (say .mp4) in the mm-sync configuration file, then issue a sync_start command to synchronize the metadata from audio as well as video files with this extension. Suppose also that your command sets the sync_file_types extended option to audio. In this case, the files table rows storing the metadata for those synchronized files should all have their ftype field set to audio. It's been observed, however, that the file type gets changed to the actual type rather than the requested one (e.g., for .mp4 video files, this field is video, meaning the files were added to the database as video files).

  Workaround: After synchronizing the metadata from files of different media types, query the database to read the relevant files rows and if necessary, manually update the ftype field for files that you want treated as a different media type (e.g., audio instead of video).

  (Ref# J547703)

Discontinued items

We've discontinued support for the following items:

• The md_title_samplerate and md_video_* metadata attributes.

  For video files, some Metadata Provider (MDP) plugins didn't write the video frame rate into md_title_samplerate and instead wrote the audio sampling rate into it, meaning applications couldn't know what information this attribute contained. Using the mm-renderer API, your applications can now read the frame rates of individual streams within a media file. This is done by calling mmr_metadata_split() after receiving a METADATA event. From the returned properties, your code can then read the sample_rate attribute for audio streams and the frame_rate attribute for video streams.

  The metadata provider library (libmd) and the mm-renderer service used to publish these attributes:

  • md_video_width
  • md_video_height
  • md_video_pixel_width
  • md_video_pixel_height

  Your applications should use the Screen API to retrieve the video display properties of the relevant output windows.

Known issues

This release contains the known issues listed below. Not all items listed are bugs.

• Smooth audio trick play isn't enabled by defining vocoder alone. Although the intended design of mm-renderer is that this attribute alone should enable playing audio at non-normal speeds without skips (i.e., smooth trick play), if you uncomment vocoder in the default mm-renderer configuration file (i.e., /etc/mm/mm-renderer.conf), audio skips are observed unless you also define vocoderspeedlimit.

  Workaround: Uncomment the lines for both attributes in the configuration file.

  (Ref# J2577692)

• Audio playback is looped when switching between audio tracks within an input file. If you attach an input with multiple audio tracks (or streams), then start playback and switch between audio tracks, playback sometimes restarts from the beginning. It's not known whether this occurs only for certain formats of audio tracks.

  (Ref# J2578149)

• During playback of a video file with multiple audio tracks, audio playback stops sometimes if you switch to audio track 3 or higher. This issue was seen when playing Quicktime (.mov) video files on Qualcomm 820A and R-Car H3 target boards, and MP4 (.m4v) files on Qualcomm 820A boards.

  (Ref# J2579124, J2579313)

• File IDs are not always updated in the database when they should be.

  Sometimes, if you synchronize a media folder and then update the modification date of a file within it (e.g., using the touch command), then resynchronize the folder, you don't see a new database entry for the file (which would include a new ID). This problem was seen when neither the ChangedFilesHaveConstantId configuration setting nor the dynamic_folder command option was enabled, meaning that mm-sync should consider any files with a changed modification date as new and add new database entries with different fids (for media files) or plids (for playlists).

  (Ref# J2177197)

• The mmmd_get() function for the metadata provider library (libmd) doesn't always return the md_title_color_depth field. This field isn't returned when other fields are listed before it, as seen in the following example using the mmcli test utility:

  > mmmd_get 1 "/test_media/pictures/100_3829.JPG" "md_title::width,height,color_depth" 0
  078.956 Command 'mmmd_get':
  metadata: md_title_width::577
            md_title_height::768
  

  Workaround: Use a separate command to get the field in isolation:

  > mmmd_get 1 "/test_media/pictures/100_3829.JPG" "md_title::color_depth" 0 617.428 Command 'mmmd_get':
  metadata: md_title_color_depth::24
  

  (Ref# J1388409)

• The behaviour of libmd when returning artwork has changed from previous releases. In the configuration file, you can specify multiple regex settings for the external artwork plugin, Extart, which define patterns for matching names of image files:

  regex=album\.jp[e]?g
  regex1=Folder\.jp[e]?g
  

  In previous releases, the plugin retrieved the file that was the earliest match; in the example above, this would be album.jpg. But in the latest release, the plugin retrieves the file that is the latest match; Folder.jpg in our example.

  Workaround: If you have code that expects libmd to return a particular file that matches one of the regex patterns, you may need to adapt it to work with another file that is returned instead, or reorder the patterns defined for the regex entries in the configuration file.

  (Ref# J1263723)

• The mm-sync service crashes after you set a priority folder during the second pass of synchronization.

  Suppose you start a synchronization and wait for the MMSYNC_EVENT_MS_1PASSCOMPLETE event to ensure that the first pass (the files pass) has finished. Then, if you send a control command to immediately synchronize a particular folder (by setting the action extended option to priority_folder_set), the mm-sync service crashes. This behavior was seen when using the API (i.e., calling mm_sync_control()) but likely occurs with client commands (i.e., issuing sync_control from the client utility).

  (Ref# J1385445)

• The mm-renderer service crashes during trick play of MKV files.

  The rendering service sometimes crashes after you change the playing speed to non-standard values (i.e., enable trick play) several times. This issue was seen on Renesas H3 and Qualcomm boards, but could happen on other platforms.

  (Ref# J2624989)
• The mm-renderer context parameters for minimum and maximum latency for audio playback are incorrectly named in the documentation.

  In the Defining Parameters section of the Multimedia Renderer Developer's Guide, these parameters are spelled as min_latency and max_latency but they should be minlatency and maxlatency.

  Workaround: Use the parameter names without underscores when preparing the parameters that are passed to mmr_context_parameters().

  (Ref# J2169068)

Getting started with the documentation

After you've installed the documentation onto your development host, you can see the multimedia guides in the IDE help. To access this help, launch the IDE and select Help > Help Contents. In the left-side navigation, you'll see a top-level entry entitled QNX Multimedia Suite. You can expand this entry to see the following multimedia guides:

Adding Multimedia to Target Images

Describes the buildfile content needed to add multimedia support to a QNX Neutrino image.

Architecture Guide

Illustrates and explains the organization of the components in the QNX Multimedia Suite.

Dictionary Object API

Provides a reference of the API used to manage dictionary objects, which are used by some multimedia services.

io-fs-media Reference

Explains the io-fs-media service and how to configure media filesystem drivers through this service.

Metadata Provider Library Reference

Describes how to use the libmd library to extract metadata from media files on attached devices.

Multimedia Playlist Library Reference

Describes how to use the libmmplaylist library to read playlist files and support playlist operations.

Multimedia Renderer Developer's Guide

Explains how to write multimedia applications that use the mm-renderer service to control playback.

Multimedia Streaming Server Reference

Describes the JSON configuration and C API of the mm-stream server, which delivers media content over IP networks.

Multimedia Synchronizer Developer's Guide

Explains how to write multimedia applications that use the mm-sync service to synchronize a device's media information to a database.

Multimedia Test Utilities Guide

Describes how to use the multimedia test utilities to prototype, test, and debug media applications.

QNX Additions to OpenMAX AL

Provides a reference of the QNX-extended OpenMAX AL API, which supports media format conversion, playback, and recording.

---

All of the guides are installed in the documentation, even if your host system doesn't have all of the multimedia software packages installed.

---

Technical support

To obtain technical support for any QNX product, visit the Support area on our website (www.qnx.com). You'll find a wide range of support options, including community forums.