The QNX Multimedia Suite and QNX SDP platform contain many components that support detecting media devices and reading their information, synchronizing databases with media metadata, and playing audio and video files.
Figure 1. Multimedia Architecture
Communication
Communication between media applications, multimedia components, and platform services is done with C APIs, QDB databases, and publish-subscribe services.
The multimedia synchronizer service, mm-sync, and the multimedia playback engine, mm-renderer, expose C APIs that allow clients to start and monitor media operations.
The QDB databases store metadata describing the media content found on devices. Querying these embedded databases for metadata is faster than reading it from files on physically separate devices. Also, this design keeps the metadata in persistent storage, so applications can read metadata of files stored on devices that are no longer attached to the system.
Device publishers provide information to media applications about mediastores (devices) currently attached to the system. The publishers write this information (and the media applications read it) through the publish-subscribe services that are part of the QNX SDP platform. The information includes hardware connectivity details, mediastore mountpoints, and device state. A full explanation of the publishers is given in the Device Publishers Developer's Guide.
The MTP driver, which runs in the io-fs-media process, publishes extra information about MTP devices, including software and vendor details and the last time a device's media content was modified.
For Apple devices, the process that runs the protocol driver publishes details about playback status, Bluetooth profiles, and more. For a description of the information written by the driver through the publish-subscribe services, see the Working with Apple Devices reference available through the Apple documentation package in QNX Software Center.
Devices with POSIX filesystems (e.g., USB sticks) don't need a driver for applications to read their media content.
With this design, your media applications can read information about any device type from the publish-subscribe services. This way, they can discover media content and react to content updates (e.g., new media files after pictures are taken by the device operator).
Application interaction with multimedia services
Your media applications can read device information through the publish-subscribe services and interact with mm-sync, mm-renderer, and QDB at any time and in any order because these services are independent of one another. For instance, an application could synchronize some or all of a device's metadata with mm-sync, query the device's QDB database for track information, and then, in response to user requests, begin playing tracks with mm-renderer. Or, it could skip the synchronization and begin playing the first track found on an attached device by invoking mm-renderer.
It's up to developers to implement the application front-end, whether it's an interactive HMI or a command interface, and the logic that calls the C APIs of the multimedia services to carry out tasks. For prototyping, testing, and debugging early application versions, the QNX Multimedia Suite offers test utilities that allow you to use the multimedia services from the command line, without having to learn their APIs.
Multimedia components
- Detecting devices and retrieving information about their media files
- Synchronizing QDB databases with media metadata
- Playing audio and video files
The other sections in this guide explain the order of interaction and the information flow between the components to carry out each of these tasks.
| Name | Description | Target path |
|---|---|---|
| mm-sync | Uploads metadata from tracks and playlists found on media devices into QDB databases. The metadata include creation and runtime details for files and playlists. | /base/usr/sbin/mm-sync |
| libmd | Reads metadata fields from files on media devices. This component library is used by mm-sync but can be linked into and called from an application. | /base/usr/lib/libmd.so |
| libmmplaylist | Retrieves playlist entries, which are track URLs referenced in playlist files. This component library is used by mm-sync but can be linked into and called from an application. | /base/usr/lib/libmmplaylist.so |
| mm-renderer | Plays audio and video tracks, and reports playback state. You can play multiple items concurrently but independently. This service also supports recording audio to a file. | /base/usr/sbin/mm-renderer |
| mmcli | Tests the multimedia APIs by forwarding commands to a loaded library or service. | /base/usr/bin/mmcli |
| mmrplay | Plays or records media through mm-renderer, based on command-line options. | /base/usr/bin/mmrplay |
| mmsyncclient | Forwards synchronization commands to mm-sync and reports synchronization status. | /base/usr/bin/mmsyncclient |
The Multimedia Test Utilities Guide explains how to configure and run mmcli and mmrplay. The Multimedia Synchronizer Developer's Guide provides instructions on using mmsyncclient.
Platform services
The QNX SDP platform includes the device publishers. When users attach devices, these components write device information through the publish-subscribe services. When users remove devices, the publishers send updates to indicate that those devices are no longer attached. Your media applications can subscribe to updates and then read them to discover new mediastores and learn their mountpoints. Through the mountpoints, applications can look for playable content.
The Device Publishers Developer's Guide explains each publisher service, the device types that it supports, and the specific fields that it publishes for subscribers (i.e., media applications) to read.