The QNX SDK for Apps and Media uses several resource managers, services, and libraries to perform the multimedia tasks of detecting mediastores, synchronizing databases with media metadata, and playing audio and video files.
Figure 1. Multimedia Architecture
Communication
As Figure 1 shows, communication between media applications, multimedia components, and OS services is done with C APIs, QDB databases, and Persistent Publish/Subscribe (PPS) objects.
The multimedia synchronizer service, mm-sync, and the multimedia rendering component, mm-renderer, expose C APIs that allow clients to start and monitor media operations.
The QDB databases store metadata describing the media content found on devices. Querying these embedded databases for metadata is faster than reading it from files on physically separate devices. Also, this design means that the metadata is kept in persistent storage, so applications can read the metadata of files stored on devices no longer attached to the system.
The PPS objects store information describing mediastores (devices) currently attached to the system. This information is published by device publishers and it includes hardware connectivity details, filesystem mountpoints, and state information. Because of this last type of information, the contents of some of these objects are dynamic.
For MTP devices, additional PPS objects are published by the MTP driver. These objects contain details about the device's software and vendor, and the last modification time of its media content. The PPS Objects Reference describes the MTP-related PPS objects. The MTP driver runs in the io-fs-media process and provides a POSIX filesystem interface for accessing the device.
For iPod devices, the iPod driver publishes several PPS objects. These objects store information on playback status, Bluetooth profiles, and more, and are explained in detail in the iPod documentation included with the QNX Apps and Media Interface for Apple iPod. Depending on the protocol in use, the iPod driver may run in io-fs-media or another process.
Devices with POSIX filesystems (e.g., USB sticks) don't need a driver running in io-fs-media; this service is used only for non-POSIX devices to provide a common mechanism to access media content. This means that for any device type, your applications can use POSIX system calls to read information from PPS, which is necessary for discovering media content and reacting to content updates (e.g., new media files after pictures are taken by the device operator).
Applications
Your media applications can read device information from PPS objects and interact with mm-sync, mm-renderer, and QDB at any time and in any order because these services are independent of one another. For instance, an application could synchronize some or all of a device's metadata with mm-sync, query the device's QDB database for specific track information, and then, in response to user requests, begin playing tracks with mm-renderer. Or, it could skip the synchronization and begin playing the first audio track found on an attached device by invoking mm-renderer.
It's up to developers to implement the application front-end, whether it's an interactive HMI component or a command interface, and the logic that invokes the multimedia services to carry out tasks. The platform ships with several sample utilities that allow you to test various multimedia services from the command line, without having to learn their APIs:
- mmcli
- Tests the multimedia APIs by forwarding commands to a loaded library or service.
- mmrplay
- Plays or records media through mm-renderer, based on command-line options.
- mmsyncclient
- Forwards synchronization commands to mm-sync and reports synchronization status.
The Multimedia Test Utilities Guide explains the purpose of mmcli, mmrplay, and mm-pnp, and how to configure and use these utilities. The Multimedia Synchronizer Developer's Guide provides usage instructions for mmsyncclient.
Multimedia components
The multimedia components work together to perform three main tasks:
- Detecting devices and retrieving their information
- Synchronizing metadata to QDB databases
- Playing audio and video files
The other sections in this guide explain the order of interaction and the information flow between the components to carry out each of these tasks.
| Name | Description | Path(s) |
|---|---|---|
| PPS objects | Store attributes describing device connectivity, driver processes, and mountpoints of device filesystems. | /pps/qnx/device/* |
| mm-sync | Synchronizes metadata from tracks and playlists on media devices into QDB databases. This metadata includes creation and runtime information for files and playlists. | /base/usr/sbin/mm-sync |
| QDB | Manages embedded databases that store metadata read from media devices. | /base/usr/sbin/qdb |
| libmd | Reads metadata fields from files on media devices. This component library is used by mm-sync but can be linked into and called from an application. | /base/usr/lib/libmd.so |
| libmmplaylist | Retrieves playlist entries, which are track URLs referenced in playlist files. This component library is used by mm-sync but can be linked into and called from an application. | /base/usr/lib/libmmplaylist.so |
| mm-renderer | Plays audio and video tracks, and reports playback state. You can play multiple items concurrently but independently. | /base/usr/sbin/mm-renderer |
| screen | Renders video output to the display. This service is used by mm-renderer but it can be used directly by applications to manipulate the video output window. | /base/usr/lib/libscreen.so |
| io-audio | Starts audio drivers to enable outputting of audio streams through hardware. This service is used by mm-renderer and shouldn't be used directly by media applications. | /proc/boot/io-audio |
| io-fs-media | Runs drivers that provide a POSIX filesystem interface to media devices. For some device types, other multimedia components use this service to read file information and media streams. | /base/usr/sbin/io-fs-media |
OS services
The OS layer includes device publishers. When users attach devices, device publishers create PPS objects and write device information into them. When users remove devices, the publishers remove the objects that store information about those devices. Your media applications can monitor the entries of the PPS directories that store the publisher objects, then read the object contents to discover new mediastores and learn their mountpoints, which the applications can explore for playable content.
The Device Publishers Developer's Guide explains the types of PPS objects that store device information, the directories in which these objects are published, and the included publisher services and the device types that they support.