The QNX SDK for Apps and Media uses several resource managers, services, and libraries to perform the multimedia tasks of detecting mediastores, synchronizing media metadata with databases, and playing audio and video files.
Figure 1. Multimedia Architecture
Communication
As Figure 1 shows, communication between media applications, multimedia components, and OS services is done with C APIs, QDB databases, and Persistent Publish/Subscribe (PPS) objects. The multimedia synchronizer service, mm-sync, and the multimedia rendering component, mm-renderer, expose C APIs that allow clients to start and monitor media operations.
The QDB databases store metadata describing the media content found on devices. Querying these embedded databases for metadata is faster than reading it from files on physically separate devices. Also, this design means that the metadata is kept in persistent storage, so applications can read the metadata of files stored on devices no longer attached to the system.
The PPS objects store information describing mediastores (devices) currently attached to the system. This information includes hardware connectivity details and the device's filesystem type and mountpoint. Your applications can use POSIX system calls to read the information in these objects, which is necessary for discovering and accessing media content.
Applications
Your media applications can read device information from PPS objects and interact with mm-sync, mm-renderer, and QDB at any time and in any order because these services are independent of one another. For instance, an application could synchronize some or all of a device's metadata with mm-sync, query the device's QDB database for specific track information, and then, in response to user requests, begin playing tracks with mm-renderer. Or, it could skip the synchronization and begin playing the first audio track found on an attached device by invoking mm-renderer.
It's up to developers to implement the application front-end, whether it's an interactive HMI component or a command interface, and the logic that invokes the multimedia services to carry out tasks. The platform ships with several sample utilities that allow you to test various multimedia services from the command line, without having to learn their APIs:
- mmcli
- Tests the multimedia APIs by forwarding commands to a loaded library or service.
- mmrplay
- Plays or records media through mm-renderer, based on command-line options.
- mmsyncclient
- Forwards synchronization commands to mm-sync and reports synchronization status.
The Multimedia Test Utilities Guide explains the purpose of mmcli, mmrplay, and mm-pnp, and how to configure and use these utilities. Usage instructions for mmsyncclient are given in the Multimedia Synchronizer Developer's Guide.
Multimedia components
The multimedia components work together to perform three main tasks:
- Detecting devices and retrieving their information
- Synchronizing metadata to QDB databases
- Playing audio and video files
The other sections in this guide explain exactly how the components interact to carry out these tasks.
| Name | Description | Path(s) |
|---|---|---|
| PPS objects | Device-specific objects store attributes describing device connectivity, driver processes, and mountpoints of device filesystems. | /pps/qnx/device/* |
| mm-sync | Synchronizes metadata from tracks and playlists on media devices into QDB databases. This metadata includes creation and runtime information for files and playlists. | $QNX_TARGET/platform/usr/sbin/mm-sync, where the $QNX_TARGET environment variable provides a partial path to the installer image binaries (e.g., /qnx660/target/qnx6/), and platform is a processor name (i.e., armle-v7 or x86) |
| QDB | Manages embedded databases that store metadata read from media devices. | $QNX_TARGET/platform/usr/sbin/qdb |
| libmd | Reads metadata fields from files on media devices. This library is used by mm-sync but can be linked into and called from an application. | $QNX_TARGET/platform/usr/lib/libmd.so |
| libmmplaylist | Retrieves playlist entries, which are track URLs referenced in playlist files. This library is used by mm-sync but can be linked into and called from an application. | $QNX_TARGET/platform/usr/lib/libmmplaylist.so |
| mm-renderer | Plays audio and video tracks, and reports playback state. You can play multiple items concurrently but independently. | $QNX_TARGET/platform/usr/sbin/mm-renderer |
| screen | Renders video output to the display. This service is used by mm-renderer but it can be used directly by applications to manipulate the video output. | $QNX_TARGET/platform/usr/lib/libscreen.so |
| io-audio | Starts audio drivers to enable outputting of audio streams through hardware. This service is used by mm-renderer and shouldn't be used directly by media applications. | $QNX_TARGET/platform/sbin/io-audio |
OS services
The OS layer includes device publishers. When users attach devices, device publishers create PPS objects and write device information into them. When users remove devices, the publishers remove the objects that store information about those devices. Your media applications can monitor the entries of the PPS directories that store the publisher objects, then read the object contents to discover new mediastores and learn their mountpoints, which the applications can explore for playable content.
The Device Publishers Developer's Guide explains the types of PPS objects that store device information, the directories in which these objects are published, and the included publisher services and the device types that they support.