Overview of Screen

Screen is a compositing windowing system.

Unlike traditional windowing systems that arbitrate access to a single buffer associated with a display, this compositing windowing system provides the means for applications to render off-screen.

Rendering to off-screen buffers allows the manipulation of window contents without having to involve the applications that are doing the rendering. Windows can be moved around, zoomed in, zoomed out, rotated, or have transparency effects applied to them, all without requiring the application to redraw or even be aware that such effects are taking place.

Screen is responsible for:

running all drivers (e.g., input, display, OpenGL ES)
allocating memory needed by application windows
displaying content when rendering completes

Screen integrates multiple graphics and user interface (UI) technologies into a single scene. This scene is rendered into one image that is associated with a display.

Figure 1. Screen: A composited windowing system.

Handling composition

The main responsibility of Screen is to combine all visible window buffers into one final image that is displayed. This responsibility is handled by the Composition Manager and is achieved using several classes of hardware. The Composition Manager can be configured to use available compositing hardware in a way that best meets the needs of a particular system.

Screen has a plug-in architecture that includes hardware-specific compositing modules and a module for OpenGL for Embedded Systems (OpenGL ES).

Screen uses GPU-accelerated operations to optimally build the final scene. You may resort to using software rendering if your hardware cannot satisfy the requests. The graphics drivers and display controllers that run within Screen are based on the OpenWF Display (WFD) API.

Communicating with applications

Applications communicate with the Composition Manager using Screen API to perform such tasks as the following:

creating and destroying windows.
creating and destroying pixmaps.
using accessor functions to set and get native window, pixmap, display, device, and buffer properties.
drawing into native buffers that are associated with windows and pixmaps
making areas, within buffers, that can be displayed.
Receiving and processing asynchronous events from Screen.
sending events to other windowed applications.

Applications can render using:

software; applications access the window buffer and write to it using the CPU.
OpenGL ES; use EGL to target the window buffer(s) with OpenGL ES calls.

Sample Screen applications

Screen provides a set of sample native applications that demonstrate what you can do with the Screen API. These sample applications are installed on your host under each target-specific directory under $QNX_TARGET.

Running these applications will help in determining whether or not you have set the appropriate Screen configuration.

/usr/bin/calib-touch: This utility loads or creates an mtouch calibration file, /etc/system/config/calib.<hostname>. If the calibration file needs to be created, the user will be prompted to calibrate.
/usr/bin/display_image: This application displays a specified image to the specified display.
/usr/bin/egl-configs: This utility queries and displays the number of all the frame buffer configurations for the specified display. If no display is specified, then the default display is used.
/usr/bin/events: This application uses a window manager connection to Screen. It prints all events to the console output as they are received.
/usr/bin/font-freetype: This application shows how to render text with FreeType and OpenGL ES 1.X.
/usr/bin/gles1-gears: This application shows windowed gears that use OpenGL ES 1.X for the rendering API.
/usr/bin/gles2-gears: This application shows windowed gears that use OpenGL ES 2.X for the rendering API.
/usr/bin/gles2-maze: This application uses OpenGL ES 2.X for the rendering; it demonstrates how to use texture as well as vertex and fragment shaders.
/usr/bin/print-gestures: This application detects and displays recognized gestures.
/usr/bin/screenshot: This application takes a screenshot of a specified size of the display and saves the output file in BMP format.
/usr/bin/sw-vsync: This application shows windowed vsync that uses software rendering.
/usr/bin/vcapture-test: This application demonstrates that you can connect to a device for video input, capture frames from that input source, and then display the captured frames using Screen.
/usr/bin/vkey: This application uses the privileged context, SCREEN_INPUT_PROVIDER_CONTEXT, to create a connection to the Composition Manager. A key sequence is injected to whichever window has input focus on the specified display. The application will exit as soon as the last character is sent.
/usr/bin/yuv-test: This application displays a YUV test pattern.