Advanced Graphics

The QNX Advanced Graphics TDK is a set of tools and runtime components for creating sophisticated displays that feature accelerated 2D and 3D graphics rendering with little CPU overhead. Based on a clean room and certified implementation of the industry standard OpenGL ES API, it allows developers to leverage a base of existing OpenGL code as well as industry programming expertise.

Development tools

• Programming environment (headers and libraries) for developing 2D and 3D applications
  
• Sample code illustrating 2D, 3D, video capture, image manipulation, cursor control and use of fonts
  

Runtime components

• Drivers for graphic controllers
  
• Runtime libraries for 2D drawing, layering control, and video capture
  
• Runtime libraries for 3D (OpenGL® ES API Certified)
  
• Runtime libraries for font and image rendering
  
• Graphic display manager (io-display)
  

Application Architecture

QNX Graphics technologies provide flexibility and performance

Technology overview

At the core of the QNX Advanced Graphics TDK is a technology called Graphics Framework (GF). GF provides the services that allow the graphic programmer to build interfaces that range from the tiniest 2D environment to feature-rich 2D and 3D environments including web browsers and email programs.

The GF architecture allows programs to render directly to hardware, making them faster and more responsive in an embedded environment where resources are limited. GF achieves this by providing direct access to graphics drivers (there is no message passing or context switching while rendering), and by using hardware acceleration where possible. This makes it perfect for embedded environments, where it can be used as the basic graphics layer for anything from a simple full screen UI to a complex windowing system for multiple application GUIs. It can also act as a porting layer for existing UIs.

For embedded 3D requirements, QNX Advanced Graphics TDK supports the OpenGL® ES 1.0 API specification, a well-defined subset of OpenGL® designed for embedded applications. The QNX implementation supports the Common profile and EGL platform interface layers.

While the graphics framework can support full screen application user interfaces, it can also handle multiple threads and multiple applications; you can even run 2D and OpenGL® ES applications alongside Photon applications, if required.

What can you build with the Advanced Graphics TDK?

2D Navigation system

 

• 2D Navigation system using alpha-blending to display menus and emergency notices
• Several graphic layers are used

 

3D Navigation system using OpenGL® ES API

•   Rotated fonts are supported

 

3D Navigation systemusing OpenGL® ES API

•   Add in Web browser on right side of screen showing web content specific to the selected ‘Point of Interest’.

 

2D Monitor display

 

• 2D Monitor display with touchscreen and using high speed trend graphs.

 

Support for 2D features

• Small and fast 2D graphic primitives
  
• Image support
  
• Font support
  
• Video capture API
  
• Layering API (where supported by hardware)
  
• 2D hardware-accelerated graphics drivers wherever possible, software fallback routines when not
  
• more...

Support for 3D features

• OpenGL® ES API Common Profile version 1.0
  
• 3D hardware-accelerated graphics drivers wherever possible, software fallback routines when not
  
• Software 3D library for graphics environment with no 3D hardware acceleration
  
• more...

Technology highlights

Reduce footprint

• Applications only load the graphical services (such as 2D draw, 3D draw, fonts, images) that they need. This reduces memory overhead
  

Leverage standards

• Broad programming expertise and industry base of OpenGL ES source code
  

Deliver optimal performance for graphical displays

• Graphics acceleration with minimal impact on CPU load
  
• Complete integration with compositing facilities of graphics devices that support layering capabilities for moving displays such as navigation or gaming
  
• 3D hardware acceleration
  
  • available on selected chipsets (e.g. Fujitsu Coral-P)
    
• 2D hardware acceleration
  
  • enables hardware-accelerated alpha blending, which is important for anti-aliased polygons or arcs, widely used in mapping applications
    

Create flicker-free animations

• Supports rapid animation using multi-buffering
  
• Standard swap call enables seamless transition between contexts or views
  

Business benefits

Save on BOM costs

Software modularity allows you to deploy only those features that are required.

Save on hardware costs

The architecture itself and high performance drivers are all built to get the best performance possible out of low-end hardware.

Get to market quicker

Standards for 3D means developers don't waste c getting up to speed.

Architecture

Here is an example of a typical application built with the QNX Advanced Graphics TDK and using Graphics Framework. Note that all of these components are optional, except for the graphics driver (devg) and the application code itself (which is statically linked with only the necessary routines required from the Advanced Graphics libraries).

Advanced Graphics application architecture

               

Other components that are included in the example above are:
• keyboard and mouse input managed by an input manager (provided with the board support package for the target system)
  
• font rendering managed by a font rendering system (such as Bitstream's Font Fusion library provided with the Advanced Graphics TDK)
  
• image support provided by an image library (provided with the Advanced Graphics TDK)
  
• OpenGL® ES library support (provided with the Advanced Graphics TDK)

 

While the Graphics Framework is intended for full screen application user interfaces, it can handle multiple threads, multiple applications, and you can even run 2D and OpenGL® ES applications alongside Photon applications, if required.

The Graphics Framework uses a client/server model, with an io-display monitor process acting as the server and managing device resources (such as memory and layers) and hardware access. The client (the graphical application) renders directly to the hardware. Clients obtain a lock (a mutex in shared memory) by calling gf_draw_begin() to ensure that only one thread or application has access to the hardware at a time. The io-display monitor process provides increased robustness by catching abnormal client termination, cleaning up resources after termination, and reviving the shared mutex if it was held by an abnormally terminated client. The shared memory mutex is given to the highest priority waiting thread, ensuring that the graphics program with highest priority will be able to draw first.

All hardware access by GF applications is brokered by the io-display monitor process. The io-display monitor process also allocates all video memory, and handles application termination (both normal and abnormal) by releasing allocated memory, and releasing locked hardware (if necessary). This control can be "invisible" to graphic client programs, as all interactions are handled internally by the library.

The diagram below illustrates the relationship of the io-display monitor process to three separate programs or clients:

• QNX Photon microGUI® environment (with a web browser, email program, etc)
  
• a custom GUI using a 3rd party 2D library that calls the Advanced Graphics TDK GF library
  

The io-display monitor process

               

The diagram above illustrates the relationship of the io-display monitor process to three separate programs or clients:
• QNX Photon microGUI® environment (with a web browser, email program, etc)
  
• a custom GUI using a 3rd party 2D library that calls the Advanced Graphics TDK GF library

Note that all three programs load in the graphics driver shared object (devg) and are directly controlling the graphics hardware. Each program will take a turn in using the graphics driver by asking for and receiving the mutex managed by the io-display monitor process. The order of the exclusive access is based on the priority of the requesting program.

Once io-display has started, applications can use the Graphics Framework library (libgf) and OpenGL® ES libraries to acquire a graphics context and render to the device.

Third party application frameworks or existing in-house frameworks can easily be used with the QNX Advanced Graphics TDK. An existing framework can choose to use any of the underlying supported features including 2D draw, 3D draw, image and font support. Simple wrapper routines can be used to thinly abstract draw operations while still providing the existing framework with the performance and real-time features inherent in the Graphics Framework.

Graphics Framework components

Within the QNX Graphics Framework there are conceptual components used to divide the graphical environment in to an understandable hierarchy. These components are:

• devices
  
• displays
  
• layers
  
• surfaces
  
• contexts
  

Devices and displays are the logical entities representing hardware components managed by GF. A device is a piece of graphics hardware, which is attached to and managed by io-display via the appropriate devg-* driver. Each device has an entry in /dev/io-display.

Each device has at least one display, which represents the video display viewed by the end-user. Some devices are multi-headed, meaning they support multiple displays.

Layers represent an area of memory that is shown on a display. Some modern graphics hardware supports multiple layers, though from the GF perspective all devices have at least one layer (layer 0). Layers provide a great deal of flexibility in laying out content on a display. For example, you can put a scrolling map on a background layer, and GUI controls for scrolling the map on a front layer. The application can scroll the map smoothly without redrawing the GUI controls, eliminating flicker and reducing CPU load. In any situation where you want to display text or a GUI over top of visual information, layering can be very useful.

Device, display, and layer hierarchy

               

This figure illustrates the relationship between a device, its displays, and their layers

A surface is piece of memory that the GF library can render to. If the surface is connected to a layer (and the layer is visible), anything rendered on the surface appears on the layer's display. The surface must have a pixel format that is supported by the layer so that objects rendered onto the surface appear correctly. Generally, a layer targets (or is associated with) a single surface. An exception is planar data, such as planar YUV commonly used for video capture, which requires three surfaces (one for each video component).

By default, the GF allocates a surface in video memory for highest performance. However, when you create the surface you can request that it be optimized for CPU access, which generally means it will be allocated in system RAM.

A context is a structure that maintains information about a rendering target between calls to rendering functions, such as the pen width, foreground and background colors, and clipping rectangle. A context targets a surface, and is an argument to all GF drawing functions. A surface can be used for 2D drawing or OpenGL ES 3D drawing. Surfaces can be combined on multiple layers before they become visible.

Contexts, surfaces, and layers

               

This figure illustrates how contexts, surfaces, and layers are relate

By using the combination of displays, layers, surfaces and draw contexts, very complex screens can be built from simple applications. The 2D navigation example below shows a graphics environment that supports four layers. Three layers are being used: one for 2D map display, one for menu/touchscreen and another for warning messages. The system designer may choose to build this as one complete program or as three separate programs. Proper use of chromakey and alphablending enable the visibility between the layers.

Adding 3D then becomes as simple as developing an OpenGL® ES program and either replacing the 2D program with the 3D upgrade or running the OpenGL® ES program on the same layer as 2D (split screen).

Adding a web browser becomes as simple as starting Photon on a graphic layer and then running a custom web browser application developed with the QNX Web Browser TDK.

Specifications

• Binaries
  
  • OpenGL ES libraries and 2D libraries for PPC, X86, SH and ARM
    
  • Software-only OpenGL ES libraries, providing 3D support for graphic chips that have no 3D hardware acceleration
    
  • io-display monitor process
    
  • managers to support simultaneous running of 3D and 2D programs
    
  • image loading library supporting multiple image formats.
    • JPEG, PNG, BMP, TGA, SGA, GIF
      
  • font libraries
    
    • TrueType, Bitmap, Font Fusion (Bitstream stroked fonts). (Sample font files are provided with QNX Momentics development suite)
      
  • Fujitsu Coral B/P/PA graphics driver: x86, PPC, SH
    
  • Fujitsu Carmine graphics driver: x86, SH
    
  • Intel Extreme2 (855GM) graphics driver: x86
    
  • VESABios graphics driver: for x86 VESA-compliant graphics controllers
    
  • VMWare graphics driver: for QNX VMWare sessions
    
  • sample applications pre-compiled for testing 2D and 3D
    
• Documentation
  
  • Graphics Framework Developer's Guide
    
  • OpenGL ES 1.0 Documentation Set
    
  • Font Fusion API Document
    
• Configuration text files
  
  • configuration files for defining video settings (initialization of graphics chips etc for easy adaptation to new display combinations without code changes
    
• Sample source code
  
  • 2D samples: draw rectangle, load and display image, load and display text, etc.
    
  • layer control: turn on/off layer, change chromacolor, etc.
    
  • 3D samples: vertex arrays, texture maps, etc.
    
  • and much more
    

If the QNX Photon microGUI windowing system is distributed with a customer product, then a Photon license must be provided for each runtime.

The Web Browser Technology Development Kit for customizing a web browser is sold separately.

System requirements

• QNX Momentics® development suite Professional Edition v6.3 and higher
  
• QNX Neutrino® RTOS v6.3 and higher (runtime)
  
• Supported hardware:
  
• X86, PPC, SH4 and ARM targets
  
• Supported graphics renderer
  
  • Fujitsu Coral-B/P/PA
    
  • Fujitsu Carmine
    
  • Intel Extreme2
    
  • VMWare session (optional, this is a useful target for testing 2D/3D programs from Windows)
    
  • VESABios X86 graphics chip support (un-accelerated)
    

Please contact your QNX account manager for a complete list of current graphics chip support.

OpenGL® ES and the oval logo are trademarks of Silicon Graphics, Inc. in the United States and/or other countries worldwide. Products bearing such trademarks incorporate intellectual property that is owned and licensed to others by Silicon Graphics, Inc.