Locating sources of high CPU usage

A key step in optimizing embedded systems is locating sources of high CPU usage. The System Profiler results show you per-process and per-thread CPU consumption over some or all of the trace period so you can spot peaks in the processor loads.

In the Summary pane, the statistics for CPU time breakdown give you an idea of how efficiently your QNX target is performing, by showing you the amount of idle time and the ratio of user time to system time. They also let you spot interrupt flooding, which occurs when interrupt time isn't as low as you expect based on your system design. A higher interrupt time could mean faulty hardware, a bad driver, or too many applications being profiled at the same time.

To find periods of high CPU usage, you can start by looking for peaks in the Process & Thread Activity bar graph. These peaks officially indicate high numbers of events at particular times, which often happen when one or more applications are using the CPU heavily. If you run the kernel event trace during an important operational phase, such as system startup or just after a certain application is launched, this graph can tell you when the embedded system is heavily loaded.

Next, to see the CPU resources consumed by individual threads, click the Switch Pane dropdown in the editor controls (Icon: Switch Pane dropdown) and select CPU Usage. This pane contains a line graph that illustrates the CPU usage history of any thread selected in the table underneath. The lines and areas representing distinct threads are stacked on each other, so the topmost line indicates the aggregate load of the selected threads.

This functionality gives you lots of flexibility, allowing you to see the CPU consumption of any combination of threads that ran during the kernel event trace.

Screenshot of CPU Usage pane in System Profiler editor
Note: In this example, we selected process threads, but the functionality works the same for interrupt handlers, which also consume CPU cycles.

The table area contains three tabs that display the times that various threads ran on individual CPUs, at specific priority levels, and within certain partitions. For all three tabs, the threads are initially sorted by their total running times across all CPUs (in descending order), but you can change the sorting order by clicking the column headers. The measurements reflect the CPU consumption over the period selected in the chart above or the Timeline pane. By default, they appear as time values, but you can also display them as percentages, by configuring the System Profiler preferences. This feature and all columns in all tabs are described in the CPU Usage reference.

Clicking table rows updates the graph to display the CPU usage history of the associated threads. Multi-select is supported, by holding the Ctrl and/or Shift keys, similar to the functionality of Windows Explorer. Below the table, a summary row displays the sums of the values for the selected threads.

The zoom feature lets you scale the line graph to better read changes in CPU consumption over short timeframes. To change the zoom level, you can use the zoom in (Icon: Zoom in button) and zoom out (Icon: Zoom out button) buttons in the upper left corner of the IDE. You can also select a region of the graph by clicking and dragging the mouse, then right-click and choose Zoom In. Above the graph, the yellow band in the timeline scale and the start and end times are updated to reflect the time period now displayed.

After learning which threads are heavy CPU consumers, you can profile the corresponding applications to see which functions are called most often and have the longest runtimes.