Postmortem profiling: Running an instrumented binary from the command line

You can view profiling results after an application finishes running, which is known as postmortem profiling. This is handy if you prefer to develop an application outside of the IDE and run it from the command line on a target, but then import the profiling results into the IDE and view them through the QNX Application Profiler perspective.

After building an application binary with either call count or function runtime instrumentation, you can copy the binary to the target. You must also copy over any libraries built with the same profiling instrumentation if you want to see results for their functions also. Then, you must set some environment variables (as explained below) to direct the data output. At this point, you can run the application.

Position sampling and call count profiling

With this profiling method, you must set PROFDIR to the directory for storing the data. When you do so, the results file is stored at PROFDIR/gmon.out.pid.process_name. If you don't set this environment variable, the file is named simply gmon.out (which makes it hard to distinguish from files produced by other profiling sessions) and is stored in the directory where the program ran.

As always when running an application from the command line, you must define the paths of any dynamic libraries used by your application, by setting LD_LIBRARY_PATH. Suppose you need libraries in /tmp/lib and you want to write the profiling results to /tmp. Your command line would then be:
% PROFDIR=/tmp LD_LIBRARY_PATH=/tmp/lib program_name &
Note: The program must exit normally for profiling data to be written. If you don't see a results file, you must fix whatever problems are preventing the application from successfully terminating. The other option is to force it to exit normally by attaching a signal handler that calls exit().
When used from the command line, this profiling method has the following restrictions:
  • Information for individual threads isn't available — The profiling results from all threads get combined into one set of values.
  • Results from shared libraries aren't available — The results contain runtime estimates for code in shared libraries but specify an unknown location, so you don't know which functions the entries refer to.
  • No sampling information is collected if you don't run as root — If you don't run the program as root, the process can't attach the necessary interrupt handler to write out position sampling data. This means you'll see only call counts and not function runtime estimates in the results.

Function runtime measurement profiling

With this profiling method, you must set QPROF_FILE to the full path of the file for storing the data (e.g., /tmp/profiler.ptrace). If you define only a filename in this variable, the results file gets written to the directory where the program ran. If you don't define the variable, no results are written. Also, the filename portion must finish with a .ptrace extension so the file can later be imported into the IDE.

You must also set QPROF_START to 1 for profiling to be enabled when the program starts running.

Profiling data gets written even if the program doesn't exit normally. However, if this happens, some data may be lost because some buffers can't be flushed to the results file. You can force the application to exit normally by attaching a signal handler that calls exit().

The restrictions of individual thread information and results from shared libraries not being available also apply to this profiling method (for details, see the explanation above for position sampling). Also, the amount of data generated can be up to 2 MB per second, so you should let the application run for only a few seconds; longer profiling sessions could produce too much data to be useful.

Suppose you want to start profiling as soon as the program starts, use libraries in /tmp/lib, and store the results in a file named for the application. Your command line would then be:
% QPROF_START=1 QPROF_FILE=/tmp/app_name.ptrace LD_LIBRARY_PATH=/tmp/lib program_name &
You can also specify the method for calculating function runtimes, and signal handlers for stopping and resuming profiling, as described in "Environment variables for controlling profiling".

Application profiling with kernel event tracing

You can run an application binary built with function instrumentation while performing a kernel event trace, to capture that application's function entrance and exit events. To do this, you must first set QPROF_KERNEL_TRACE to 1 (to make the kernel start logging events), then run tracelogger in the background (to make this utility start writing the events to a trace file). Note that QPROF_FILE must not be defined when the tracing variable is set to 1.

The tracelogger utility must be run for long enough that the application can be started and execute until its normal exit point, or long enough to generate valuable profiling data. For information on the command-line options for setting the output file and tracing time, see the tracelogger entry in the Utilities Reference. The output file will be a kernel event log (.kev) file (like when the trace is run through the IDE), which can be opened in the QNX System Profiler perspective.

Viewing results

To view the profiling results, you must either attach the Application Profiler tool to one of the application's processes or wait until the application exits and then copy the results file from the target to the host and import it into the IDE.