Locate events of interest

Trace event log files contain a wealth of information, but unfortunately that information is often contained in thousands, if not millions, of events. The System Profiler tools can reduce and filter the data contained in a log file, helping you focus on the events that are most important.

Requirements

There are no specific requirements for this use case, but some of the topics may not apply, depending on the types of events that have been captured.

Procedure

The data in a log file is often most useful during investigations involving the Timeline editor pane. This pane displays data with a very fine granularity and allows you to single step through the execution flow of an activity of interest. You can display the Timeline pane via the menu item System Profiler > Display > Switch Pane > Timeline, or through the editor's dropdown menu for switching panes ().

Timeline filters

The first level of data reduction is to use the Filters view to remove data that is insignificant for tracing the problem you're interested in. By using filters along with zooming and searching capabilities, you can quickly reduce the overall data set.

The Filters view is synchronized with the active System Profiler editor. You can display this view via the menu option Window > Show View > Other… > Filters or by right-clicking and choosing Show Filters View in the Timeline pane.

The view contains three tabs that provide you with different types of filtering:
  • Owners—shows a list of event owners or sources, letting you choose which event owners are displayed. Unselecting an owner removes its timeline from the Timeline pane.

  • Events—similar to the Owners tab, but it provides filtering capabilities for individual events rather than for the owners.

    For information about event types, see "Classes and events" in the Events and the Kernel chapter in the System Analysis Toolkit guide.

  • Partitions—provides filtering capabilities for showing only data related to specific partitions.

The view menu offers additional filter options, such as Configure Filters:

The Configure System Profiler Filters dialog lists the preconfigured filters available for use. These filters are often based on more sophisticated criteria for determining if certain events, owners, or partitions are to be displayed.

Trace Event Log filter synchronization

By default, the Trace Event Log view displays an event list using the same filters as the currently active editor. Sometimes, though, it's useful to temporarily unfilter the list to see the raw content of the log file. You can do this by toggling the Enable/disable event filtering button () in the view's toolbar.

Timeline find

When you're looking at an event stream, sometimes you need to quickly navigate through it. One mechanism for doing this is to move between events by using the toolbar commands—Next, Previous, Next Event In Selection, Previous Event In Selection.

Another more flexible mechanism is to use the Find functionality of the Timeline pane. Selecting Edit > Find/Replace opens a dialog similar to the one found in many text editors.

The dialog supports searching a restricted set of event owners (based on the Timeline pane selection) as well as searching forwards and backwards through the log file. This is convenient when you know what type of event you're looking for in an event sequence (e.g., the next RUNNING state for a thread).

The Find dialog moves the selection marker in the Timeline pane to the matching event.

Trace Search

If you need to generate a collection of events matching a certain condition, or to construct a complicated expression (perhaps including event data) to find the events you're looking for, you need the power of trace event conditions and the Trace Search tool.

The Trace Search tool is invoked by selecting Search > Search and then clicking the Trace Search tab in the resulting dialog. This tab contains fields for defining an advanced search:

Searching is based on trace conditions. These describe the criteria for matching an event and can be based on any data that an event provides (ownership, data payload, etc). Unlike the other search dialogs in the IDE, the Trace Search can search for events only in the currently active System Profiler editor.

The following steps explain how to define a search condition to locate all MsgSendv calls that may have been made for write system calls, and then view the results:
  1. Add a new condition via the Add button in the search dialog. This brings up a new dialog in which you can specify the Kernel Calls class, MsgSendv Enter code, and write function (under event details). String values, such as functions names, are matched using regular expressions.

  2. After you've defined the condition, it shows up in the Defined Conditions table on the Trace Search tab. You can combine individual conditions to form Boolean expressions if required.

  3. In the Search Expression dropdown, select the new condition, if necessary. Then, click Search. The IDE performs the search and opens the Search view, which contains the results.
    System Profiler: Search results

Search results are also marked in the Timeline pane to show the event distribution over the log period. Double-clicking a result entry (assuming it isn't filtered) moves the selection marker to that event:
System Profiler: search results on the timeline

Exporting filtered log files

Often, the log files of captured kernel events are large and contain a lot of nonessential data for the problem at hand. But generally, you don't know this until after you've performed some basic analysis.

You can use the File > Save As menu option to create a new log file based on the current log file, using the System Profiler editor. You can restrict the new log file to contain just those events in the selected area, and you can also use the current filter settings (event and event owner) to reduce the amount of additional data that gets stored in the new file.

The new file contains the same attribute information as the original file (the system version, system boot time, number of CPUs, etc). Any event owners (interrupts, processes, and threads) referenced by events in the new file are created with timestamps matching the start time of this event log.