This chapter includes:
Gathering many events generates a lot of data, which requires memory and processor time. It also makes the task of interpreting the data more difficult.
Because the amount of data that the instrumented kernel generates can be overwhelming, the SAT supports several types of filters that you can use to reduce the amount of data to be processed:
The static and dynamic rules filters affect the amount of data being logged into the kernel buffers; filtered data is discarded — you save processing time and memory, but there's a chance that some of the filtered data could have been useful.
In contrast, the post-processing facility doesn't discard data; it simply doesn't use it — if you've saved the data, you can use it later.
Most of the events don't indicate what caused the event to occur. For example, an event for entering MsgSendv() doesn't indicate which thread in which process called it; you have to infer it during interpretation from a previous thread-running event. You have carefully choose what you filter to avoid losing this context.
You can use the static rules filter to track or filter events for all classes, certain events in a class, or even events related to specific process and thread IDs. You can select events in an additive or subtractive manner; you can start with no events and then add specific classes or events, or you can start with all events and then exclude specific ones.
The static rules filter is the best, most efficient method of data reduction. It generally frees up the processor while significantly reducing the data rate. This filter is also useful for gathering large amounts of data periodically, or after many hours of logging without generating gigabytes of data in the interim.
You set up this filter using various TraceEvent() commands. For information about the different classes, see “Classes and events” in the “Events and the Kernel” chapter of this guide.
You can set up process- and thread-specific tracing (using the _NTO_TRACE_SET* and
_NTO_TRACE_CLR* commands described below) only for the following classes:
|
Here's a general outline for using the static rules filter:
TraceEvent(_NTO_TRACE_DELALLCLASSES);
The _NTO_TRACE_DELALLCLASSES command doesn't suppress any process- and thread-specific tracing that might have previously been set up. You need to clear it separately, by using the following TraceEvent() commands:
To clear: | Call TraceEvent() with these arguments: |
---|---|
Process-specific tracing of all events of a given class | _NTO_TRACE_CLRCLASSPID, int class |
Process- and thread-specific tracing of all events of a given class | _NTO_TRACE_CLRCLASSTID, int class |
Process-specific tracing of the given event in a given class | _NTO_TRACE_CLREVENTPID, int class, int event |
Process- and thread-specific tracing of a given event and class | _NTO_TRACE_CLREVENTTID, int class, int event |
For example, you might want to start by turning off all filtering:
TraceEvent(_NTO_TRACE_DELALLCLASSES); TraceEvent(_NTO_TRACE_CLRCLASSPID, _NTO_TRACE_KERCALL); TraceEvent(_NTO_TRACE_CLRCLASSTID, _NTO_TRACE_KERCALL); TraceEvent(_NTO_TRACE_CLRCLASSPID, _NTO_TRACE_THREAD); TraceEvent(_NTO_TRACE_CLRCLASSTID, _NTO_TRACE_THREAD); TraceEvent(_NTO_TRACE_CLRCLASSPID, _NTO_TRACE_VTHREAD); TraceEvent(_NTO_TRACE_CLRCLASSTID, _NTO_TRACE_VTHREAD); TraceEvent(_NTO_TRACE_CLRCLASSPID, _NTO_TRACE_SYSTEM); TraceEvent(_NTO_TRACE_CLRCLASSTID, _NTO_TRACE_SYSTEM); TraceEvent(_NTO_TRACE_CLRCLASSPID, _NTO_TRACE_COMM); TraceEvent(_NTO_TRACE_CLRCLASSTID, _NTO_TRACE_COMM);
To: | Call TraceEvent() with these arguments: |
---|---|
Enable the tracing of all classes and events | _NTO_TRACE_ADDALLCLASSES |
Enable the tracing of all events in a specific class | _NTO_TRACE_ADDCLASS, class |
Enable the tracing of a specific event in a specific class | _NTO_TRACE_ADDEVENT, class, event |
Disable the tracing of all events in a specific class | _NTO_TRACE_DELCLASS, class |
Disable the tracing of a specific event in a specific class | _NTO_TRACE_DELEVENT, class, event |
To trace: | Call TraceEvent() with these arguments: |
---|---|
All events for the given class that are for the process with the given ID | _NTO_TRACE_SETCLASSPID, int class, pid_t pid |
All events for the given class that are for the process and thread with the given IDs | _NTO_TRACE_SETCLASSTID, int class, pid_t pid, uint32_t tid |
All events of the given type in the given class that are for the process with the given ID | _NTO_TRACE_SETEVENTPID, int class, int event, pid_t pid |
All events of the given type in the given class that are for the process and thread with the given IDs | _NTO_TRACE_SETEVENTTID, int class, int event, pid_t pid, uint32_t tid |
You can set up class or event filtering for one process or thread at a time. For example, the following sets up filtering for different classes for different processes:
TraceEvent(_NTO_TRACE_SETCLASSPID, _NTO_TRACE_KERCALL, pid_1); TraceEvent(_NTO_TRACE_SETCLASSTID, _NTO_TRACE_THREAD, pid_2, tid_1);
but the second call in the following overrides the setting made in the first call:
TraceEvent(_NTO_TRACE_SETCLASSPID, _NTO_TRACE_KERCALL, pid_1); TraceEvent(_NTO_TRACE_SETCLASSTID, _NTO_TRACE_KERCALL, pid_2, tid_1);
For an example that uses the static filter, see the five_events.c example in the “Tutorials” chapter.
The dynamic rules filter can do all the filtering that the static filter does — and more — but it isn't as quick. This filter lets you register functions (event handlers) that decide whether or not to log a given event.
If you want to use a dynamic rules filter, be sure that you've also set up a
static rules filter that logs the events you want to examine.
For example, if you want to dynamically examine events in the
_NTO_TRACE_THREAD class, also call:
TraceEvent(_NTO_TRACE_ADDCLASS, _NTO_TRACE_THREAD): |
For an example of using the dynamic rules filter, see the eh_simple.c example in the Tutorials chapter.
To set up the dynamic rules filter, you must call ThreadCtl() with the _NTO_TCTL_IO flag to get I/O privileges:
if (ThreadCtl(_NTO_TCTL_IO, 0)!=EOK) { fprintf(stderr, "argv[0]: Failed to obtain I/O privileges\n"); return (-1); }
and then call TraceEvent() with one of these commands:
TraceEvent(_NTO_TRACE_ADDCLASSEVHANDLER, class, int (*event_hdlr)(event_data_t*), event_data_t* data_struct);
TraceEvent(_NTO_TRACE_ADDEVENTHANDLER, class, event, int (*event_hdlr)(event_data_t*), event_data_t* data_struct);
The additional arguments are:
int event_hdlr (event_data_t *event_data);
The dynamic filter is an event handler that works like an interrupt handler. When this filter is used, a section of your custom code is executed. The code can test for a set of conditions before determining whether the event should be stored.
The only library functions that you can call in your event handler are those that are safe to call from an interrupt handler. For a list of these functions, see the Summary of Safety Information appendix in the QNX Neutrino Library Reference. If you call an unsafe function—such as printf()—in your event handler, you'll crash your entire system. Your event handler must also be reentrant. |
If you want to log the current event, return a non-zero value; to discard the event, return 0. Here's a very simple event handler that says to log all of the given events:
int event_handler(event_data_t* dummy_pt) { return(1); }
If you use both types of dynamic filters (event handler and class event handler), and they both apply to a particular event, the event is logged if both event handlers return a non-zero value. |
In addition to deciding whether or not the event should be logged, you can use the dynamic rules filter to output events to external hardware or to perform other tasks— it's up to you because it's your code. Naturally, you should write the code as efficiently as possible in order to minimize the overhead.
You can access the information about the intercepted event within the event handler by examining the event_data_t structure passed as an argument to the event handler. The layout of the event_data_t structure (declared in <sys/trace.h>) is as follows:
/* event data filled by an event handler */ typedef struct { __traceentry header; /* same as traceevent header */ uint32_t* data_array; /* initialized by the user */ uint32_t el_num; /* number of elements returned */ void* area; /* user data */ uint32_t feature_mask;/* bits indicate valid features */ uint32_t feature[_NTO_TRACE_FI_NUM]; /* feature array - additional data */ } event_data_t;
The event_data_t structure includes a pointer to an
array for the data arguments of the event.
You must provide an array, and it must be large enough to hold the
data for the event or events that you're handling (see the
Current Trace Events and Data
appendix).
For example:
event_data_t e_d_1; uint32_t data_array_1[20]; /* 20 elements for potential args. */ e_d_1.data_array = data_array_1; If you don't provide the data array, or it isn't big enough, your data segment could become corrupted. |
You can use the following macros, defined in <sys/trace.h>, to work with the header of an event:
The bits of the feature_mask member are related to any additional features (arguments) that you can access inside the event handler. All standard data arguments — the ones that correspond to the data arguments of the trace event — are delivered without changes within the data_array.
There are two constants associated with each additional feature:
The currently defined features are:
Feature | Parameter mask | Index |
---|---|---|
Process ID | _NTO_TRACE_FMPID | _NTO_TRACE_FIPID |
Thread ID | _NTO_TRACE_FMTID | _NTO_TRACE_FITID |
If any particular bit of the feature_mask is set to 1, then you can access the feature corresponding to this bit within the feature array. Otherwise, you must not access the feature. For example, if the expression:
feature_mask & _NTO_TRACE_FMPID
is TRUE, then you can access the additional feature corresponding to identifier _NTO_TRACE_FMPID as:
my_pid = feature[_NTO_TRACE_FIPID];
To remove event handlers, call TraceEvent() with these commands:
TraceEvent(_NTO_TRACE_DELCLASSEVHANDLER, class);
TraceEvent(_NTO_TRACE_DELEVENTHANDLER, class, event);
The post-processing facility is different from the other filters in that it reacts to the events without permanently discarding them (or having to choose not to). Because the processing is done on the captured data, often saved as a file, you could make multiple passes on the same data without changing it—one pass could count the number of thread state changes, another pass could display all the kernel events.
The post-processing facility is really a collection of callback functions that decide what to do for each event. One example of post-processing is the traceprinter utility itself. It prints all the events instead of filtering them, but the principles are the same.
We'll look at traceprinter in more detail in the Interpreting Trace Data chapter.