Handling ionotify(), poll(), and select()

Updated: April 19, 2023

A client uses ionotify(), poll(), and select() to ask a resource manager about the status of certain conditions (e.g., whether input data is available). The conditions may or may not have been met. The resource manager can be asked to:

The poll() and select() functions differ from ionotify() in that most of the work is done in the library. For example, the client code would be unaware that any event is involved, nor would it be aware of the blocking function that waits for the event. This is all hidden in the library code for poll() and select().

However, from a resource manager's point of view, there's no difference between ionotify(), poll(), and select(); they're handled with the same code.

For more information on the ionotify(), poll(), and select() functions, see the QNX Neutrino C Library Reference.

Since ionotify(), poll(), and select() require the resource manager to do the same work, they all send the _IO_NOTIFY or _IO_NOTIFY64 message to the resource manager. The io_notify handler is responsible for handling this message. Let's start by looking at the format of the message itself:

struct _io_notify {
    uint16_t                    type;
    uint16_t                    combine_len;
    int32_t                     action;
    int32_t                     flags;
    struct __sigevent32         event;

    /* The fields `mgr` to `timo` are only valid if (flags & _NOTIFY_COND_EXTEN)
     * The full header must be present regardless of the flags.
     */
    int32_t                     mgr[2];    /* For use by manager */
    int32_t                     flags_extra_mask;
    int32_t                     flags_exten;
    int32_t                     nfds;
    int32_t                     fd_first;
    int32_t                     nfds_ready;
    int64_t                     timo;
    /* struct pollfd            fds[nfds]; */
};

struct _io_notify64 {
    uint16_t                    type;
    uint16_t                    combine_len;
    int32_t                     action;
    int32_t                     flags;
    struct __sigevent32         old_event;

    /* The fields `mgr` to `timo` are only valid if (flags & _NOTIFY_COND_EXTEN)
     * The full header must be present regardless of the flags.
     */
    int32_t                     mgr[2];    /* For use by manager */
    int32_t                     flags_extra_mask;
    int32_t                     flags_exten;
    int32_t                     nfds;
    int32_t                     fd_first;
    int32_t                     nfds_ready;
    int64_t                     timo;
    union {
        struct __sigevent32     event32;
        struct __sigevent64     event64;
    };
    /* struct pollfd            fds[nfds]; */
};

struct _io_notify_reply {
    uint32_t                    zero;
    uint32_t                    flags;   /* actions above */
    int32_t                     flags2;  /* flags above */
    struct __sigevent32         event;

    /* Following fields only updated by new managers (if valid) */
    int32_t                     mgr[2];    /* For use by manager */
    int32_t                     flags_extra_mask;
    int32_t                     flags_exten;
    int32_t                     nfds;
    int32_t                     fd_first;
    int32_t                     nfds_ready;
    int64_t                     timo;
    /* struct pollfd            fds[nfds]; */
};

struct _io_notify_reply64 {
    uint32_t                    zero;
    uint32_t                    flags;  /* actions */
    int32_t                     flags2; /* flags above */
    struct __sigevent32         old_event;

    /* Following fields only updated by new managers (if valid) */
    int32_t                     mgr[2];    /* For use by manager */
    int32_t                     flags_extra_mask;
    int32_t                     flags_exten;
    int32_t                     nfds;
    int32_t                     fd_first;
    int32_t                     nfds_ready;
    int64_t                     timo;
    union {
        struct __sigevent32     event32;
        struct __sigevent64     event64;
    };
    /* struct pollfd            fds[nfds]; */
};

typedef union {
    struct _io_notify           i;
    struct _io_notify64         i64;
    struct _io_notify_reply     o;
    struct _io_notify_reply64   o64;
} io_notify_t;
Note: The code samples used in this chapter are not always POSIX-compliant.

As with all resource manager messages, we've defined a union that contains the input structure (coming into the resource manager), and a reply or output structure (going back to the client). The io_notify handler is prototyped with the argument:

io_notify_t *msg

which is the pointer to the union containing the message. The main items in the input structure are:

The type member has the value _IO_NOTIFY or _IO_NOTIFY64.

The combine_len field has meaning for a combine message; see the Combine Messages chapter.

The action member is used by the iofunc_notify() helper function to tell it whether it should:

Since iofunc_notify() looks at this, you don't have to worry about it.

The flags member contains the conditions that the client is interested in and can be any mixture of the following:

_NOTIFY_COND_INPUT
This condition is met when there are one or more units of input data available (i.e., clients can now issue reads). The number of units defaults to 1, but you can change it. The definition of a unit is up to you: for a character device such as a serial port, it would be a character; for a POSIX message queue, it would be a message. Each resource manager selects an appropriate object.
_NOTIFY_COND_OUTPUT
This condition is met when there's room in the output buffer for one or more units of data (i.e., clients can now issue writes). The number of units defaults to 1, but you can change it. The definition of a unit is up to you—some resource managers may default to an empty output buffer, while others may choose some percentage of the buffer empty.
_NOTIFY_COND_OBAND
The condition is met when one or more units of out-of-band data are available. The number of units defaults to 1, but you can change it. The definition of out-of-band data is specific to the resource manager.
_NOTIFY_COND_EXTEN
The conditions are defined with some extended flags; used internally.

The event* member is what the resource manager delivers once a condition is met.

A resource manager needs to keep a list of clients that want to be notified as conditions are met, along with the events to use to do the notifying. When a condition is met, the resource manager must traverse the list to look for clients that are interested in that condition, and then deliver the appropriate event. As well, if a client closes its file descriptor, then any notification entries for that client must be removed from the list.

To make all this easier, the following structure and helper functions are provided for you to use in a resource manager:

iofunc_notify_t structure
Contains the three notification lists, one for each possible condition. Each is a list of the clients to be notified for that condition.
iofunc_notify()
Adds or removes notification entries; also polls for conditions. Call this function inside your io_notify handler function.
iofunc_notify_trigger(), iofunc_notify_trigger_strict()
Sends notifications to queued clients. Call either of these functions when one or more conditions have been met. When the client closes its file descriptor, call iofunc_notify_trigger_strict() for each condition.
iofunc_notify_remove(), iofunc_notify_remove_strict()
Removes notification entries from the list. Call iofunc_notify_remove_strict() when the client closes its file descriptor.
CAUTION:

In a multi-threaded server, you must serialize access to each iofunc_notify_t structure. In many cases, this is done implicitly by the default locking on the extended attribute structure that holds device-specific data and notification data copied from that other structure. (For an example of this implicit locking, see the io_write handler in the coding example shown in the next section.) But if an iofunc_notify_t structure is accessed somewhere that this default locking has not occurred (e.g., in an interrupt-handling thread), then explicit locking must be done, either of the extended attribute structure or with an explicitly added lock for the iofunc_notify_t structure.

Failure to serialize access to these structures can lead to server-side race conditions, which can introduce bugs that are difficult to detect and fix.

Note: Don't return _RESMGR_NOREPLY from an io_notify handler, as it may be called multiple times from a single message if handling multiple file descriptors from a client call to select() or poll(). This is handled for you if you're using iofunc_notify().