Caution: This version of this document is no longer maintained. For the latest documentation, see

Handling Other Messages

This chapter includes:

Custom messages

Although most of the time your resource manager will handle messages from clients, there still could be times when you need to control the behavior of the resource manager itself. For example, if the resource manager is for a serial port, you'll likely need a way to change the baud rate, and so on.

There are various ways you could send control information to a resource manager:

Handling devctl() messages

The devctl() function is a general-purpose mechanism for communicating with a resource manager. Clients can send data to, receive data from, or both send and receive data from a resource manager. The prototype of the client devctl() call is:

int devctl( int fd,
            int dcmd, 
            void * data, 
            size_t nbytes, 
            int * return_info);

The following values (described in detail in the devctl() documentation in the QNX Neutrino Library Reference) map directly to the _IO_DEVCTL message itself:

struct _io_devctl {
        uint16_t                  type;
        uint16_t                  combine_len;
        int32_t                   dcmd;
        int32_t                   nbytes;
        int32_t                   zero;
/*      char                      data[nbytes]; */

struct _io_devctl_reply {
        uint32_t                  zero;
        int32_t                   ret_val;
        int32_t                   nbytes;
        int32_t                   zero2;
/*      char                      data[nbytes]; */
    } ;

typedef union {
        struct _io_devctl         i;
        struct _io_devctl_reply   o;
} io_devctl_t;

As with most resource manager messages, we've defined a union that contains the input structure (coming into the resource manager), and a reply or output structure (going back to the client). The io_devctl resource manager handler is prototyped with the argument:

io_devctl_t *msg

which is the pointer to the union containing the message.

The type member has the value _IO_DEVCTL.

The combine_len field has meaning for a combine message; see the Combine Messages chapter.

The nbytes value is the nbytes that's passed to the devctl() function. The value contains the size of the data to be sent to the device driver, or the maximum size of the data to be received from the device driver.

The most interesting item of the input structure is the dcmd argument that's passed to the devctl() function. This command is formed using the macros defined in <devctl.h>:

#define _POSIX_DEVDIR_NONE        0
#define _POSIX_DEVDIR_TO          0x80000000
#define _POSIX_DEVDIR_FROM        0x40000000
#define __DIOF(class, cmd, data)  ((sizeof(data)<<16) + ((class)<<8) + (cmd) + _POSIX_DEVDIR_FROM)
#define __DIOT(class, cmd, data)  ((sizeof(data)<<16) + ((class)<<8) + (cmd) + _POSIX_DEVDIR_TO)
#define __DIOTF(class, cmd, data) ((sizeof(data)<<16) + ((class)<<8) + (cmd) + _POSIX_DEVDIR_TOFROM)
#define __DION(class, cmd)        (((class)<<8) + (cmd) + _POSIX_DEVDIR_NONE)

It's important to understand how these macros pack data to create a command. An 8-bit class (defined in <devctl.h>) is combined with an 8-bit subtype that's manager-specific, and put together in the lower 16 bits of the integer.

The upper 16 bits contain the direction (TO, FROM) as well as a hint about the size of the data structure being passed. This size is only a hint put in to uniquely identify messages that may use the same class and code but pass different data structures.

In the following example, a command is generated to indicate that the client is sending data to the server (TO), but not receiving anything in return. The only bits that the library or the resource manager layer look at are the TO and FROM bits to determine which arguments are to be passed to MsgSend().

struct _my_devctl_msg {

#define MYDCMD  __DIOT(_DCMD_MISC, 0x54, struct _my_devctl_msg) 

Note: The size of the structure that's passed as the last field to the __DIO* macros must be less than 214 == 16 KB. Anything larger than this interferes with the upper two directional bits.

The data directly follows this message structure, as indicated by the /* char data[nbytes] */ comment in the _io_devctl structure.

Sample code for handling _IO_DEVCTL messages

You can add the following code samples to either of the /dev/null examples provided in the Simple device resource manager examples section of the Bones of a Resource Manager chapter. Both of those code samples provided the name /dev/sample. With the changes indicated below, the client can use devctl() to set and retrieve a global value (an integer in this case) that's maintained in the resource manager.

The first addition defines what the devctl() commands are going to be. This is generally put in a common or shared header file:

typedef union _my_devctl_msg {
        int tx;             /* Filled by client on send */
        int rx;             /* Filled by server on reply */
} data_t;

#define MY_CMD_CODE      1
#define MY_DEVCTL_SETGET __DIOTF(_DCMD_MISC, MY_CMD_CODE + 2, union _my_devctl_msg)

In the above code, we defined three commands that the client can use:

Sets the server's global variable to the integer the client provides.
Gets the value of the server's global variable and puts it into the client's buffer.
Sets the server's global variable to the integer that the client provides, and then returns the previous value of the server's global variable in the client's buffer.

Add this code to the main() function:

/* For handling _IO_DEVCTL, sent by devctl() */
io_funcs.devctl = io_devctl;

And the following code gets added before the main() function:

int io_devctl(resmgr_context_t *ctp, io_devctl_t *msg,
              RESMGR_OCB_T *ocb);

int global_integer = 0;

Now, you need to include the new handler function to handle the _IO_DEVCTL message (see the text following the listing for additional notes):

int io_devctl(resmgr_context_t *ctp, io_devctl_t *msg,
              RESMGR_OCB_T *ocb) {
    int     nbytes, status, previous;

    union {  /* See note 1 */
        data_t  data;
        int     data32;
        /* ... other devctl types you can receive */
    } *rx_data;

     Let common code handle DCMD_ALL_* cases.
     You can do this before or after you intercept devctls, depending
     on your intentions.  Here we aren't using any predefined values,
     so let the system ones be handled first. See note 2.
    if ((status = iofunc_devctl_default(ctp, msg, ocb)) !=
         _RESMGR_DEFAULT) {
    status = nbytes = 0;

     Note this assumes that you can fit the entire data portion of
     the devctl into one message.  In reality you should probably
     perform a MsgReadv() once you know the type of message you
     have received to get all of the data, rather than assume
     it all fits in the message.  We have set in our main routine
     that we'll accept a total message size of up to 2 KB, so we
     don't worry about it in this example where we deal with ints.

    /* Get the data from the message. See Note 3. */
    rx_data = _DEVCTL_DATA(msg->i);

     Three examples of devctl operations:
     SET: Set a value (int) in the server
     GET: Get a value (int) from the server
     SETGET: Set a new value and return the previous value
    switch (msg->i.dcmd) {
        global_integer = rx_data->data32;
        nbytes = 0;

        rx_data->data32 = global_integer; /* See note 4 */
        nbytes = sizeof(rx_data->data32);
        previous = global_integer; 
        global_integer = rx_data->data.tx;

        /* See note 4. The rx data overwrites the tx data
           for this command. */

        rx_data->data.rx = previous;
        nbytes = sizeof(rx_data->data.rx);


    /* Clear the return message. Note that we saved our data past
       this location in the message. */
    memset(&msg->o, 0, sizeof(msg->o));

     If you wanted to pass something different to the return
     field of the devctl() you could do it through this member.
     See note 5.
    msg->o.ret_val = status;

    /* Indicate the number of bytes and return the message */
    msg->o.nbytes = nbytes;
    return(_RESMGR_PTR(ctp, &msg->o, sizeof(msg->o) + nbytes));

Here are the notes for the above code:

  1. We define a union for all the possible types of received data. The MY_DEVCTL_SETVAL and MY_DEVCTL_GETVAL commands use the data32 member, and the MY_DEVCTL_SETGET uses the data member, of type data_t, which is a union of the received and transmitted data.
  2. The default devctl() handler is called before we begin to service our messages. This allows normal system messages to be processed. If the message isn't handled by the default handler, then it returns _RESMGR_DEFAULT to indicate that the message might be a custom message. This means that we should check the incoming command against commands that our resource manager understands.
  3. The data to be passed follows directly after the io_devctl_t structure. You can get a pointer to this location by using the _DEVCTL_DATA(msg->i) macro defined in <devctl.h>. The argument to this macro must be the input message structure — if it's the union message structure or a pointer to the input message structure, the pointer won't point to the right location.

    For your convenience, we've defined a union of all of the messages that this server can receive. However, this won't work with large data messages. In this case, you'd use resmgr_msgread() to read the message from the client. Our messages are never larger than sizeof( int) and this comfortably fits into the minimum receive buffer size.

  4. The data being returned to the client is placed at the end of the reply message. This is the same mechanism used for the input data, so we can use the _DEVCTL_DATA() function to get a pointer to this location. With large replies that wouldn't necessarily fit into the server's receive buffer, you should use one of the reply mechanisms described in the Methods of returning and replying section in the Handling Read and Write Messages chapter. Again, in this example, we're only returning an integer that fits into the receive buffer without any problem.
  5. The last argument to the devctl() function is a pointer to an integer. If this pointer is provided, then the integer is filled with the value stored in the msg->o.ret_val reply message. This is a convenient way for a resource manager to return simple status information without affecting the core devctl() operation. It's not used in this example.

If you add the following handler code, a client should be able to open /dev/sample and subsequently set and retrieve the global integer value:

int main(int argc, char **argv) {
    int     fd, ret, val;
    data_t  data;

    if ((fd = open("/dev/sample", O_RDONLY)) == -1) {

    /* Find out what the value is set to initially */
    val = -1;
    ret = devctl(fd, MY_DEVCTL_GETVAL, &val, sizeof(val), NULL);
    printf("GET returned %d w/ server value %d \n", ret, val);

    /* Set the value to something else */
    val = 25;
    ret = devctl(fd, MY_DEVCTL_SETVAL, &val, sizeof(val), NULL);
    printf("SET returned %d \n", ret);

    /* Verify we actually did set the value */
    val = -1;
    ret = devctl(fd, MY_DEVCTL_GETVAL, &val, sizeof(val), NULL);
    printf("GET returned %d w/ server value %d == 25? \n", ret, val);

    /* Now do a set/get combination */
    memset(&data, 0, sizeof(data));
    data.tx = 50;
    ret = devctl(fd, MY_DEVCTL_SETGET, &data, sizeof(data), NULL);
    printf("SETGET returned with %d w/ server value %d == 25?\n",
           ret, data.rx);

    /* Check set/get worked */
    val = -1;
    ret = devctl(fd, MY_DEVCTL_GETVAL, &val, sizeof(val), NULL);
    printf("GET returned %d w/ server value %d == 50? \n", ret, val);


Handling ionotify() and select()

A client uses ionotify() and select() to ask a resource manager about the status of certain conditions (e.g. whether input data is available). The conditions may or may not have been met. The resource manager can be asked to:

The select() function differs from ionotify() in that most of the work is done in the library. For example, the client code would be unaware that any event is involved, nor would it be aware of the blocking function that waits for the event. This is all hidden in the library code for select().

However, from a resource manager's point of view, there's no difference between ionotify() and select(); they're handled with the same code.

For more information on the ionotify() and select() functions, see the QNX Neutrino Library Reference.

Note: If multiple threads in the same client perform simultaneous operations with select() and ionotify(), notification races may occur.

Since ionotify() and select() require the resource manager to do the same work, they both send the _IO_NOTIFY message to the resource manager. The io_notify handler is responsible for handling this message. Let's start by looking at the format of the message itself:

struct _io_notify {
    uint16_t                    type;
    uint16_t                    combine_len;
    int32_t                     action;
    int32_t                     flags;
    struct sigevent             event;

struct _io_notify_reply {
    uint32_t                    flags;

typedef union {
    struct _io_notify           i;
    struct _io_notify_reply     o;
} io_notify_t;

Note: The code samples used in this chapter are not always POSIX-compliant.

As with all resource manager messages, we've defined a union that contains the input structure (coming into the resource manager), and a reply or output structure (going back to the client). The io_notify handler is prototyped with the argument:

io_notify_t *msg

which is the pointer to the union containing the message. The items in the input structure are:

The type member has the value _IO_NOTIFY.

The combine_len field has meaning for a combine message; see the Combine Messages chapter.

The action member is used by the iofunc_notify() helper function to tell it whether it should:

Since iofunc_notify() looks at this, you don't have to worry about it.

The flags member contains the conditions that the client is interested in and can be any mixture of the following:

This condition is met when there are one or more units of input data available (i.e. clients can now issue reads). The number of units defaults to 1, but you can change it. The definition of a unit is up to you: for a character device such as a serial port, it would be a character; for a POSIX message queue, it would be a message. Each resource manager selects an appropriate object.
This condition is met when there's room in the output buffer for one or more units of data (i.e. clients can now issue writes). The number of units defaults to 1, but you can change it. The definition of a unit is up to you — some resource managers may default to an empty output buffer while others may choose some percentage of the buffer empty.
The condition is met when one or more units of out-of-band data are available. The number of units defaults to 1, but you can change it. The definition of out-of-band data is specific to the resource manager.
The conditions are defined with the following extended flags:

The event member is what the resource manager delivers once a condition is met.

A resource manager needs to keep a list of clients that want to be notified as conditions are met, along with the events to use to do the notifying. When a condition is met, the resource manager must traverse the list to look for clients that are interested in that condition, and then deliver the appropriate event. As well, if a client closes its file descriptor, then any notification entries for that client must be removed from the list.

To make all this easier, the following structure and helper functions are provided for you to use in a resource manager:

iofunc_notify_t structure
Contains the three notification lists, one for each possible condition. Each is a list of the clients to be notified for that condition.
Adds or removes notification entries; also polls for conditions. Call this function inside your io_notify handler function.
Sends notifications to queued clients. Call this function when one or more conditions have been met.
Removes notification entries from the list. Call this function when the client closes its file descriptor.

Note: Don't return _RESMGR_NOREPLY from an io_notify handler, as it may be called multiple times from a single message if handling multiple file descriptors from a client call to select() or poll(). This is handled for you if you're using iofunc_notify().

Sample code for handling _IO_NOTIFY messages

You can add the following code samples to either of the examples provided in the Simple device resource manager examples section of the Bones of a Resource Manager chapter. Both of those code samples provided the name /dev/sample. With the changes indicated below, clients can use writes to send it data, which it'll store as discrete messages. Other clients can use either ionotify() or select() to request notification when that data arrives. When clients receive notification, they can issue reads to get the data.

You'll need to replace this code that's located above the main() function:

#include <sys/iofunc.h>
#include <sys/dispatch.h>

static resmgr_connect_funcs_t    connect_funcs;
static resmgr_io_funcs_t         io_funcs;
static iofunc_attr_t             attr;

with the following:

struct device_attr_s;
#define IOFUNC_ATTR_T   struct device_attr_s

#include <sys/iofunc.h>
#include <sys/dispatch.h>

 * Define a structure and variables for storing the data that
 * is received. When clients write data to us, we store it here.
 * When clients do reads, we get the data from here.  Result: a
 * simple message queue.
typedef struct item_s {
    struct item_s   *next;
    char            *data;
} item_t;

/* the extended attributes structure */
typedef struct device_attr_s {
    iofunc_attr_t   attr;
    iofunc_notify_t notify[3];  /* notification list used by
                                   iofunc_notify*() */
    item_t          *firstitem; /* the queue of items */
    int             nitems;     /* number of items in the queue */
} device_attr_t;

/* We only have one device; device_attr is its attribute structure */

static device_attr_t    device_attr;

int io_read( resmgr_context_t *ctp, io_read_t  *msg,
             RESMGR_OCB_T *ocb);
int io_write( resmgr_context_t *ctp, io_write_t *msg,
              RESMGR_OCB_T *ocb);
int io_notify( resmgr_context_t *ctp, io_notify_t *msg,
              RESMGR_OCB_T *ocb);
int io_close_ocb( resmgr_context_t *ctp, void *reserved,
                  RESMGR_OCB_T *ocb);

static resmgr_connect_funcs_t  connect_funcs;
static resmgr_io_funcs_t       io_funcs;

We need a place to keep data that's specific to our device. A good place for this is in an attribute structure that we can associate with the name we registered: /dev/sample. So, in the code above, we defined device_attr_t and IOFUNC_ATTR_T for this purpose. We talk more about this type of device-specific attribute structure in the Extending the POSIX-Layer Data Structures chapter.

We need two types of device-specific data:

Note that we removed the definition of attr, since we use device_attr instead.

Of course, we have to give the resource manager library the address of our handlers so that it'll know to call them. In the code for main() where we called iofunc_func_init(), we'll add the following code to register our handlers:

/* initialize functions for handling messages */
iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &connect_funcs,
                 _RESMGR_IO_NFUNCS, &io_funcs);

/* For handling _IO_NOTIFY, sent as a result of client
   calls to ionotify() and select() */
io_funcs.notify = io_notify;

io_funcs.write = io_write; = io_read;
io_funcs.close_ocb = io_close_ocb;

And, since we're using device_attr in place of attr, we need to change the code wherever we use it in main(). So, you'll need to replace this code:

/* initialize attribute structure used by the device */
iofunc_attr_init(&attr, S_IFNAM | 0666, 0, 0);

/* attach our device name */
id = resmgr_attach(dpp,            /* dispatch handle        */
                   &resmgr_attr,   /* resource manager attrs */
                   "/dev/sample",  /* device name            */
                   _FTYPE_ANY,     /* open type              */
                   0,              /* flags                  */
                   &connect_funcs, /* connect routines       */
                   &io_funcs,      /* I/O routines           */
                   &attr);         /* handle                 */

with the following:

/* initialize attribute structure used by the device */
iofunc_attr_init(&device_attr.attr, S_IFNAM | 0666, 0, 0);
device_attr.firstitem = NULL;
device_attr.nitems = 0;

/* attach our device name */
id = resmgr_attach(dpp,            /* dispatch handle        */
                   &resmgr_attr,   /* resource manager attrs */
                   "/dev/sample",  /* device name            */
                   _FTYPE_ANY,     /* open type              */
                   0,              /* flags                  */
                   &connect_funcs, /* connect routines       */
                   &io_funcs,      /* I/O routines           */
                   &device_attr);  /* handle                 */

Note that we set up our device-specific data in device_attr. And, in the call to resmgr_attach(), we passed &device_attr (instead of &attr) for the handle parameter.

Now, you need to include the new handler function to handle the _IO_NOTIFY message:

io_notify( resmgr_context_t *ctp, io_notify_t *msg,
           RESMGR_OCB_T *ocb)
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             trig;
     * 'trig' will tell iofunc_notify() which conditions are
     * currently satisfied.  'dattr->nitems' is the number of
     * messages in our list of stored messages.

    trig = _NOTIFY_COND_OUTPUT; /* clients can always give us data */
    if (dattr->nitems > 0)
        trig |= _NOTIFY_COND_INPUT; /* we have some data available */
     * iofunc_notify() will do any necessary handling, including
     * adding the client to the notification list if need be.

    return (iofunc_notify( ctp, msg, dattr->notify, trig,
                           NULL, NULL));

As stated above, our io_notify handler will be called when a client calls ionotify() or select(). In our handler, we're expected to remember who those clients are, and what conditions they want to be notified about. We should also be able to respond immediately with conditions that are already true. The iofunc_notify() helper function makes this easy.

The first thing we do is to figure out which of the conditions we handle have currently been met. In this example, we're always able to accept writes, so in the code above we set the _NOTIFY_COND_OUTPUT bit in trig. We also check nitems to see if we have data and set the _NOTIFY_COND_INPUT if we do.

We then call iofunc_notify(), passing it the message that was received (msg), the notification lists (notify), and which conditions have been met (trig). If one of the conditions that the client is asking about has been met, and the client wants us to poll for the condition before arming, then iofunc_notify() will return with a value that indicates what condition has been met and the condition will not be armed. Otherwise, the condition will be armed. In either case, we'll return from the handler with the return value from iofunc_notify().

Earlier, when we talked about the three possible conditions, we mentioned that if you specify _NOTIFY_COND_INPUT, the client is notified when there's one or more units of input data available and that the number of units is up to you. We said a similar thing about _NOTIFY_COND_OUTPUT and _NOTIFY_COND_OBAND. In the code above, we let the number of units for all these default to 1. If you want to use something different, then you must declare an array such as:

int notifycounts[3] =  { 10, 2, 1 };

This sets the units for: _NOTIFY_COND_INPUT to 10; _NOTIFY_COND_OUTPUT to 2; and _NOTIFY_COND_OBAND to 1. We would pass notifycounts to iofunc_notify() as the second to last parameter.

Then, as data arrives, we notify whichever clients have asked for notification. In this sample, data arrives through clients sending us _IO_WRITE messages and we handle it using an io_write handler.

io_write(resmgr_context_t *ctp, io_write_t *msg,
         RESMGR_OCB_T *ocb)
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             i;
    char            *p;
    int             status;
    char            *buf;
    item_t          *newitem;

    if ((status = iofunc_write_verify(ctp, msg, ocb, NULL))
         != EOK)
        return (status);

    if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
        return (ENOSYS);

    if (msg->i.nbytes > 0) {
        /* Get and store the data */
        if ((newitem = malloc(sizeof(item_t))) == NULL)
            return (errno);
        if ((newitem->data = malloc(msg->i.nbytes+1)) ==
            NULL) {
            return (errno);
        /* reread the data from the sender's message buffer */
        resmgr_msgread(ctp, newitem->data, msg->i.nbytes,
        newitem->data[msg->i.nbytes] = NULL;

        if (dattr->firstitem)
            newitem->next = dattr->firstitem;
            newitem->next = NULL;
        dattr->firstitem = newitem;

         * notify clients who may have asked to be notified
         * when there is data
        if (IOFUNC_NOTIFY_INPUT_CHECK(dattr->notify,
            dattr->nitems, 0))
            iofunc_notify_trigger(dattr->notify, dattr->nitems,
    /* set up the number of bytes (returned by client's
       write()) */
    _IO_SET_WRITE_NBYTES(ctp, msg->i.nbytes);

    if (msg->i.nbytes > 0)
        ocb->attr->attr.flags |= IOFUNC_ATTR_MTIME |

    return (_RESMGR_NPARTS(0));

The important part of the above io_write handler is the code within the following section:

if (msg->i.nbytes > 0) {

Here we first allocate space for the incoming data, and then use resmgr_msgread() to copy the data from the client's send buffer into the allocated space. Then, we add the data to our queue.

Next, we pass the number of input units that are available to IOFUNC_NOTIFY_INPUT_CHECK() to see if there are enough units to notify clients about. This is checked against the notifycounts that we mentioned above when talking about the io_notify handler. If there are enough units available then we call iofunc_notify_trigger() telling it that nitems of data are available (IOFUNC_NOTIFY_INPUT means input is available). The iofunc_notify_trigger() function checks the lists of clients asking for notification (notify) and notifies any that asked about data being available.

Any client that gets notified will then perform a read to get the data. In our sample, we handle this with the following io_read handler:

io_read(resmgr_context_t *ctp, io_read_t *msg, RESMGR_OCB_T *ocb)
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             status;
    if ((status = iofunc_read_verify(ctp, msg, ocb, NULL)) != EOK)
        return (status);

    if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
        return (ENOSYS);

    if (dattr->firstitem) {
        int     nbytes;
        item_t  *item, *prev;
        /* get last item */
        item = dattr->firstitem;
        prev = NULL;
        while (item->next != NULL) {
            prev = item;
            item = item->next;

         * figure out number of bytes to give, write the data to the 
         * client's reply buffer, even if we have more bytes than they
         * are asking for, we remove the item from our list
        nbytes = min (strlen (item->data), msg->i.nbytes);

        /* set up the number of bytes (returned by client's read()) */
        _IO_SET_READ_NBYTES (ctp, nbytes);

         * write the bytes to the client's reply buffer now since we
         * are about to free the data
        resmgr_msgwrite (ctp, item->data, nbytes, 0);

        /* remove the data from the queue */
        if (prev)
            prev->next = item->next;
            dattr->firstitem = NULL;
    } else {
        /* the read() will return with 0 bytes */
        _IO_SET_READ_NBYTES (ctp, 0);

    /* mark the access time as invalid (we just accessed it) */

    if (msg->i.nbytes > 0)
        ocb->attr->attr.flags |= IOFUNC_ATTR_ATIME;

    return (EOK);

The important part of the above io_read handler is the code within this section:

if (firstitem) {

We first walk through the queue looking for the oldest item. Then we use resmgr_msgwrite() to write the data to the client's reply buffer. We do this now because the next step is to free the memory that we're using to store that data. We also remove the item from our queue.

Lastly, if a client closes its file descriptor, we must remove the client from our list. This is done using a io_close_ocb handler:

int io_close_ocb( resmgr_context_t *ctp, void *reserved,
                  RESMGR_OCB_T *ocb)
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;

     * A client has closed its file descriptor or has terminated.
     * Remove the client from the notification list.
    iofunc_notify_remove(ctp, dattr->notify);

    return (iofunc_close_ocb_default(ctp, reserved, ocb));

In the io_close_ocb handler, we called iofunc_notify_remove() and passed it ctp (contains the information that identifies the client) and notify (contains the list of clients) to remove the client from the lists.

Handling out-of-band (_IO_MSG) messages

An _IO_MSG message lets a client send an “out-of-band” or control message to a resource manager, by way of a file descriptor. This interface is more general than an ioctl() or devctl(), but less portable.

The format of the message is specific to the resource manager, aside from the header, which we'll look at shortly. The client program sets up the message and uses MsgSend() to send it to the resource manager. The resource manager must set up an io_msg handler in order to receive the message; there isn't a default handler.

The message header is defined in <sys/iomsg.h> and looks like this:

struct _io_msg {
    _Uint16t    type;
    _Uint16t    combine_len;
    _Uint16t    mgrid;
    _Uint16t    subtype;

The fields include:

Set this to sizeof (struct _io_msg).
A unique ID for your resource manager. The <sys/iomgr.h> header file defines some IDs that are reserved for various QNX resource managers. If you're sure that your resource manager will never get an _IO_MSG message that isn't intended for it (for example, the resource manager will only run in an embedded system), you can use an ID in the range from _IOMGR_PRIVATE_BASE through _IOMGR_PRIVATE_MAX. If your resource manager will be used in a more open system, contact QNX Software Systems and reserve a manager ID or range of IDs.
Use this field to distinguish different types of _IO_MSG messages that you want your resource manager to handle.

Any data should follow this header. For example:

typedef struct {
    struct _io_msg hdr;

    /* Add any required data fields here. */

} my_msg_t;

The client program would then do something like this:


my_msg_t msg, my_reply;
int fd, status;

fd = open ("/dev/sample", O_RDWR);
msg.hdr.type = _IO_MSG;
msg.hdr.combine_len = sizeof( msg.hdr );
msg.hdr.mgrid = MY_MGR_ID;
msg.hdr.subtype = 0;

/* Fill in the additional fields as required. */
status = MsgSend( fd, &msg, sizeof( msg ), &my_reply,
                  sizeof (my_reply));

The resource manager registers a function to handle the _IO_MSG messages:

/* Initialize the functions for handling messages */
iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &connect_funcs,
                 _RESMGR_IO_NFUNCS, &io_funcs);

io_funcs.msg = my_io_msg;

This handler processes the message as appropriate. For example:

int my_io_msg (resmgr_context_t *ctp, io_msg_t *msg,
               RESMGR_OCB_T *ocb)
    my_msg_t my_msg;
    MsgRead (ctp->rcvid, &my_msg, sizeof (my_msg), 0);
    if (my_msg.hdr.mgrid != MY_MGR_ID)
        return (ENOSYS);

    /* Process the data as required. */
    /* Reply if necessary and tell the library that we've
       already replied. */

    MsgReply( ctp->rcvid, 0, &my_reply, sizeof(my_reply));
    return (_RESMGR_NOREPLY);

Note that the handler returns ENOSYS if the mgrid member of the header isn't the correct manager ID. This handler replies to the client, and then returns _RESMGR_NOREPLY to tell the library that there's no need for it to do the reply.

Handling private messages and pulses

A resource manager may need to receive and handle pulses, perhaps because an interrupt handler has returned a pulse or some other thread or process has sent a pulse.

The main issue with pulses is that they have to be received as a message. This means that a thread has to explicitly perform a MsgReceive() in order to get the pulse. But unless this pulse is sent to a different channel than the one that the resource manager is using for its main messaging interface, it will be received by the library. Therefore, we need to see how a resource manager can associate a pulse code with a handler routine and communicate that information to the library.

You can use the pulse_attach() function to associate a pulse code with a handler function. When the dispatch layer receives a pulse, it will look up the pulse code and see which associated handler to call to handle the pulse message.

You may also want to define your own private message range to communicate with your resource manager. Note that the range 0x0 to 0x1FF is reserved for the OS. To attach a range, you use the message_attach() function.

In this example, we create the same resource manager, but this time we also attach to a private message range and attach a pulse, which is then used as a timer event:

#include <stdio.h>
#include <stddef.h>
#include <stdlib.h>

#define THREAD_POOL_PARAM_T     dispatch_context_t
#include <sys/iofunc.h>
#include <sys/dispatch.h>

static resmgr_connect_funcs_t   connect_func;
static resmgr_io_funcs_t        io_func;
static iofunc_attr_t            attr;

timer_tick( message_context_t *ctp, int code, unsigned flags,
            void *handle)
    union sigval             value = ctp->msg->pulse.value;
     *  Do some useful work on every timer firing
     *  ....
    printf("received timer event, value %d\n", value.sival_int);
    return 0;

message_handler( message_context_t *ctp, int code, unsigned flags,
                 void *handle)
    printf("received private message, type %d\n", code);
    return 0;

main(int argc, char **argv) {
    thread_pool_attr_t    pool_attr;
    resmgr_attr_t         resmgr_attr;
    struct sigevent       event;
    struct _itimer        itime;
    dispatch_t            *dpp;
    thread_pool_t         *tpp;
    resmgr_context_t      *ctp;
    int                   timer_id;
    int                   id;

    if((dpp = dispatch_create()) == NULL) {
                "%s: Unable to allocate dispatch handle.\n",
        return EXIT_FAILURE;

    memset(&pool_attr, 0, sizeof pool_attr);
    pool_attr.handle = dpp;
    /*  We are doing resmgr and pulse-type attaches.
     *  If you're going to use custom messages or pulses with 
     *  the message_attach() or pulse_attach() functions,
     *  then you MUST use the dispatch functions 
     *  (i.e. dispatch_block(),  dispatch_handler(), ...),
     *  NOT the resmgr functions (resmgr_block(), resmgr_handler()).
    pool_attr.context_alloc = dispatch_context_alloc;
    pool_attr.block_func = dispatch_block; 
    pool_attr.unblock_func = dispatch_unblock;
    pool_attr.handler_func = dispatch_handler;
    pool_attr.context_free = dispatch_context_free;
    pool_attr.lo_water = 2;
    pool_attr.hi_water = 4;
    pool_attr.increment = 1;
    pool_attr.maximum = 50;

    if((tpp = thread_pool_create(&pool_attr, POOL_FLAG_EXIT_SELF)) == NULL) {
        fprintf(stderr, "%s: Unable to initialize thread pool.\n",argv[0]);
        return EXIT_FAILURE;

    iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &connect_func, _RESMGR_IO_NFUNCS,
    iofunc_attr_init(&attr, S_IFNAM | 0666, 0, 0);
    memset(&resmgr_attr, 0, sizeof resmgr_attr);
    resmgr_attr.nparts_max = 1;
    resmgr_attr.msg_max_size = 2048;

    if((id = resmgr_attach(dpp, &resmgr_attr, "/dev/sample", _FTYPE_ANY, 0,
                 &connect_func, &io_func, &attr)) == -1) {
        fprintf(stderr, "%s: Unable to attach name.\n", argv[0]);
        return EXIT_FAILURE;

    /* We want to handle our own private messages, of type 0x5000 to 0x5fff */
    if(message_attach(dpp, NULL, 0x5000, 0x5fff, &message_handler, NULL) == -1) {
        fprintf(stderr, "Unable to attach to private message range.\n");
         return EXIT_FAILURE;

    /* Initialize an event structure, and attach a pulse to it */
    if((event.sigev_code = pulse_attach(dpp, MSG_FLAG_ALLOC_PULSE, 0, &timer_tick,
                                        NULL)) == -1) {
        fprintf(stderr, "Unable to attach timer pulse.\n");
         return EXIT_FAILURE;

    /* Connect to our channel */
    if((event.sigev_coid = message_connect(dpp, MSG_FLAG_SIDE_CHANNEL)) == -1) {
        fprintf(stderr, "Unable to attach to channel.\n");
        return EXIT_FAILURE;

    event.sigev_notify = SIGEV_PULSE;
    event.sigev_priority = -1;
    /* We could create several timers and use different sigev values for each */
    event.sigev_value.sival_int = 0;

    if((timer_id = TimerCreate(CLOCK_REALTIME, &event)) == -1) {;
        fprintf(stderr, "Unable to attach channel and connection.\n");
        return EXIT_FAILURE;

    /* And now set up our timer to fire every second */
    itime.nsec = 1000000000;
    itime.interval_nsec = 1000000000;
    TimerSettime(timer_id, 0, &itime, NULL);

    /* Never returns */

We can either define our own pulse code (e.g. #define OurPulseCode 57), or we can ask the pulse_attach() function to dynamically generate one for us (and return the pulse code value as the return code from pulse_attach()) by specifying the pulse code as _RESMGR_PULSE_ALLOC.

See the pulse_attach(), MsgSendPulse(), MsgDeliverEvent(), and MsgReceive() functions in the QNX Neutrino Library Reference for more information on receiving and generating pulses.

Handling open(), dup(), and close() messages

The resource manager library provides another convenient service for us: it knows how to handle dup() messages.

Suppose that the client executed code that eventually ended up performing:

fd = open ("/dev/sample", O_RDONLY);
fd2 = dup (fd);
fd3 = dup (fd);
close (fd3);
close (fd2);
close (fd);

Our resource manager would get an _IO_CONNECT message for the first open(), followed by two _IO_DUP messages for the two dup() calls. Then, when the client executed the close() calls, we would get three _IO_CLOSE messages.

Since the dup() functions generate duplicates of the file descriptors, we don't want to allocate new OCBs for each one. And since we're not allocating new OCBs for each dup(), we don't want to release the memory in each _IO_CLOSE message when the _IO_CLOSE messages arrive! If we did that, the first close would wipe out the OCB.

The resource manager library knows how to manage this for us; it keeps count of the number of _IO_DUP and _IO_CLOSE messages sent by the client. Only on the last _IO_CLOSE message will the library synthesize a call to our _IO_CLOSE_OCB handler.

Note: Most users of the library will want to have the default functions manage the _IO_DUP and _IO_CLOSE messages; you'll most likely never override the default actions.

Handling mount()

Mount requests can provide a very convenient and flexible interface for programs that need to enable and disable components of their resource managers' systems.

The main areas to consider when using and building mount functionality into your resource manager are:

These components represent the stream of communication for the mount request. Let's start in the middle with the mount() function call and work our way out.

The mount() function call is at the bottom of the utility and represents a client's access point to the resource manager. The function is implemented in the C library, defined in <sys/mount.h>, and described in the QNX Neutrino Library Reference.

mount() function call

The prototype for mount() is as follows:

int mount( const char *special_device, 
           const char *mount_directory, 
           int flags, 
           const char *mount_type, 
           const void *mount_data, 
           int mount_datalen); 

The argument that we need to consider here is the flags field.

To support the mounting of non-existent special devices (such as NFS devices) or arbitrary strings (such as the name of shared object or DLL), we need to massage the arguments to this function slightly because the mount utility has two methods (-T and -t) for specifying the mount type.

In the general case where special_device is an actual device, a typical mount command may look like:

% mount -t qnx4 /dev/hd0t77 /mnt/fs

In this case the special_device is /dev/hd0t77, the mount_directory is /mnt/fs, and the mount type is qnx4. In this case, the mount request should be directed only to the process responsible for managing the special_device. That is to say the resource manager that has provided the /dev/hd0t77 path into the pathname space. In this type of scenario, the resource manager is given an OCB for the special_device (/dev/hd0t77), rather than the string /dev/hd0t77. This simplifies the processing in the resource manager since having the OCB, which is an internal data pointer, for the special device implies that the server doesn't have to recursively communicate with itself to get a handle for the device.

A less frequently used, but very useful case, is where the special_device isn't an actual device. For example:

% mount -T io-pkt /lib/dll/

Note that the mountpoint is missing from the command line. In this case, NULL (or /) acts as an implied mount_directory, which causes the process handling the request (i.e. the currently running variant of io-pkt) to take the appropriate action when it receives the mount request. The special_device is /lib/dll/ and the type is io-pkt.

In this case, you want to avoid having the special device interpreted as being provided by the same process that will handle the mount request. So while the file /lib/dll/ is probably handled by some filesystem process, we're actually interested in mounting a network interface that's managed by the io-pkt process. Ideally, the mount callout will receive only the special device string /lib/dll/, and not the OCB for the device.

The behavioral difference between the -t and -T options for the mount utility can be obtained by ORing in _MFLAG_OCB to the standard mount() flags parameter. If you don't want to use the OCB method of performing the mount request, use -T, which we translate to _MFLAG_OCB.

Mount requests are connection requests, which means they operate on a path in the same way that the open() or unlink() calls do. The requests are sent along the path specified by dir. When a resource manager receives a request to mount something, the information is already provided in the same way that it would be for an open() for creation request, namely in the msg->connect.path variable.

For more information on the name-resolution process, see the example Using MsgSend() and MsgReply() in the Bones of a Resource Manager chapter in this guide.

Mount in the resource manager

Your resource manager will be called upon to perform a mount request via the mount function callout in the resmgr_connect_funcs_t structure, defined as:

int mount( resmgr_context_t *ctp,
           io_mount_t *msg,
           RESMGR_HANDLE_T *handle,
           io_mount_extra_t *extra);

The only field here that differs from the other connect functions is the io_mount_extra_t structure. It's defined in <sys/iomsg.h> as:

typedef struct _io_mount_extra {
    uint32_t flags; /* _MOUNT_? or ST_? flags above */
    uint32_t nbytes; /* Size of entire structure */
    uint32_t datalen; /* Length of the data structure following */
    uint32_t zero[1];

    union { /* If EXTRA_MOUNT_PATHNAME these set*/
        struct { /* Sent from client to resmgr framework */
            struct _msg_info info; /* Special info on first mount,
                                      path info on remount */

        } cl;

        struct { /* Server receives this structure filled in */
            void * ocb; /* OCB to the special device */
            void * data; /* Server specific data of len datalen */
            char * type; /* Character string with type information */
            char * special; /* Optional special device info */
            void * zero[4]; /* Padding */
        } srv;
    } extra;
} io_mount_extra_t;

This structure is provided with all of the pointers already resolved, so you can use it without doing any extra fiddling.

The members are:

Flag fields provided to the mount command containing the common mount flags defined in <sys/mount.h>.
Size of the entire mount-extra message:
sizeof(_io_mount_extra) + datalen + strlen(type)
+ 1 + strlen(special) + 1
Size of the data pointer.
Used by the resource manager layer.
OCB of the special device if it was requested via the _MOUNT_OCB flag. NULL otherwise.
Pointer to the user data of length datalen.
Null-terminated string containing the mount type, such as nfs, cifs, or qnx4.
Null-terminated string containing the special device if it was requested via the _MOUNT_SPEC flag. NULL otherwise.

In order to receive mount requests, the resource manager should register a NULL path with an FTYPE of _FTYPE_MOUNT and with the flags _RESMGR_FLAG_FTYPEONLY. This would be done with code that looks something like:

mntid = resmgr_attach(
           dpp, /* Dispatch pointer */
           &resmgr_attr, /* Dispatch attributes */
           NULL, /* Attach at "/" */

           /* We are a directory and want only matching ftypes */

           mount_connect, /* Only mount filled in */
           NULL, /* No io handlers */
           & handle); /* Handle to pass to mount callout */

Again, we're attaching at the root of the filesystem so that we'll be able to receive the full path of the new mount requests in the msg->connect structure.

Adding the _RESMGR_FLAG_FTYPEONLY flag ensures that this request is used only when there's an _FTYPE_MOUNT-style of connection. Once this is done, the resource manager is ready to start receiving mount requests from users.

An outline of a sample mount handler would look something like this:

int io_mount( ... ) {

   Do any sanity checks that you need to do.

   Check type against our type with strcmp(), since
   there may be no name for REMOUNT/UNMOUNT flags.

   Error with ENOENT out if no match.

   If no name, check the validity of the REMOUNT/UNMOUNT request.

   Parse arguments or set up your data structure.

   Check to see if we are remounting (_MOUNT_REMOUNT)

      Change flags, etc., if you can remount.
      Return EOK.

   Check to see if we are unmounting _MOUNT_UNMOUNT

      Change flags, etc., if you can unmount.
      Return EOK.

   Create a new node and attach it at the msg->connect.path
   point (unless some other path is implied based on the
   input variables and the resource manager) with resmgr_attach().

   Return EOK.

What's important to notice here is that each resource manager that registers a mount handler will potentially get a chance to examine the request to see if it can handle it. This means that you have to be rigorous in your type- and error-checking to make sure that the request is indeed destined for your manager. If your manager returns anything other than ENOSYS or ENOENT, it's assumed that the request was valid for this manager, but there was some other sort of error. Only errors of ENOSYS or ENOENT cause the request to “fall through” to other resource managers.

When you unmount, you would perform any cleanup and integrity checks that you need, and then call resmgr_detach() with the ctp->id field. In general, you should support umounted calls only on the root of a mounted filesystem.

mount utility

By covering the mount() library function and the operation in the resource manager, we've pretty well covered the mount utility. The usage for the utility is shown here for reference:

mount [-wreuv] -t type [-o options] [special] mntpoint

mount [-wreuv] -T type [-o options] special [mntpoint]


The options are:

Indicates the special device, if it's present, is generally a real device and the same server will handle the mountpoint.
Indicates the special device isn't a real device but rather a key for the server. The server will automatically create an appropriate mountpoint if mntpoint isn't specified.
Increases the verbosity.
Mount read/write.
Mount read-only.
Mount for update (remount).

However, if you're writing a mount handler, there may be occasions when you want to do custom parsing of arguments and provide your own data structure to your server. This is why the mount command will always first try and call out to a separate program named mount_XXX, where XXX is the type that you specified with the -t option. To see just what would be called (in terms of options, etc.), you can use the -v option, which should provide you with the command line that would be exec()'ed.

In order to help with the argument parsing, there's a utility function, mount_parse_generic_args(), that you can call to process the common options. The function is defined in <sys/mount.h> as:

char *mount_parse_generic_args(char *options, int *flags);

This function parses the given options, removes any options that it recognizes, and sets or clears the appropriate bits in the flags. It returns a pointer to the modified version of options containing any options that it didn't recognize, or NULL if it recognized all the options. You use mount_parse_generic_args() like this:

while ((c = getopt(argv, argc, "o:"))) {
   switch (c) {

      case 'o':

        if ((mysteryop = mount_parse_generic_args(optarg, &flags))) {

           /* You can do your own getsubopt-type processing here.
              The common options are removed from mysteryop. */


For more information about the stripped options and the corresponding flags, see the entry for mount_parse_generic_args(), in the QNX Neutrino Library Reference.

Handling stat()

Your resource manager will receive an _IO_STAT message when a client calls stat(), lstat(), or fstat(). You usually don't need to provide your own handler for this message. The prototype for the io_stat handler is as follows:

int io_stat ( resmgr_context_t *ctp,
              io_stat_t *msg,
              RESMGR_OCB_T *ocb)

The default handler for the _IO_STAT message, iofunc_stat_default(), calls iofunc_time_update() to ensure that the time entries in the ocb->attr structure are current and valid, and then calls the iofunc_stat() helper function to fill in the stat structure based on the information in the ocb->attr structure.

The io_stat_t structure holds the _IO_STAT message received by the resource manager:

struct _io_stat {
    uint16_t                    type;
    uint16_t                    combine_len;
    uint32_t                    zero;

typedef union {
    struct _io_stat             i;
    struct stat                 o;
} io_stat_t;

As with all the I/O messages, this structure is a union of an input message (coming to the resource manager) and an output or reply message (going back to the client). The i member is a structure of type _io_stat that contains the following members:

If the message is a combine message, _IO_COMBINE_FLAG is set in this member. For more information, see the Combine Messages chapter of this guide.

The o member is a structure of type stat; for more information, see the entry for stat() in the QNX Neutrino Library Reference.

If you write your own handler, it should return the status via the helper macro _RESMGR_STATUS() and the struct stat via message reply.

If your resource manager is for a filesystem, you might want to include the stat information in the reply for other messages. For more information, see Returning directory entries from _IO_READ in the Filesystem Resource Managers chapter of this guide.

Handling lseek()

Your resource manager will receive an _IO_LSEEK message when a client calls lseek(), fseek(), or rewinddir().

Note: A resource manager that handles directories will also need to interpret the _IO_LSEEK message for directory operations.

The prototype for the io_lseek handler is as follows:

int io_lseek ( resmgr_context_t *ctp,
               io_lseek_t *msg,
               RESMGR_OCB_T *ocb)

The default handler, iofunc_lseek_default(), simply calls the iofunc_lseek() helper function.

The io_lseek_t structure is (once again), a union of an input message and an output message:

struct _io_lseek {
  uint16_t         type;
  uint16_t         combine_len;
  short            whence;
  uint16_t         zero;
  uint64_t         offset;

typedef union {
  struct _io_lseek i;
  uint64_t         o;
} io_lseek_t;

The whence and offset members are passed from the client's lseek() function. The routine should adjust the OCB's offset parameter after interpreting the whence and offset parameters from the message and should return the new offset or an error.

The handler should return the status via the helper macro _RESMGR_STATUS(), and optionally (if no error occurred, and if the message isn't part of a combine message) the current offset.