Leaving the client blocked, replying later

An example of a resource manager that would reply to clients later is a pipe resource manager. If the client is doing a read of your pipe but you have no data for the client, then you have a choice:

• You can reply with an error (EAGAIN).

  Or:

• You can leave the client blocked and later, when your write handler function is called, you can reply to the client with the new data.

Another example might be if the client wants you to write to some device but doesn't want to get a reply until the data has been fully written out. Here are the sequence of events that might follow:

1. Your resource manager does some I/O to the hardware to tell it that data is available.
2. The hardware generates an interrupt when it's ready for a packet of data.
3. You handle the interrupt by writing data out to the hardware.
4. Many interrupts may occur before all the data is written — only then would you reply to the client.

The first issue, though, is whether the client wants to be left blocked. If the client doesn't want to be left blocked, then it opens with the O_NONBLOCK flag:

fd = open("/dev/sample", O_RDWR | O_NONBLOCK);

The default is to allow you to block it.

One of the first things done in the read and write samples above was to call some POSIX verification functions: iofunc_read_verify() and iofunc_write_verify(). If we pass the address of an int as the last parameter, then on return the functions will stuff that int with nonzero if the client doesn't want to be blocked (O_NONBLOCK flag was set) or with zero if the client wants to be blocked.

int    nonblock;                                                     
                                                                     
if ((status = iofunc_read_verify (ctp, msg, ocb,
                                  &nonblock)) != EOK) 
    return (status);                                                 
                                                                     
...                                                                  
                                                                     
int    nonblock;                                                     
                                                                     
if ((status = iofunc_write_verify (ctp, msg, ocb,
                                   &nonblock)) != EOK)
    return (status);

When it then comes time to decide if we should reply with an error or reply later, we do:

if (nonblock) {
    /* client doesn't want to be blocked */
    return (EAGAIN);                                          
} else {                                                      
    /*
    *  The client is willing to be blocked.
    *  Save at least the ctp->rcvid so that you can
    *  reply to it later.
    */
    ...
    return (_RESMGR_NOREPLY);
}                                                             

The question remains: How do you do the reply yourself? The only detail to be aware of is that the rcvid to reply to is ctp->rcvid. If you're replying later, then you'd save ctp->rcvid and use the saved value in your reply:

MsgReply(saved_rcvid, 0, buffer, nbytes);

Or:

iov_t    iov[2];

SETIOV(&iov[0], &header, sizeof(header));
SETIOV(&iov[1], &buffers[i], sizeof(buffers[i]));
MsgReplyv(saved_rcvid, 0, iov, 2);

Note that you can fill up the client's reply buffer as data becomes available by using resmgr_msgwrite() and resmgr_msgwritev(). Just remember to do the MsgReply*() at some time to unblock the client.

There's another way to resume the blocked operation, but it isn't as efficient as the other methods: you can call resmgr_msg_again(). This function restores the resmgr_context_t structure to the way it was when your resource manager received the message associated with the rcvid, and then processes the message again as if it had just been received.