Locking, unlocking, and combine message handling

Updated: April 19, 2023

We saw the client side of a combine message when we looked at readblock() (in “Combine messages”). The client was able to atomically construct a message that contained multiple resource manager “submessages” corresponding to the functions lseek() and read(). From the client's perspective, the two (or more) functions were at least sent atomically (and, due to the nature of message passing, will be received atomically by the resource manager). What we haven't yet talked about is how we ensure that the messages are processed atomically.

This discussion applies not only to combine messages, but to all I/O messages received by the resource manager library (except the close message, which we'll come back to shortly).

Note: The very first thing that the resource manager library does is to lock the attribute structure corresponding to the resource being used by the received message. Then, it processes one or more submessages from the incoming message. Finally, it unlocks the attribute structure.

This ensures that the incoming messages are handled atomically, for no other thread in the resource manager (in the case of a multithreaded resource manager, of course) can “jump in” and modify the resource while a thread is busy using it. Without the locking in place, two client threads could both issue what they believe to be an atomic combine message (say lseek() and read()). Since the resource manager might have two different threads running in it and processing messages, the two resource manager threads could possibly preempt each other, and the lseek() components could interfere with each other. With locking and unlocking, this is prevented, because each message that accesses a resource will be completed in its entirety atomically.

Locking and unlocking the resource is handled by default helper functions (iofunc_lock_ocb_default() and iofunc_unlock_ocb_default()), which are placed in the I/O table at the lock_ocb and unlock_ocb positions. You can, of course, override these functions if you want to perform further actions during this locking and unlocking phase.

Note that the resource is unlocked before the close OCB I/O function handler is called. This is necessary because this particular function handler frees the OCB, which would effectively invalidate the pointer used to access the attributes structure, which is where the lock is stored! Also note that none of the connect functions do this locking, because the handle that's passed to them does not have to be an attribute structure (and the locks are stored in the attribute structure).