An example of synchronization

Let's look at an example of synchronization in a multithreaded program.

If the data is the most important part of your application, and that data gets corrupted, then your application might be of little use. It's pretty straightforward to make sure a single-threaded application doesn't accidentally corrupt its data. Multithreaded applications are quite another story.

Without synchronizing

The problem with multithreaded applications is that you could have one section of code modifying your data, while another section is reading and using that same data. If the write and read overlap, you could have some serious complications!

Let's illustrate this with a short application:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
 
typedef struct { 
   int a;
   int b;
   int result;
   int result2;
   int use_count;
   int use_count2;
   int max_use;
   int max_use2;
} app_data;

void *user_thread(void *data)
{ 
   int uses = 0;
   app_data *td = (app_data*)data;

   while(uses < td->max_use) { 
      if (td->a == 5) { 
         td->result += (td->a + td->b);
         td->use_count++;
         uses++;
      } 
      usleep(1);
   } 
   return 0;
} 

void *changer_thread(void *data)
{ 
   app_data *td = (app_data*)data;

   while ((td->use_count + td->use_count2) < (td->max_use + td->max_use2)) { 
      if (td->a == 5) { 
         td->a = 50;
         td->b = td->a + usleep(1000);
      } else { 
         td->a = 5;
         td->b = td->a + usleep(1000);
      } 
      usleep(1);
   } 
   return 0;
} 

int main(int argc, char **argv)
{ 
   pthread_t ct, ut;
   app_data td = {5,5,0,0,0,0,100,0};
   void *retval;

   pthread_create(&ut, NULL, user_thread, &td);
   pthread_create(&ct, NULL, changer_thread, &td);
   pthread_join(ct, &retval);
   pthread_join(ut, &retval);
   printf("result should be %d; it's %d\n", td.max_use * (5 + 5), td.result);

   return EXIT_SUCCESS;
} 

The user_thread() function reads the data in the passed app_data structure and uses the a and b variables max_use times whenever a is set to five. Its local uses variable counts the number of times the function has used the data. The function also increments the application's use_count, in case someone else wants to keep track of how many times the data has been used. This function adds a small usleep() to make sure the scheduler gives other threads a chance to run.

The changer_thread() function modifies the data, changing a and b until they've been used elsewhere—in user_thread(), in this case—the number of times specified by max_use plus max_use2. The change is simple: changer_thread() simply toggles the value of a between 5 and 50 and fakes a CPU-intensive calculation for b by calling usleep(1000), which returns 0 unless an error occurs. This means that a will be changed, and 1 ms later b will be changed. Therefore there's a 1 ms gap where a and b shouldn't be used elsewhere in the application since their values are still in flux.

The main() routine creates an instance of the app_data structure, spawns the two threads, waits for them to finish, and then prints the results. If you build the program and run it, the results are something like this:

result should be 1000; it's 3205

Your actual result might be different, depending on your CPU speed. Why isn't it what we expected?

The problem is that user_thread() is waiting until a is set to 5, and when it is, it adds the a and b values (which should both be 5) to the result variable. However, since the b variable is taking so long to get calculated, user_thread() sometimes uses the old b value (50) instead of the new one (5). You need to protect your data somehow so that the user_thread() can't use the data until you've set it!

Using a mutex

Let's use a mutex to make sure that the data is accessed by only one thread at a time.

Mutexes are very easy to use. Once you've created a mutex, you simply put a pthread_mutex_lock() in front of the code you want to protect, and pthread_mutex_unlock() right after that code. In this case, you want to protect both the modification and use of the a and b variables.

Here's a summary of the changes to make to the program:

Here's the program, with the updates shown in bold:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
 
typedef struct { 
   int a;
   int b;
   int result;
   int result2;
   int use_count;
   int use_count2;
   int max_use;
   int max_use2;
   pthread_mutex_t mutex;
} app_data;

void *user_thread(void *data)
{ 
   int uses = 0;
   app_data *td = (app_data*)data;

   while(uses < td->max_use) { 
      pthread_mutex_lock(&td->mutex);
      if (td->a == 5) { 
         td->result += (td->a + td->b);
         td->use_count++;
         uses++;
      } 
      pthread_mutex_unlock(&td->mutex);
      usleep(1);
   } 
   return 0;
} 

void *changer_thread(void *data)
{ 
   app_data *td = (app_data*)data;

   while ((td->use_count + td->use_count2) < (td->max_use + td->max_use2)) { 
      pthread_mutex_lock(&td->mutex);
      if (td->a == 5) { 
         td->a = 50;
         td->b = td->a + usleep(1000);
      } else { 
         td->a = 5;
         td->b = td->a + usleep(1000);
      } 
      pthread_mutex_unlock(&td->mutex);
      usleep(1);
   } 
   return 0;
} 

int main(int argc, char **argv)
{ 
   pthread_t ct, ut;
   app_data td = {5,5,0,0,0,0,100,0};
   void *retval;

   pthread_mutex_init(&td.mutex, NULL);
   pthread_create(&ut, NULL, user_thread, &td);
   pthread_create(&ct, NULL, changer_thread, &td);
   pthread_join(ct, &retval);
   pthread_join(ut, &retval);
   pthread_mutex_destroy(&td.mutex);
   printf("result should be %d; it's %d\n", td.max_use * (5 + 5), td.result);

   return EXIT_SUCCESS;
} 

This program should give the results we want:

result should be 1000; it's 1000

More than one reader

What if you have one thread that's changing the data, but more than one other thread that just wants to read it? Let's add a thread that needs to read a and b as well:

void *subtracter_thread(void *data)
{ 
   int use=0;
   app_data *td=(app_data*)data;

   while(use < td->max_use2) { 
      pthread_mutex_lock(&td->mutex);
      if (td->a == 50) { 
         td->result2 -= (td->a + td->b);
         use++;
         td->use_count2++;
      } 
      pthread_mutex_unlock(&td->mutex);
      usleep(1);
   } 
   return 0;
} 

We need to modify main() to spawn the new thread and set the max_use2 member in the app_data structure:

int main(int argc, char **argv)
{ 
   pthread_t ct, ut, st;
   app_data td = {5,5,0,0,0,0,100,100};
   void *retval;

   pthread_mutex_init(&td.mutex, NULL);
   pthread_create(&ut, NULL, user_thread, &td);
   pthread_create(&ct, NULL, changer_thread, &td);
   pthread_create(&st, NULL, subtracter_thread, &td);
   pthread_join(ct, &retval);
   pthread_join(ut, &retval);
   pthread_join(st, &retval);
   pthread_mutex_destroy(&td.mutex);
   printf("result should be %d; it's %d\n", td.max_use * (5 + 5), td.result);
   printf("result2 should be %d; it's %d\n", -(td.max_use2 * (50 + 50)),
          td.result2);

   return EXIT_SUCCESS;
} 

If you compile and run this, you get just what you wanted:

result should be 1000; it's 1000
result2 should be -10000; it's -10000

But wait: shouldn't user_thread() and subtracter_thread() be able to read a and b at the same time? (Remember that with mutexes, anything wrapped in the lock/unlock pair can't be executed at the same time as another piece of code wrapped in the lock/unlock pair.) Neither of them changes a or b, so it would be nice if you could let both of them read these members at the same time—just not when changer_thread() is changing them.

Using read-write locks

With readers/writer locks (or rwlocks), you can easily implement this kind of behavior. Unlike mutexes, rwlocks can be locked either as read or as write. As long as they're not locked for write access, any threads can lock for read and unlock as much as they want. However, if they're locked for write access, all read locks occurring afterwards are blocked until the write lock is unlocked.

Here's how to make our program use an rwlock:

Here's the updated code, with the changes in bold:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
 
typedef struct { 
   int a;
   int b;
   int result;
   int result2;
   int use_count;
   int use_count2;
   int max_use;
   int max_use2;
   pthread_rwlock_t rwl;
} app_data;

void *user_thread(void *data)
{ 
   int uses = 0;
   app_data *td = (app_data*)data;

   while(uses < td->max_use) { 
      pthread_rwlock_rdlock(&td->rwl);
      if (td->a == 5) { 
         td->result += (td->a + td->b);
         td->use_count++;
         uses++;
      } 
      pthread_rwlock_unlock(&td->rwl);
      usleep(1);
   } 
   return 0;
} 

void *changer_thread(void *data)
{ 
   app_data *td = (app_data*)data;

   while ((td->use_count + td->use_count2) < (td->max_use + td->max_use2)) { 
      pthread_rwlock_wrlock(&td->rwl);
      if (td->a == 5) { 
         td->a = 50;
         td->b = td->a + usleep(1000);
      } else { 
         td->a = 5;
         td->b = td->a + usleep(1000);
      } 
      pthread_rwlock_unlock(&td->rwl);
      usleep(1);
   } 
   return 0;
} 

void *subtracter_thread(void *data)
{ 
   int use=0;
   app_data *td=(app_data*)data;

   while(use < td->max_use2) { 
      pthread_rwlock_rdlock(&td->rwl);
      if (td->a == 50) { 
         td->result2 -= (td->a + td->b);
         use++;
         td->use_count2++;
      } 
      pthread_rwlock_unlock(&td->rwl);
      usleep(1);
   } 
   return 0;
} 

int main(int argc, char **argv)
{ 
   pthread_t ct, ut, st;
   app_data td = {5,5,0,0,0,0,100,100};
   void *retval;

   pthread_rwlock_init(&td.rwl, NULL);
   pthread_create(&ut, NULL, user_thread, &td);
   pthread_create(&ct, NULL, changer_thread, &td);
   pthread_create(&st, NULL, subtracter_thread, &td);
   pthread_join(ct, &retval);
   pthread_join(ut, &retval);
   pthread_join(st, &retval);
   pthread_rwlock_destroy(&td.rwl);
   printf("result should be %d; it's %d\n", td.max_use * (5 + 5), td.result);
   printf("result2 should be %d; it's %d\n", -(td.max_use2 * (50 + 50)),
          td.result2);

   return EXIT_SUCCESS;
} 

Now subtracter_thread() and user_thread() can read the data at the same time, but changer_thread() can't access the data at the same time as either subtracter_thread() or user_thread().