I've been using QNX operating systems since 1982 when I was in my late teens, and started developing commercial applications under QNX a few years later. During those years, I've seen QNX's user base grow from just dozens of individuals to the large following it has today.
In the early days, people used QNX because they had a keen appreciation for the simplicity and elegance that the QNX operating system offered. It was, I believe, this simplicity and elegance that allowed those individuals to excel in their work. In those days you'd hear stories about how a team of programmers working with a competing OS were unable to complete projects after months of struggling, where a much smaller team of programmers who knew QNX finished the project in a much shorter time. And the end result was smaller and faster.
 |
For clarity, there are several QNX operating system versions. The original operating system, known simply as “QUNIX” was followed by “QUNIX 2”. Both of these ran on the early x86 processors. The next version was QNX 4 (note the name change from “QUNIX”). Finally, the latest version is known as QNX Neutrino or (incorrectly) as QNX 6. In this book, we refer to this latest version, and call it simply “Neutrino”. |
Over the years, as I've worked with QNX, I've tried to adhere to the “simpler is better” philosophy. This is summed up most elegantly by J. Gall in his 1986 book, Systemantics:
A complex system that works is invariably found to have evolved from a simple system that worked.... A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.
This is the same idea behind object-oriented design, and the fundamental “design philosophy” of UNIX — lots of small tools with well-defined tasks, joined together to make a complex system.
Confirming the points made in all kinds of software design books, it's been my experience that trying to design a complex working system from scratch just doesn't work. As Gall says, you have to start with a simple system. And, as Frederick P. Brooks, Jr. says in the original version of The Mythical Man-Month, you should build a prototype, and be prepared to throw it away. Unfortunately, even some of today's management doesn't quite understand this basic philosophy. I personally think that The Mythical Man-Month should be required reading — by managers and software developers alike.
In this book I want to share my experience with the QNX Neutrino operating system, and show you lots of simple systems that work. These examples were developed over the years; some of them are even in commercial use today. The companies that I wrote them for on contract have graciously let me publish the source code (please see the “Thanks!” section, below). A few of the utilities were developed especially for this book — they were projects that I ported from QNX 4 to Neutrino, or were written from scratch to illustrate interesting features of the OS.
What you'll find here are, for all intents and purposes, recipes — small, well-defined pieces of code, or utilities that do something on their own, and can be used as part of a larger system. I will explain step-by-step what the code does, why it was written the way it was (for example, what were the constraints? what was I thinking? :-)), how the code can be expanded, and so on.
This book is divided into these sections:
In this first section of the book, I discuss the various “big picture” and architectural issues that you need to keep in mind. This is a condensation (some would say “brain dump”) of the work I've done with QNX operating systems over the years — I've seen what works, and what doesn't. I'll share both, because I think it's important to know what doesn't work too, so you can understand why it didn't work out.
In this chapter, the main() function and its friend, the option processing function optproc(), are discussed. It's in its own chapter so we don't have to talk about the things I use in almost every project.
High availability is a very interesting topic. In this chapter, I discuss the concept of high availability (what it is, how it's measured, and how to achieve it). We'll discuss such things as Mean Time Between Failures (MTBF), Mean Time To Repair (MTTR), and the formula used to calculate availability.
I'll also talk about how you can design your systems to be “highly-available” and some of the problems that you'll run into. Unfortunately, in a lot of today's designs, high availability is done as an afterthought — this almost always leads to disaster.
By thinking about high availability up front, you'll be able to benefit from having the architectural insight necessary to design highly-available systems.
Next, I present an article about the basic philosophy that's useful when building a system based on message passing — the fundamental Inter Process Communications (IPC) model used by all QNX operating systems. We'll take a hypothetical security system example, and examine how the design is derived to meet various goals. We'll look at the design of the individual pieces (things like swipe card readers and door lock actuators) and see how they fit into a larger system. We'll also discuss things like scalability — how to make your software design scale from a small security system to a large, multi-site security system.
The second section of the book contains the “meat” — a smattering of small, useful, and well-documented utilities. Each has been tested on a live Neutrino system. Where there are deficiencies, they're noted — software is an ever-evolving creation. I've also tried to pick utilities that demonstrate something interesting that's “special” about Neutrino, and perhaps not well-understood.
You won't find a graphics ray-trace utility here, because that's not Neutrino-specific (even though it may be poorly understood by the general population of programmers). What you will find are examples of resource managers, high-availability, IPC, and other topics.
Each of the headings below is one chapter in the “recipes” section.
This chapter describes a utility that illustrates how to generate graphical images on-the-fly within a resource manager. You've seen web counters on various web pages; the count goes up every time someone accesses the web page. In this chapter, I'll show you how this can be done with a neat twist — the web counter looks and acts just like a plain, ordinary file. The “magic” is all done via a resource manager. You'll see how to maintain context on a per-open and per-device basis, how to handle the file content generation, and so on. The chapter presents the project in three phases — a kind of “building-block” approach.
This project is a data acquisition server, written for Century Aluminum in Kentucky. There are two major parts to this project: card drivers (for the PCL-711, ISO-813, and DIO-144 analog/digital I/O cards), and a master server that collects data from the cards and puts it into shared memory.
Several other utilities are discussed as well, such as showsamp, which gets the data from shared memory. This chapter is a good insight on how to handle I/O, as well as shared-memory management.
Many people want to write filesystems, or things that look like filesystems, for Neutrino. The easiest filesystem to understand is a RAM-disk, because we don't need to deal with the “on-media” format — all of our data is stored in RAM, and the data itself is simply allocated from the pool of available memory. Reading, writing, seeking, block management, pathname parsing, directory management, etc. are discussed. This is an extensive chapter that serves as a foundation for the tar Filesystem Manager chapter (immediately following) and also serves as a good basis for any projects you may wish to pursue that need a filesystem (or a filesystem-like) interface.
This chapter presents another filesystem, one that manages .tar files. It builds on the ideas and content of the RAM-disk chapter (above) and shows how to manage an indexed file — a virtual file that is hosted as a portion of a real, disk-based file.
Additional topics at the end give you some ideas of other types of filesystems that can be constructed.
Finally, the last section of the book contains appendixes with useful reference material, as well as some additional general topics.
This appendix provides additional information about the basics of filesystems, not only how files, directories, and symlinks are organized, but also how they map to the resource manager OCB and attributes structures. Fundamentally, all filesystems are a mapping between some physical (or abstract) hierarchical data representation onto the native resource manager structures.
The /proc filesystem is where QNX Neutrino stores all of the information about processes and threads — how much CPU time they've used, how much memory they have allocated, how many threads are running, what state they are in, etc. This chapter serves as a handy reference for the /proc filesystem, and shows you what information is available, where it is, and how to get it.
This appendix tells you where to get an archive of the programs discussed in this book, and describes the contents and structure of the archive.
Finally, you'll find a glossary that defines the terms used in this book.
The general rule when writing is, “Write what you know about.” As a result of this rule, you won't find anything in this book about TCP/IP, the Integrated Development Environment (I use make and vi), the Graphical User Interface (GUI, called “Photon” under Neutrino), and so on.
As I've programmed over the years, I've found the following books and references to be quite useful and enlightening:
I'd like to extend a gracious “Thank You” to many people who helped out with this book.
First, the QNX Software Systems folks. John Garvey's support when I was writing the RAM-disk section was invaluable. A lot of the behavior of the functions is obscure, and John patiently answered all my questions in the conferences, especially about symlinks. Brian Stecher reviewed copies of the book, provided the foreword, and supplied many details of the /proc filesystem that I wouldn't have been able to fathom on my own. Peter van der Veen also reviewed the book, and supplied insightful comments on the /proc filesystem and resource managers. Dan Dodge, David Gibbs, Adam Mallory, Peter Martin, Kirk Russell, and Xiaodan Tang reviewed the book and pointed out key omissions and clarified many of the issues involved. Kirk also helped out with the “horror” of mkisofs :-)
Other reviewers included David Alessio, Kirk A. Bailey, Bill Caroselli, Mario Charest, Rick Duff, and Alexander Koppel (who took me to task on several topics, thank you!). Thanks for taking the time to read through the book and for your comments; this book is better because of your efforts!
Once again, (for the third time now), Chris Herborth was tricked into editing this book. Thanks for putting up with my writing style and my demands on your time, Chris!
Finally, John Ostrander reviewed the final cut of the book, and once again provided his outstanding and patient copy-editing finesse.
My wife put up with my crawling off to the basement (again) but at least this time she was busy with school :-) Thanks Christine!
I worked for Century Aluminum on contract during the spring/summer of 2003. They were converting their aluminum smelter lines to use Neutrino, and needed a few device drivers written to manage data acquisition. Out of this was born the ADIOS chapter — my sincere thanks to the people involved with the project for letting me use the source code I developed.