Banning the Bomb:
A Nuclear Weapon Watchdog

A. David McKinnon, Charles W. Hubbard, Harry S. Miley Pacific Northwest National Laboratory

Since discussions of a comprehensive nuclear test ban first began back in the '50s, several treaties have been implemented by the international community to restrict nuclear weapons testing in space, in the atmosphere, and under water. But none have gone so far as to ban all nuclear weapons tests. That is, until the UN's General Assembly voted overwhelmingly last September to adopt the Comprehensive Nuclear Test Ban Treaty (CTBT).

Signing the treaty was the easy part, compared with the task of ensuring that all nations comply with the ban. Making sure the CTBT is adopted globally hinges ultimately on the ability to detect violations. To this end, the nations signing the treaty are implementing the International Monitoring System (IMS), a network of data centers and various verification technologies - seismic, infrasound, hydroacoustic, and so on - to search for and analyze clues that indicate illegal nuclear testing.

One such verification technology is the Radionuclide Aerosol Sampler/Analyzer (RASA), a QNX-based technology we've developed at the US Department of Energy's Pacific Northwest National Laboratory in Richland, Washington. RASA is a system that detects airborne radionuclides - the radioactive species of elements like barium, molybdenum, and so on - that are released into the atmosphere by a nuclear explosion. What's so important about radionuclide detection technology is that unlike seismic, infrasound, or hydroacoustic detection technologies, it provides irrefutable physical evidence of a treaty violation because it captures a small part of the weapon itself.

RASA requirements

In designing a radionuclide detection device for IMS, we had to keep several factors firmly in the forefront. First, the measurement system had to be capable of detecting very small amounts of radioactive material. Depending on weather conditions at the time of a blast, radionuclides may travel halfway around the globe and be days old by the time they reach a measurement station. By this time, elements with a short half-life will have disappeared, leaving only a small fraction of the original radioactive material.

Second, we had to think a lot about the overall cost of the system. Our plans called for 80 measurement stations worldwide to better the chance that a radioactive cloud would be detected within the critical first weeks after detonation. So, the system had to be affordable for all countries, including developing nations.

Third, and most importantly, because some of the detectors will be placed in remote locations, we had to develop a completely automated system that could run for many months at a time without restocking.

We managed to roll all of these requirements, and more, into RASA. Sensitive enough to detect a nuclear explosion even after the cloud of radioactive debris has traveled completely around the globe, RASA can run continuously for 200 days without restocking and has complete remote access and control. RASA is also affordable, quiet enough not to disturb nearby residential areas or wilderness habitats, and consumes minimal power. It can even tolerate power failures!

Resembling a small furnace in size and shape (3' x 6' x 6'), the radionuclide sampler houses a Ziatech 8902 ruggedized 486 running QNX 4 and the physical measurement system: a set of continuous filter paper strips, a segmented sample head, and a high-resolution germanium gamma-ray detector. The paper strips are advanced mechanically into the sample head through which large volumes of air are filtered. After a decay period, the strips are placed in front of the gamma-ray detector. To monitor RASA's state-of-health and log onsite conditions, the system also includes secondary sensors for temperature, voltage, air velocity, and other environmental conditions.

Robust and stable

One of our first tasks was to select an OS for the project. Back in December 1992 we reviewed several operating systems and chose QNX because it offered a robust realtime control system and a reliable method of communicating data and commands to and from the various data centers. A stable operating system was also a crucial requirement for RASA because we knew an unacceptable mean time between failure (MTBF) and mean time to repair (MTTR) could make or break the international acceptance of our system. We knew QNX had a good track record in critical systems.

QNX had other selling features too, including a simple IPC scheme, multitasking capability, TCP/IP connectivity, and POSIX-compliance. Although we had no prior experience with preemptive multitasking or a message-passing architecture, we quickly learned the value of these features as the project progressed.

A modular approach

When we started development, the CTBT wasn't yet signed so technical and political guidelines were constantly being revised; by necessity, we approached the project with modularity in mind. Starting with an object-oriented approach to the control software, we wrote all RASA system control software in C++ and made extensive use of C++ object classes throughout. For instance, all servers are derived from a common-base-server object that gives each server an identical set of core abilities (e.g. ping, shutdown) and a consistent message-passing interface. This object-oriented approach also greatly increased our ability to maintain the system and reuse our code.

Hardware interface

This modular approach let us separate the control software from the hardware-specific server processes. Access to the hardware interface is provided by the digital and analog I/O server processes - the only processes that have direct access to the underlying hardware. All other processes access the hardware through requests sent to the server processes.

Consider the analog server. The analog server presents a sequential set of analog channels to the client, each of which can report from 0 to 10V. Internally, the corresponding channels don't have to be sequential or even exist on the same analog card. In fact, it's possible to use multiple cards from different manufacturers simultaneously: each card's particular set of input channels can be mapped into the sequential, logical channel space presented by the server. A similar physical-to-logical channel mapping is performed by the digital I/O server.

As we worked through several RASA prototypes this technique made it possible to change the underlying hardware without affecting any of the clients requiring digital or analog services. Also, by allowing only a single process to access the hardware directly, we avoided conflicts between multiple servers attempting simultaneous hardware access. The operating system automatically queues server requests, which are then handled sequentially by the server.

As we moved to an automated field-ready system, abstracting the hardware allowed the software team to keep up with rapid hardware changes and upgrades. Without the hardware abstraction, hardware changes would have required changes in the control modules. As it was, only the hardware-specific server processes had to be changed, which saved us a lot of time.

Interface hiding is a recurring theme in RASA: all of the subsystem-control servers hide the details of their underlying interfaces. QNX native message passing provided a means to support data abstraction such that interface hiding could occur at the process level within the control system. And finally, the preemptive multitasking nature of QNX allowed us to develop efficient task-specific processes. Integrating these processes into a robust, unified, control structure was easy thanks to QNX message passing.

Client/server architecture

Each RASA subsystem or group of related components (e.g. the temperature sensors) is governed by a separate server process. The power-system monitor, air-volume accumulator, sensor monitor, meteorological-data monitor, nuclear data-acquisition system, and motor controls all communicate with the hardware either through the digital/ analog servers or through the RS-232 interface provided by QNX.

These servers present client interfaces that relate to the device being controlled or the quantity being measured, as opposed to the actual hardware signals that facilitate the controls and measurements. For instance, the sensor server replies to messages of the form "What temperature is the intake air?" as opposed to "What voltage is analog channel 4 reporting?" The sensor server responds to this sort of message by obtaining the appropriate sensor voltage from the analog server and converting it to temperature units before passing the answer back to the original requester.

This added layer of abstraction frees the client from having to deal with precise conversion details. It isn't even necessary for the client to know that the particular temperature sensor being queried returns an analog voltage at all. In fact, during early development of RASA, several sensor technologies were investigated, including temperature sensors that reported via digital signals, analog voltages, and even RS-232 data streams. In its current form RASA uses only analog temperature sensors, but any combination of the above technologies can be substituted without making any change to the software above the level of the sensor server.

Control etc.

Operation of RASA is governed by the supervisory control process patterned after the state machine. To help implement the state machine design in software, we developed a simple, powerful, library that allows users to quickly implement a state machine skeleton with only a few lines of C++ code. States and transitions can easily be added or removed as changes are made to the system's control algorithm.

X-based GUI

An easy-to-use GUI is a baseline requirement for most applications these days, and RASA was no different. Keeping in mind that this system would eventually be used worldwide, we settled on X for the project mainly because X servers run on a wide variety of computing platforms.

We used the X Windows development toolkit for QNX along with Tcl/Tk, a high-level scripting language that considerably shortened our development time. Tcl/Tk allowed us to write in just a few lines what would have required pages of X Windows code. The language can also be easily extended to create new commands. For example, we developed RASA-specific Tcl/Tk commands that passed messages to the various RASA server processes.

Communicating with RASA

Because we took a modular approach to the communications subsystem as well, RASA can be configured to use any one of several different communications methods. The Internet provides the fastest communications path to the system. However, since many locations throughout the world don't have Internet access, we designed RASA to also use standard land-based phone lines and modems, or cellular- or satellite-based modems and phone lines - whatever's most convenient.

Any combination of sensor technologies can be substituted with minimal software changes.

We used an extension to Tcl/Tk called Expect to develop the data transmission subsystems.

The key strength of Expect is that it can be used to automate any task or series of tasks a user performs from the command line. Our Expect scripts automatically start a Point-to-Point Protocol (PPP) session with an Internet service provider. We could have accomplished this with a chat script, but with Expect's superior error checking this can be done during the PPP session.

We developed another set of Expect scripts to automatically transfer the accumulated RASA data to a centralized data center. These scripts transfer one or more files at a time via FTP. When a communications failure occurs (e.g. a disconnected phone line during a PPP session), the data is automatically retransmitted at a later time. The data-transfer communications subsystem has proven itself very well during field tests - it worked without a problem, despite many disruptions of the phone service.

Field trials

A series of RASA system designs have been built and tested since the project began. RASA Mark 3, the third prototype and the first system to be completely automated, participated in an extensive field trial at McClelland Air Force Base in July 1995. The field trial was intended to uncover components and subsystems unsuitable for remote installations - and it did. During the course of the six-week-long test, a number of temperature sensors, optical sensors, power supplies, voltage sensors, and assorted other components failed. But QNX never failed to allow us access to the system - to query the status of individual servers, and diagnose the day's data - even when the control software had crashed!

Based on our initial success with RASA, QNX has been selected for two more data acquisition and control systems.

We were also able to implement many software upgrades remotely. For instance, at one point we had problems with the logic on the optical sensors - which monitor advancement of the filter paper. A bank of three sensors was used in a majority logic mode to guard against possible failure. During testing, the status logger revealed one continuously noisy sensor, while a second sensor only briefly saw the printed barcode on the paper as it advanced. This situation would cause the filter paper to occasionally come up short. We were able to replace the faulty majority logic code with code requiring unanimous logic, and solved the problem without a visit to the site.

These field trials are really just the beginning of the road for RASA. The system will face even bigger challenges when fielded in such diverse climates as Antarctica, Hawaii, Guam, and Alaska. But it's comforting to know that if there are problems, they can be diagnosed and solved from thousands of miles away.

In the works

Based on our initial success with RASA, QNX has been selected for two more data acquisition and control systems in support of the CTBT. The first, a system to measure radioactive xenon leaking from underground nuclear test sites, is a much more complicated measurement device than RASA: it will have a hundred valves, a dozen heaters, many temperature sensors, several mass flow controllers, and a cryogenic cooler. The xenon measurement system will also sport a Photon-based front end.

The second project, dubbed R-TARAC (Real-Time Airborne Radioactive Analyzer and Collector), is a sensor designed to be installed in an aircraft and flown into potential sources of radioactive effluent such as reactor accidents, volcanoes, and so on.

What's next for RASA?

Although RASA has already been chosen to take part in the IMS, more advanced software development is still in store for the system in the coming months. While past development centered on obtaining a working, robust system, RASA will soon get many sophisticated new features. Specifically, we'll be adding methods to:

• track the performance of a large network
• catch problems before failures occur
• schedule routine maintenance and restocking
• authenticate data out of, and commands into, the station

A Photon-based GUI is also planned for RASA. The system was originally developed with X, but now that Photon is an option, we plan to replace our existing resource-intensive GUI. Other proposed software developments will include the use of network news protocols (NNTP) for data transmission and simple network management protocols (SNMP) for transmitting state-of-health queries and alarm notices. The use of NNTP and SNMP will allow a data center to use existing Internet-based software to support future RASA systems and other verification technologies in the field.

The modular nature of both QNX and our control system will definitely help us accomplish these goals without having to scrap any of our existing software.

RASA Mark 4 has been selected by the US National Data Center for US CTBT verification, and will become commercially available for worldwide CTBT verification and other environmental monitoring uses later this year.