Shutdown

In io-net, there was a pretty straightforward, two-stage shutdown. Shutting down in io-pkt is a bit more complicated than that.

First off, note the speedo_stop() function previously mentioned. It can be externally called by io-pkt, or internally by the driver itself. This is necessary, but not sufficient.

In addition, the driver needs to register another callback function via the shutdownhook_establish() io-pkt function, which it calls in the speedo_attach() function. The callback function it registers is speedo_shutdown(), which simply calls speedo_stop() with the correct parameters.

The shutdownhook mechanism is special — it's called from the SIGSEGV handler of io-pkt to quiesce hardware. If this isn't done, hardware may continue via DMA to write to memory that the system thinks is freed. This is very bad — DMA must be halted, and hardware interrupts should be masked when the driver's shutdownhook() function is called.

It's worth mentioning that the driver callback function that the driver registers with io-pkt via the shutdownhook_establish() io-pkt function should do as little as possible, and ideally just hit some hardware registers to primarily halt DMA and hopefully mask the hardware interrupt — especially if the interrupt is shared!

The problem is that if the driver does unnecessary work in the shutdown callback, it risks faulting again. If it does, then other hardware driver threads may not get a chance to quiesce their hardware, which can be bad.

Note that the speedo_stop() function, which is certainly generic, isn't particularly well-coded in this regard; to avoid memory leaks, it scans through transmissions which are in progress, and calls m_free() to release mbufs. But if the SIGSEGV occurred because of mbuf heap corruption, then a second SIGSEGV could easily occur during the subsequent m_free() calls. As you can see, shutdown isn't trivial to get correct for all circumstances.