Capability ID 0x11 (MSI-X)

The MSI-X capability module enables the use of Message Signalled Interrupts eXtenions (MSI-X). You should configure the MSI-X capability with the APIs listed below before enabling it. Specifically, you should check the number of device-supported interrupts, and then select the number that you intend to support in the driver. The number of driver-supported interrupts must be less than or equal to the number of device-supported interrupts.

You must enable the MSI-X capability (see pci_device_cfg_cap_enable()) before you call pci_device_read_irq(). When the capability is enabled, bit 2 (Bus Master) of the Command Register is also set. This bit isn't automatically cleared when the capability is disabled.

The MSI and MSI-X capabilities are mutually exclusive. If you want to switch to using MSI-X on a device that has MSI enabled, you must first disable the MSI capability (see pci_device_cfg_cap_disable()). Failure to do this results in an error of PCI_ERR_MSI_ENABLED.

cap_msix_pba_t *cap_msix_get_pba_ptr(pci_devhdl_t hdl, pci_cap_t cap)

pci_err_t cap_msix_mask_irq_entry(pci_devhdl_t hdl, pci_cap_t cap, uint_t irq_entry)

pci_err_t cap_msix_unmask_irq_entry(pci_devhdl_t hdl, pci_cap_t cap, uint_t irq_entry)

pci_err_t cap_msix_mask_irq_all(pci_devhdl_t hdl, pci_cap_t cap)

pci_err_t cap_msix_unmask_irq_all(pci_devhdl_t hdl, pci_cap_t cap)

The cap_msix_mask_irq_entry() and cap_msix_unmask_irq_entry() APIs allow driver software to mask and unmask each of the supported device interrupt sources. There's no direct relationship between a device's interrupt entry or the assigned MSI-X vector and the assigned IRQs returned from pci_device_read_irq(). InterruptMask() and InterruptUnmask() mask and unmask an IRQ, but cap_msix_mask_irq_entry() and cap_msix_unmask_irq_entry() (if supported) mask and unmask the interrupt sources associated with the MSI-X interrupt entry. Some or all of these entries may be configured to share the same IRQ. You can obtain the number of interrupt sources from cap_msix_get_nirq().

Interrupt disposition

By default, and unless unmodified by calling cap_msix_set_irq_entry(), each device interrupt source is assigned a unique MSI-X vector—notwithstanding the availability of MSI-X vectors when the capability is enabled—and hence a unique IRQ. Although a unique IRQ is assigned, this doesn't necessarily mean the IRQ isn't shared from a system perspective. This depends on the interrupt controller, and hence is platform-dependent.

This configuration is established when you initially read the capability with pci_device_read_cap(). If software modifies the disposition of one or more interrupt sources and then enables the capability, the disposition of each source will be as configured. If you subsequently disable the capability and then reenable it, the disposition is maintained. If you want to reset the disposition, you can either:

• set the disposition of each entry to itself

  or:

• disable the capability, free() the pci_cap_t, and then reread the capability; see pci_device_read_cap()

You can set the disposition of an interrupt source by specifying the irq_entry to be operated on, along with a disposition parameter:

• If the disposition is -1, the interrupt source irq_entry is marked as unused and doesn't have a vector assigned when you call pci_device_cfg_cap_enable().
• If disposition is greater than or equal to 0, but isn't equal to irq_entry, then irq_entry shares the same assigned MSI-X vector, and hence IRQ, as the entry identified by disposition.
• If disposition equals irq_entry, the interrupt source has its own MSI-X vector and IRQ assigned, and doesn't share with any other source.

Interrupt entries marked as unused (-1) are masked and can't be unmasked. Otherwise the mask state of the entry isn't altered, but the entry will have been masked when the capability was disabled in order to make disposition changes.

cap_msix_set_irq_entry(hdl, msix_cap, 2, 2);
cap_msix_set_irq_entry(hdl, msix_cap, 4, 2);

Attempts to do the reverse are rejected.

When you call pci_device_read_irq() to retrieve the list of assigned IRQs, the list is ordered to correspond with each of the n entries (starting from 0) that aren't marked as unused. For example, suppose that a device supports 256 MSI-X interrupts as identified by cap_msix_get_nirq(). Then suppose that only 64 interrupts were allocated, as identified by pci_device_read_irq(). This could be because of system limitations or configuration, or because the driver chose to use only this many by setting the disposition of various sources to shared. In this case, each of the 64 IRQs returned (0 through 63) correspond to MSI-X entries 0 through 63. If the number of requested and allocated IRQs is the same as the number of supported IRQs, this 1:1 relationship is exactly as you'd expect.

unused --> entry 0
irq[0] --> IRQ for entry 1
irq[1..3] --> IRQ for entries 2, 3, 4 respectively
unused --> entries 5 and 6
irq[4..9] --> IRQ for entries 7, 8, 9, 10, 11, 12 respectively
irq[10] --> IRQ for entries 13/14 (shared)
irq[11..17] --> IRQ for entries 15, 16, 17, 18, 19, 20, 21 respectively
irq[18] --> IRQ for entries 22/23 (shared)
irq[19..63] --> IRQ for entries 24 through 68 respectively

The cap_msix_get_pba_ptr() function returns a read-only pointer to the Pending Bits Array. The PBA allows driver software to query any pending interrupts. If an error occurred, the function returns NULL. The PBA is organized with bit 0 corresponding to interrupt source/entry 0, and bit n corresponding to the number of supported MSI-X interrupts the device supports minus 1.

When you disable the MSI-X capability, all interrupt sources are automatically masked. You must disable the capability before making any changes to the interrupt disposition.