The following terminology can be used to describe the hierarchy of PCI
devices.
Hierarchy
The tree structured PCI Express I/O interconnect topology.
Hierarchy Domain
The part of a hierarchy originating from a single root port.
Note:
PCI Server terminology is slightly different and maps to the PCI Express (PCIe)
terminology as described below.
PCI Domain
Domain containing an entire PCIe defined Hierarchy.
From a PCI perspective a
domain consists of a Compute Complex which itself consists of one or more
processing elements e.g., CPU, caches, MMUs, etc. A PCI subsystem consisting
of Root Complexes, Root Ports, etc. is directly accessible to the processing
elements. Communication between domains is possible with external mechanisms
including, but not limited to, Non-Transparent Bridging (NTB). It is
possible for more than one PCI Domain to exist within a system and the
ability to run multiple PCI server instances in a separate, independent
processing domain currently exists.Figure 1Communication between different domains in a system
PCI Segment
Segment identical to a PCI Express Specification defined Hierarchy Domain.
It
describes how the PCI hierarchy within a domain is divided into multiple
sub-hierarchies. A segment must originate at a single root port. However, a
separate segment can exist for root complex integrated endpoints.
PCI Partitioning
PCI Partitioning is the process of grouping segments of a hierarchy together
within a domain.
The PCI Routing Identifiers or BDF space used to represent each of the
components within a PCI system is global across all segments associated with a
partition. However, it is separate between the segments associated with
different partitions. Each partition provides for a separate BDF space.
A PCI
Domain can consist of a single partition with all segments managed by a
single pci-server instance, a single partition with
segments managed by multiple pci-server instances or
multiple partitions each with their segments managed by one or more
pci-server instances.
Figure 2 shows the
operating environment and relationship between various PCI resources. It
shows the PCI components in a PCI subsystem with four separate root ports
within a single compute domain.Figure 2Operating environment and relationship between various PCI
resources Although there is no explicit partition, all four segments are grouped
together and managed by a single pci-server instance
which maintains information about the domain hierarchy in two files (i.e.,
/dev/shmem/pci/pci_db and
/dev/shmem/pci/pci_hw). The BDF space for this
hierarchy is global and consists of the following specification:
B[255..0]:D[31..0]:F[7..0]
Along with this, the PCI subsystem enables explicit partitioning. You can
either choose a descriptive name for the partition or "pci" will be used as
the default name.
Figure 3 shows the same representative system as in
Figure 2 but with explicit partitioning. Figure 3Operating environment with explicit partitioning
In this example, although there is an explicit partition with
the default name "pci", all 4 segments are still grouped together and
managed by a single pci-server instance which maintains
information about the domain hierarchy in two files, this time with the
partition name included in their location in the namespace,
dev/shmem/pci/pci_db and /dev/shmem/pci/pci_hw/.
The BDF space for this hierarchy is also global and consists of the
same specification:
B[255..0]:D[31..0]:F[7..0]
Note:
The namespace node registered by the
pci-server takes the partition into account in this
example. This is because the partition name or server identifier name is not
specified. The defaults names are "pci" and "server_id_1".
