Reading a PCIe fabric as a graph

A PCI Express fabric is a graph: endpoints, switches, bridges, and a root complex are vertices, and links are edges. A packet reaches the root when at least one directed walk exists from its endpoint to the root complex, and each hop along that walk is trained, configured, and permitted by the routing rules in force. A loop appears when two or more vertices are connected by more than one distinct path and no rule breaks the cycle, which in a correctly configured tree topology does not occur.

How is a path chosen?

Path selection in PCI Express is not a single decision made at one point. It is the result of three layers that must agree.

First, enumeration. Configuration software walks the fabric from the root complex downward, assigning bus, device, and function numbers. Each switch records which of its downstream ports leads to which bus range. A packet addressed to a bus number is forwarded out the port whose recorded range contains that number.

Second, routing rules. Memory requests, I/O requests, and configuration requests each carry an address or a bus number. A switch compares the address against its base and limit registers. If the address falls inside a range assigned to a downstream port, the packet goes that way. If it falls outside every range, the switch may forward it upstream toward the root. This is why a path is not chosen by a global algorithm at runtime: it is chosen by the register values that enumeration wrote earlier.

Third, link training. A path is only usable if every link along it has completed training and is in L0. A link that is down, in recovery, or in a low power state does not carry the packet, and the switch treats that port as unavailable.

The practical consequence is that two endpoints on the same switch may reach the root by different numbers of hops, and the fabric does not attempt to balance them. The path is whatever the address ranges and the trained links allow. For teams documenting this kind of structure, a methodical approach to reading the underlying pages and indicators is described by the French magazine Live on web and SEO, which treats measurement and correction as separate steps.

What makes a path reachable?

A path is reachable when four conditions hold at the same time.

• The endpoint is enumerated. It has a bus, device, and function number, and the root complex has a record of it.
• Every link on the path is trained. Training establishes lane polarity, lane order, and link speed, and a link that has not reached L0 cannot carry a TLP.
• Every switch on the path has a routing entry that covers the destination address. A missing or stale base and limit register makes the path unreachable even if the link is up.
• No access control rule blocks the transaction. ACS, ATS, and similar mechanisms can restrict peer to peer traffic even when a physical path exists.

Reachability is therefore a property of the configuration state, not of the wiring alone. A cable can be present, a link can be trained, and the path can still be unreachable because a register was never written or was written with the wrong range.

This is also why a fabric diagram that shows only physical connections is incomplete. The diagram answers where a packet could go if everything were configured; the register state answers where it will go.

Where would a loop appear?

A loop would appear wherever two distinct paths connect the same pair of vertices and no rule prevents a packet from traversing both. In a tree, which is the intended PCI Express topology, there is exactly one path between any two vertices, so no loop exists.

A loop becomes possible when the physical topology is not a tree. Examples include two switches connected by more than one link, a bridge that forwards a packet back toward a port it arrived from, or a misconfigured switch whose upstream and downstream ranges overlap. In each case the graph contains a cycle.

PCI Express avoids persistent loops in two ways. Enumeration builds a spanning tree and assigns each switch a single upstream port, so the logical topology is a tree even if the physical cabling is not. And routing rules are directional: a packet that arrives on a downstream port is not normally forwarded back out of that same port. A loop in the logical graph would require a configuration error that breaks one of these two properties.

When a loop is suspected, the useful check is to trace the address ranges. If two switches both claim the same bus range, or if a switch claims a range that includes its own upstream bus, the logical graph has a cycle and packets can circulate.

Topology terms