Hardware Reference
In-Depth Information
CPU
Memory
Cache
Root complex
Port 1
Port 2
Port 3
PCI bridge
PCIe device
PCIe device
Switch
PCI bus
PCIe device
PCIe device
PCIe device
PCIe device
Figure 2-32.
Sample architecture of a PCIe system with three PCIe ports.
Several things stand out immediately about PCIe. First, the connections be-
tween the devices are serial, that is, 1-bit wide rather than 8-, 16-, 32-, or 64-bits
wide. While one might think that a 64-bit-wide connection would have a higher
bandwidth than a 1-bit wide connection, in practice, differences in propagation
time of the 64 bits, called the
skew
, means relatively low speeds have to be used.
With a serial connection much higher speeds can be used and this more than offsets
the loss of parallelism. PCI buses run at a maximum clock rate of 66 MHz. With
64 bits transferred per cycle, the data rate is 528 MB/sec. With a clock rate of 8
Gbps, even with serial transfer, the data rate of PCIe is 1 GB/sec. Furthermore, de-
vices are not limited to a single wire pair to communicate with the root complex or
a switch. A device can have up to 32 wire pairs, called
lanes
. These lanes are not
synchronous, so skew is not important here. Most motherboards have a 16-lane
slot for the graphics card, which in PCIe 3.0 will give the graphics card a band-
width of 16 GB/sec, about 30 times faster than what a PCI graphics card can get.
This bandwidth is necessary for increasingly demanding applications, such as 3D.
Second, all communication is point to point. When the CPU wants to talk to a
device, it sends a packet to the device and generally later gets an answer. The
packet goes through the root complex, which is on the motherboard, and then on to
the device, possibly through a switch (or if the device is a PCI device, through the
PCI bridge). This evolution from a system in which all devices listened to the
same bus to one using point-to-point communications parallels the development of
Ethernet (a popular local area network), which also started with a broadcast chan-
nel but now uses switches to enable point-to-point communication.
