]>
Commit | Line | Data |
---|---|---|
e15b4d68 AB |
1 | The Linux RapidIO Subsystem |
2 | ||
3 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
4 | ||
5 | The RapidIO standard is a packet-based fabric interconnect standard designed for | |
6 | use in embedded systems. Development of the RapidIO standard is directed by the | |
7 | RapidIO Trade Association (RTA). The current version of the RapidIO specification | |
8 | is publicly available for download from the RTA web-site [1]. | |
9 | ||
10 | This document describes the basics of the Linux RapidIO subsystem and provides | |
11 | information on its major components. | |
12 | ||
13 | 1 Overview | |
14 | ---------- | |
15 | ||
16 | Because the RapidIO subsystem follows the Linux device model it is integrated | |
17 | into the kernel similarly to other buses by defining RapidIO-specific device and | |
18 | bus types and registering them within the device model. | |
19 | ||
20 | The Linux RapidIO subsystem is architecture independent and therefore defines | |
21 | architecture-specific interfaces that provide support for common RapidIO | |
22 | subsystem operations. | |
23 | ||
24 | 2. Core Components | |
25 | ------------------ | |
26 | ||
27 | A typical RapidIO network is a combination of endpoints and switches. | |
28 | Each of these components is represented in the subsystem by an associated data | |
29 | structure. The core logical components of the RapidIO subsystem are defined | |
30 | in include/linux/rio.h file. | |
31 | ||
32 | 2.1 Master Port | |
33 | ||
34 | A master port (or mport) is a RapidIO interface controller that is local to the | |
35 | processor executing the Linux code. A master port generates and receives RapidIO | |
36 | packets (transactions). In the RapidIO subsystem each master port is represented | |
37 | by a rio_mport data structure. This structure contains master port specific | |
38 | resources such as mailboxes and doorbells. The rio_mport also includes a unique | |
39 | host device ID that is valid when a master port is configured as an enumerating | |
40 | host. | |
41 | ||
42 | RapidIO master ports are serviced by subsystem specific mport device drivers | |
43 | that provide functionality defined for this subsystem. To provide a hardware | |
44 | independent interface for RapidIO subsystem operations, rio_mport structure | |
45 | includes rio_ops data structure which contains pointers to hardware specific | |
46 | implementations of RapidIO functions. | |
47 | ||
48 | 2.2 Device | |
49 | ||
50 | A RapidIO device is any endpoint (other than mport) or switch in the network. | |
51 | All devices are presented in the RapidIO subsystem by corresponding rio_dev data | |
52 | structure. Devices form one global device list and per-network device lists | |
53 | (depending on number of available mports and networks). | |
54 | ||
55 | 2.3 Switch | |
56 | ||
57 | A RapidIO switch is a special class of device that routes packets between its | |
58 | ports towards their final destination. The packet destination port within a | |
59 | switch is defined by an internal routing table. A switch is presented in the | |
60 | RapidIO subsystem by rio_dev data structure expanded by additional rio_switch | |
61 | data structure, which contains switch specific information such as copy of the | |
62 | routing table and pointers to switch specific functions. | |
63 | ||
64 | The RapidIO subsystem defines the format and initialization method for subsystem | |
65 | specific switch drivers that are designed to provide hardware-specific | |
66 | implementation of common switch management routines. | |
67 | ||
68 | 2.4 Network | |
69 | ||
70 | A RapidIO network is a combination of interconnected endpoint and switch devices. | |
71 | Each RapidIO network known to the system is represented by corresponding rio_net | |
72 | data structure. This structure includes lists of all devices and local master | |
73 | ports that form the same network. It also contains a pointer to the default | |
74 | master port that is used to communicate with devices within the network. | |
75 | ||
76 | 3. Subsystem Initialization | |
77 | --------------------------- | |
78 | ||
79 | In order to initialize the RapidIO subsystem, a platform must initialize and | |
80 | register at least one master port within the RapidIO network. To register mport | |
81 | within the subsystem controller driver initialization code calls function | |
82 | rio_register_mport() for each available master port. After all active master | |
83 | ports are registered with a RapidIO subsystem, the rio_init_mports() routine | |
84 | is called to perform enumeration and discovery. | |
85 | ||
86 | In the current PowerPC-based implementation a subsys_initcall() is specified to | |
87 | perform controller initialization and mport registration. At the end it directly | |
88 | calls rio_init_mports() to execute RapidIO enumeration and discovery. | |
89 | ||
90 | 4. Enumeration and Discovery | |
91 | ---------------------------- | |
92 | ||
93 | When rio_init_mports() is called it scans a list of registered master ports and | |
94 | calls an enumeration or discovery routine depending on the configured role of a | |
95 | master port: host or agent. | |
96 | ||
97 | Enumeration is performed by a master port if it is configured as a host port by | |
98 | assigning a host device ID greater than or equal to zero. A host device ID is | |
99 | assigned to a master port through the kernel command line parameter "riohdid=", | |
100 | or can be configured in a platform-specific manner. If the host device ID for | |
101 | a specific master port is set to -1, the discovery process will be performed | |
102 | for it. | |
103 | ||
104 | The enumeration and discovery routines use RapidIO maintenance transactions | |
105 | to access the configuration space of devices. | |
106 | ||
107 | The enumeration process is implemented according to the enumeration algorithm | |
108 | outlined in the RapidIO Interconnect Specification: Annex I [1]. | |
109 | ||
110 | The enumeration process traverses the network using a recursive depth-first | |
111 | algorithm. When a new device is found, the enumerator takes ownership of that | |
112 | device by writing into the Host Device ID Lock CSR. It does this to ensure that | |
113 | the enumerator has exclusive right to enumerate the device. If device ownership | |
114 | is successfully acquired, the enumerator allocates a new rio_dev structure and | |
115 | initializes it according to device capabilities. | |
116 | ||
117 | If the device is an endpoint, a unique device ID is assigned to it and its value | |
118 | is written into the device's Base Device ID CSR. | |
119 | ||
120 | If the device is a switch, the enumerator allocates an additional rio_switch | |
121 | structure to store switch specific information. Then the switch's vendor ID and | |
122 | device ID are queried against a table of known RapidIO switches. Each switch | |
123 | table entry contains a pointer to a switch-specific initialization routine that | |
124 | initializes pointers to the rest of switch specific operations, and performs | |
125 | hardware initialization if necessary. A RapidIO switch does not have a unique | |
126 | device ID; it relies on hopcount and routing for device ID of an attached | |
127 | endpoint if access to its configuration registers is required. If a switch (or | |
128 | chain of switches) does not have any endpoint (except enumerator) attached to | |
129 | it, a fake device ID will be assigned to configure a route to that switch. | |
130 | In the case of a chain of switches without endpoint, one fake device ID is used | |
131 | to configure a route through the entire chain and switches are differentiated by | |
132 | their hopcount value. | |
133 | ||
134 | For both endpoints and switches the enumerator writes a unique component tag | |
135 | into device's Component Tag CSR. That unique value is used by the error | |
136 | management notification mechanism to identify a device that is reporting an | |
137 | error management event. | |
138 | ||
139 | Enumeration beyond a switch is completed by iterating over each active egress | |
140 | port of that switch. For each active link, a route to a default device ID | |
141 | (0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written | |
142 | into the routing table. The algorithm recurs by calling itself with hopcount + 1 | |
143 | and the default device ID in order to access the device on the active port. | |
144 | ||
145 | After the host has completed enumeration of the entire network it releases | |
146 | devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint | |
088024b1 | 147 | in the system, it sets the Discovered bit in the Port General Control CSR |
e15b4d68 AB |
148 | to indicate that enumeration is completed and agents are allowed to execute |
149 | passive discovery of the network. | |
150 | ||
151 | The discovery process is performed by agents and is similar to the enumeration | |
152 | process that is described above. However, the discovery process is performed | |
153 | without changes to the existing routing because agents only gather information | |
154 | about RapidIO network structure and are building an internal map of discovered | |
155 | devices. This way each Linux-based component of the RapidIO subsystem has | |
156 | a complete view of the network. The discovery process can be performed | |
157 | simultaneously by several agents. After initializing its RapidIO master port | |
158 | each agent waits for enumeration completion by the host for the configured wait | |
159 | time period. If this wait time period expires before enumeration is completed, | |
160 | an agent skips RapidIO discovery and continues with remaining kernel | |
161 | initialization. | |
162 | ||
163 | 5. References | |
164 | ------------- | |
165 | ||
166 | [1] RapidIO Trade Association. RapidIO Interconnect Specifications. | |
167 | http://www.rapidio.org. | |
168 | [2] Rapidio TA. Technology Comparisons. | |
169 | http://www.rapidio.org/education/technology_comparisons/ | |
170 | [3] RapidIO support for Linux. | |
171 | http://lwn.net/Articles/139118/ | |
172 | [4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005 | |
173 | http://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf |