]>
Commit | Line | Data |
---|---|---|
684ffa2d MCC |
1 | Media Controller devices |
2 | ------------------------ | |
3 | ||
4 | Media Controller | |
5 | ~~~~~~~~~~~~~~~~ | |
6 | ||
74604b73 | 7 | The media controller userspace API is documented in |
da83c888 | 8 | :ref:`the Media Controller uAPI book <media_controller>`. This document focus |
684ffa2d MCC |
9 | on the kernel-side implementation of the media framework. |
10 | ||
11 | Abstract media device model | |
12 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
13 | ||
14 | Discovering a device internal topology, and configuring it at runtime, is one | |
15 | of the goals of the media framework. To achieve this, hardware devices are | |
16 | modelled as an oriented graph of building blocks called entities connected | |
17 | through pads. | |
18 | ||
19 | An entity is a basic media hardware building block. It can correspond to | |
20 | a large variety of logical blocks such as physical hardware devices | |
21 | (CMOS sensor for instance), logical hardware devices (a building block | |
22 | in a System-on-Chip image processing pipeline), DMA channels or physical | |
23 | connectors. | |
24 | ||
25 | A pad is a connection endpoint through which an entity can interact with | |
26 | other entities. Data (not restricted to video) produced by an entity | |
27 | flows from the entity's output to one or more entity inputs. Pads should | |
28 | not be confused with physical pins at chip boundaries. | |
29 | ||
30 | A link is a point-to-point oriented connection between two pads, either | |
31 | on the same entity or on different entities. Data flows from a source | |
32 | pad to a sink pad. | |
33 | ||
34 | Media device | |
35 | ^^^^^^^^^^^^ | |
36 | ||
74604b73 MCC |
37 | A media device is represented by a :c:type:`struct media_device <media_device>` |
38 | instance, defined in ``include/media/media-device.h``. | |
39 | Allocation of the structure is handled by the media device driver, usually by | |
40 | embedding the :c:type:`media_device` instance in a larger driver-specific | |
41 | structure. | |
684ffa2d MCC |
42 | |
43 | Drivers register media device instances by calling | |
74604b73 MCC |
44 | :cpp:func:`__media_device_register()` via the macro ``media_device_register()`` |
45 | and unregistered by calling :cpp:func:`media_device_unregister()`. | |
684ffa2d MCC |
46 | |
47 | Entities | |
48 | ^^^^^^^^ | |
49 | ||
74604b73 MCC |
50 | Entities are represented by a :c:type:`struct media_entity <media_entity>` |
51 | instance, defined in ``include/media/media-entity.h``. The structure is usually | |
52 | embedded into a higher-level structure, such as | |
53 | :ref:`v4l2_subdev` or :ref:`video_device` | |
54 | instances, although drivers can allocate entities directly. | |
684ffa2d MCC |
55 | |
56 | Drivers initialize entity pads by calling | |
74604b73 | 57 | :cpp:func:`media_entity_pads_init()`. |
684ffa2d MCC |
58 | |
59 | Drivers register entities with a media device by calling | |
74604b73 | 60 | :cpp:func:`media_device_register_entity()` |
684ffa2d | 61 | and unregistred by calling |
74604b73 | 62 | :cpp:func:`media_device_unregister_entity()`. |
684ffa2d MCC |
63 | |
64 | Interfaces | |
65 | ^^^^^^^^^^ | |
66 | ||
74604b73 MCC |
67 | Interfaces are represented by a |
68 | :c:type:`struct media_interface <media_interface>` instance, defined in | |
69 | ``include/media/media-entity.h``. Currently, only one type of interface is | |
70 | defined: a device node. Such interfaces are represented by a | |
71 | :c:type:`struct media_intf_devnode <media_intf_devnode>`. | |
684ffa2d MCC |
72 | |
73 | Drivers initialize and create device node interfaces by calling | |
74604b73 | 74 | :cpp:func:`media_devnode_create()` |
684ffa2d | 75 | and remove them by calling: |
74604b73 | 76 | :cpp:func:`media_devnode_remove()`. |
684ffa2d MCC |
77 | |
78 | Pads | |
79 | ^^^^ | |
74604b73 MCC |
80 | Pads are represented by a :c:type:`struct media_pad <media_pad>` instance, |
81 | defined in ``include/media/media-entity.h``. Each entity stores its pads in | |
82 | a pads array managed by the entity driver. Drivers usually embed the array in | |
83 | a driver-specific structure. | |
684ffa2d MCC |
84 | |
85 | Pads are identified by their entity and their 0-based index in the pads | |
86 | array. | |
74604b73 MCC |
87 | |
88 | Both information are stored in the :c:type:`struct media_pad`, making the | |
89 | :c:type:`media_pad` pointer the canonical way to store and pass link references. | |
684ffa2d MCC |
90 | |
91 | Pads have flags that describe the pad capabilities and state. | |
92 | ||
74604b73 MCC |
93 | ``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data. |
94 | ``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data. | |
95 | ||
96 | .. note:: | |
684ffa2d | 97 | |
74604b73 MCC |
98 | One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must |
99 | be set for each pad. | |
684ffa2d MCC |
100 | |
101 | Links | |
102 | ^^^^^ | |
103 | ||
74604b73 MCC |
104 | Links are represented by a :c:type:`struct media_link <media_link>` instance, |
105 | defined in ``include/media/media-entity.h``. There are two types of links: | |
684ffa2d | 106 | |
74604b73 | 107 | **1. pad to pad links**: |
684ffa2d MCC |
108 | |
109 | Associate two entities via their PADs. Each entity has a list that points | |
110 | to all links originating at or targeting any of its pads. | |
111 | A given link is thus stored twice, once in the source entity and once in | |
112 | the target entity. | |
113 | ||
114 | Drivers create pad to pad links by calling: | |
74604b73 MCC |
115 | :cpp:func:`media_create_pad_link()` and remove with |
116 | :cpp:func:`media_entity_remove_links()`. | |
684ffa2d | 117 | |
74604b73 | 118 | **2. interface to entity links**: |
684ffa2d MCC |
119 | |
120 | Associate one interface to a Link. | |
121 | ||
122 | Drivers create interface to entity links by calling: | |
74604b73 MCC |
123 | :cpp:func:`media_create_intf_link()` and remove with |
124 | :cpp:func:`media_remove_intf_links()`. | |
684ffa2d MCC |
125 | |
126 | .. note:: | |
127 | ||
128 | Links can only be created after having both ends already created. | |
129 | ||
130 | Links have flags that describe the link capabilities and state. The | |
74604b73 MCC |
131 | valid values are described at :cpp:func:`media_create_pad_link()` and |
132 | :cpp:func:`media_create_intf_link()`. | |
684ffa2d MCC |
133 | |
134 | Graph traversal | |
135 | ^^^^^^^^^^^^^^^ | |
136 | ||
137 | The media framework provides APIs to iterate over entities in a graph. | |
138 | ||
139 | To iterate over all entities belonging to a media device, drivers can use | |
140 | the media_device_for_each_entity macro, defined in | |
74604b73 MCC |
141 | ``include/media/media-device.h``. |
142 | ||
143 | .. code-block:: c | |
684ffa2d | 144 | |
74604b73 | 145 | struct media_entity *entity; |
684ffa2d | 146 | |
74604b73 MCC |
147 | media_device_for_each_entity(entity, mdev) { |
148 | // entity will point to each entity in turn | |
149 | ... | |
150 | } | |
684ffa2d MCC |
151 | |
152 | Drivers might also need to iterate over all entities in a graph that can be | |
153 | reached only through enabled links starting at a given entity. The media | |
154 | framework provides a depth-first graph traversal API for that purpose. | |
155 | ||
74604b73 MCC |
156 | .. note:: |
157 | ||
158 | Graphs with cycles (whether directed or undirected) are **NOT** | |
159 | supported by the graph traversal API. To prevent infinite loops, the graph | |
160 | traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``, | |
161 | currently defined as 16. | |
684ffa2d MCC |
162 | |
163 | Drivers initiate a graph traversal by calling | |
74604b73 | 164 | :cpp:func:`media_entity_graph_walk_start()` |
684ffa2d MCC |
165 | |
166 | The graph structure, provided by the caller, is initialized to start graph | |
167 | traversal at the given entity. | |
168 | ||
169 | Drivers can then retrieve the next entity by calling | |
74604b73 | 170 | :cpp:func:`media_entity_graph_walk_next()` |
684ffa2d | 171 | |
74604b73 | 172 | When the graph traversal is complete the function will return ``NULL``. |
684ffa2d MCC |
173 | |
174 | Graph traversal can be interrupted at any moment. No cleanup function call | |
175 | is required and the graph structure can be freed normally. | |
176 | ||
177 | Helper functions can be used to find a link between two given pads, or a pad | |
178 | connected to another pad through an enabled link | |
74604b73 MCC |
179 | :cpp:func:`media_entity_find_link()` and |
180 | :cpp:func:`media_entity_remote_pad()`. | |
684ffa2d MCC |
181 | |
182 | Use count and power handling | |
183 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
184 | ||
185 | Due to the wide differences between drivers regarding power management | |
186 | needs, the media controller does not implement power management. However, | |
74604b73 MCC |
187 | the :c:type:`struct media_entity <media_entity>` includes a ``use_count`` |
188 | field that media drivers | |
684ffa2d MCC |
189 | can use to track the number of users of every entity for power management |
190 | needs. | |
191 | ||
74604b73 MCC |
192 | The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by |
193 | media drivers and must not be | |
684ffa2d | 194 | touched by entity drivers. Access to the field must be protected by the |
74604b73 | 195 | :c:type:`media_device`.\ ``graph_mutex`` lock. |
684ffa2d MCC |
196 | |
197 | Links setup | |
198 | ^^^^^^^^^^^ | |
199 | ||
200 | Link properties can be modified at runtime by calling | |
74604b73 | 201 | :cpp:func:`media_entity_setup_link()`. |
684ffa2d MCC |
202 | |
203 | Pipelines and media streams | |
204 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
205 | ||
206 | When starting streaming, drivers must notify all entities in the pipeline to | |
207 | prevent link states from being modified during streaming by calling | |
74604b73 | 208 | :cpp:func:`media_entity_pipeline_start()`. |
684ffa2d MCC |
209 | |
210 | The function will mark all entities connected to the given entity through | |
211 | enabled links, either directly or indirectly, as streaming. | |
212 | ||
74604b73 MCC |
213 | The :c:type:`struct media_pipeline <media_pipeline>` instance pointed to by |
214 | the pipe argument will be stored in every entity in the pipeline. | |
215 | Drivers should embed the :c:type:`struct media_pipeline <media_pipeline>` | |
216 | in higher-level pipeline structures and can then access the | |
217 | pipeline through the :c:type:`struct media_entity <media_entity>` | |
218 | pipe field. | |
684ffa2d | 219 | |
74604b73 MCC |
220 | Calls to :cpp:func:`media_entity_pipeline_start()` can be nested. |
221 | The pipeline pointer must be identical for all nested calls to the function. | |
684ffa2d | 222 | |
74604b73 MCC |
223 | :cpp:func:`media_entity_pipeline_start()` may return an error. In that case, |
224 | it will clean up any of the changes it did by itself. | |
684ffa2d MCC |
225 | |
226 | When stopping the stream, drivers must notify the entities with | |
74604b73 | 227 | :cpp:func:`media_entity_pipeline_stop()`. |
684ffa2d | 228 | |
74604b73 MCC |
229 | If multiple calls to :cpp:func:`media_entity_pipeline_start()` have been |
230 | made the same number of :cpp:func:`media_entity_pipeline_stop()` calls | |
231 | are required to stop streaming. | |
232 | The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last | |
233 | nested stop call. | |
684ffa2d | 234 | |
74604b73 | 235 | Link configuration will fail with ``-EBUSY`` by default if either end of the |
684ffa2d | 236 | link is a streaming entity. Links that can be modified while streaming must |
74604b73 | 237 | be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag. |
684ffa2d MCC |
238 | |
239 | If other operations need to be disallowed on streaming entities (such as | |
240 | changing entities configuration parameters) drivers can explicitly check the | |
241 | media_entity stream_count field to find out if an entity is streaming. This | |
242 | operation must be done with the media_device graph_mutex held. | |
243 | ||
244 | Link validation | |
245 | ^^^^^^^^^^^^^^^ | |
246 | ||
74604b73 MCC |
247 | Link validation is performed by :cpp:func:`media_entity_pipeline_start()` |
248 | for any entity which has sink pads in the pipeline. The | |
249 | :c:type:`media_entity`.\ ``link_validate()`` callback is used for that | |
250 | purpose. In ``link_validate()`` callback, entity driver should check | |
251 | that the properties of the source pad of the connected entity and its own | |
252 | sink pad match. It is up to the type of the entity (and in the end, the | |
253 | properties of the hardware) what matching actually means. | |
684ffa2d MCC |
254 | |
255 | Subsystems should facilitate link validation by providing subsystem specific | |
256 | helper functions to provide easy access for commonly needed information, and | |
257 | in the end provide a way to use driver-specific callbacks. | |
258 | ||
259 | .. kernel-doc:: include/media/media-device.h | |
260 | ||
261 | .. kernel-doc:: include/media/media-devnode.h | |
262 | ||
263 | .. kernel-doc:: include/media/media-entity.h |