]>
Commit | Line | Data |
---|---|---|
e1302912 SL |
1 | i.MX Video Capture Driver |
2 | ========================= | |
3 | ||
4 | Introduction | |
5 | ------------ | |
6 | ||
7 | The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which | |
8 | handles the flow of image frames to and from capture devices and | |
9 | display devices. | |
10 | ||
11 | For image capture, the IPU contains the following internal subunits: | |
12 | ||
13 | - Image DMA Controller (IDMAC) | |
14 | - Camera Serial Interface (CSI) | |
15 | - Image Converter (IC) | |
16 | - Sensor Multi-FIFO Controller (SMFC) | |
17 | - Image Rotator (IRT) | |
18 | - Video De-Interlacing or Combining Block (VDIC) | |
19 | ||
20 | The IDMAC is the DMA controller for transfer of image frames to and from | |
21 | memory. Various dedicated DMA channels exist for both video capture and | |
22 | display paths. During transfer, the IDMAC is also capable of vertical | |
23 | image flip, 8x8 block transfer (see IRT description), pixel component | |
24 | re-ordering (for example UYVY to YUYV) within the same colorspace, and | |
25 | even packed <--> planar conversion. It can also perform a simple | |
26 | de-interlacing by interleaving even and odd lines during transfer | |
27 | (without motion compensation which requires the VDIC). | |
28 | ||
29 | The CSI is the backend capture unit that interfaces directly with | |
30 | camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses. | |
31 | ||
32 | The IC handles color-space conversion, resizing (downscaling and | |
33 | upscaling), horizontal flip, and 90/270 degree rotation operations. | |
34 | ||
35 | There are three independent "tasks" within the IC that can carry out | |
36 | conversions concurrently: pre-process encoding, pre-process viewfinder, | |
37 | and post-processing. Within each task, conversions are split into three | |
38 | sections: downsizing section, main section (upsizing, flip, colorspace | |
39 | conversion, and graphics plane combining), and rotation section. | |
40 | ||
41 | The IPU time-shares the IC task operations. The time-slice granularity | |
42 | is one burst of eight pixels in the downsizing section, one image line | |
43 | in the main processing section, one image frame in the rotation section. | |
44 | ||
45 | The SMFC is composed of four independent FIFOs that each can transfer | |
46 | captured frames from sensors directly to memory concurrently via four | |
47 | IDMAC channels. | |
48 | ||
49 | The IRT carries out 90 and 270 degree image rotation operations. The | |
50 | rotation operation is carried out on 8x8 pixel blocks at a time. This | |
51 | operation is supported by the IDMAC which handles the 8x8 block transfer | |
52 | along with block reordering, in coordination with vertical flip. | |
53 | ||
54 | The VDIC handles the conversion of interlaced video to progressive, with | |
55 | support for different motion compensation modes (low, medium, and high | |
56 | motion). The deinterlaced output frames from the VDIC can be sent to the | |
57 | IC pre-process viewfinder task for further conversions. The VDIC also | |
58 | contains a Combiner that combines two image planes, with alpha blending | |
59 | and color keying. | |
60 | ||
61 | In addition to the IPU internal subunits, there are also two units | |
62 | outside the IPU that are also involved in video capture on i.MX: | |
63 | ||
64 | - MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus | |
65 | interface. This is a Synopsys DesignWare core. | |
66 | - Two video multiplexers for selecting among multiple sensor inputs | |
67 | to send to a CSI. | |
68 | ||
69 | For more info, refer to the latest versions of the i.MX5/6 reference | |
70 | manuals [#f1]_ and [#f2]_. | |
71 | ||
72 | ||
73 | Features | |
74 | -------- | |
75 | ||
76 | Some of the features of this driver include: | |
77 | ||
78 | - Many different pipelines can be configured via media controller API, | |
79 | that correspond to the hardware video capture pipelines supported in | |
80 | the i.MX. | |
81 | ||
82 | - Supports parallel, BT.565, and MIPI CSI-2 interfaces. | |
83 | ||
84 | - Concurrent independent streams, by configuring pipelines to multiple | |
85 | video capture interfaces using independent entities. | |
86 | ||
87 | - Scaling, color-space conversion, horizontal and vertical flip, and | |
88 | image rotation via IC task subdevs. | |
89 | ||
90 | - Many pixel formats supported (RGB, packed and planar YUV, partial | |
91 | planar YUV). | |
92 | ||
93 | - The VDIC subdev supports motion compensated de-interlacing, with three | |
94 | motion compensation modes: low, medium, and high motion. Pipelines are | |
95 | defined that allow sending frames to the VDIC subdev directly from the | |
96 | CSI. There is also support in the future for sending frames to the | |
97 | VDIC from memory buffers via a output/mem2mem devices. | |
98 | ||
99 | - Includes a Frame Interval Monitor (FIM) that can correct vertical sync | |
100 | problems with the ADV718x video decoders. | |
101 | ||
102 | ||
103 | Entities | |
104 | -------- | |
105 | ||
106 | imx6-mipi-csi2 | |
107 | -------------- | |
108 | ||
109 | This is the MIPI CSI-2 receiver entity. It has one sink pad to receive | |
110 | the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has | |
111 | four source pads, corresponding to the four MIPI CSI-2 demuxed virtual | |
112 | channel outputs. Multpiple source pads can be enabled to independently | |
113 | stream from multiple virtual channels. | |
114 | ||
115 | This entity actually consists of two sub-blocks. One is the MIPI CSI-2 | |
116 | core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block | |
117 | is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the | |
118 | four virtual channels streams, providing four separate parallel buses | |
119 | containing each virtual channel that are routed to CSIs or video | |
120 | multiplexers as described below. | |
121 | ||
122 | On i.MX6 solo/dual-lite, all four virtual channel buses are routed to | |
123 | two video multiplexers. Both CSI0 and CSI1 can receive any virtual | |
124 | channel, as selected by the video multiplexers. | |
125 | ||
126 | On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected | |
127 | by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1 | |
128 | and IPU2-CSI0, respectively, and virtual channel 3 is routed to | |
129 | IPU2-CSI1 (again selected by a video mux). | |
130 | ||
131 | ipuX_csiY_mux | |
132 | ------------- | |
133 | ||
134 | These are the video multiplexers. They have two or more sink pads to | |
135 | select from either camera sensors with a parallel interface, or from | |
136 | MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a | |
137 | single source pad that routes to a CSI (ipuX_csiY entities). | |
138 | ||
139 | On i.MX6 solo/dual-lite, there are two video mux entities. One sits | |
140 | in front of IPU1-CSI0 to select between a parallel sensor and any of | |
141 | the four MIPI CSI-2 virtual channels (a total of five sink pads). The | |
142 | other mux sits in front of IPU1-CSI1, and again has five sink pads to | |
143 | select between a parallel sensor and any of the four MIPI CSI-2 virtual | |
144 | channels. | |
145 | ||
146 | On i.MX6 Quad, there are two video mux entities. One sits in front of | |
147 | IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual | |
148 | channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to | |
149 | select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two | |
150 | sink pads). | |
151 | ||
152 | ipuX_csiY | |
153 | --------- | |
154 | ||
155 | These are the CSI entities. They have a single sink pad receiving from | |
156 | either a video mux or from a MIPI CSI-2 virtual channel as described | |
157 | above. | |
158 | ||
159 | This entity has two source pads. The first source pad can link directly | |
160 | to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links | |
161 | that require no IDMAC memory buffer transfer. | |
162 | ||
163 | When the direct source pad is routed to the ipuX_ic_prp entity, frames | |
164 | from the CSI can be processed by one or both of the IC pre-processing | |
165 | tasks. | |
166 | ||
167 | When the direct source pad is routed to the ipuX_vdic entity, the VDIC | |
168 | will carry out motion-compensated de-interlace using "high motion" mode | |
169 | (see description of ipuX_vdic entity). | |
170 | ||
171 | The second source pad sends video frames directly to memory buffers | |
172 | via the SMFC and an IDMAC channel, bypassing IC pre-processing. This | |
173 | source pad is routed to a capture device node, with a node name of the | |
174 | format "ipuX_csiY capture". | |
175 | ||
176 | Note that since the IDMAC source pad makes use of an IDMAC channel, it | |
177 | can do pixel reordering within the same colorspace. For example, the | |
178 | sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. | |
179 | If the sink pad is receiving YUV, the output at the capture device can | |
180 | also be converted to a planar YUV format such as YUV420. | |
181 | ||
182 | It will also perform simple de-interlace without motion compensation, | |
183 | which is activated if the sink pad's field type is an interlaced type, | |
184 | and the IDMAC source pad field type is set to none. | |
185 | ||
186 | This subdev can generate the following event when enabling the second | |
187 | IDMAC source pad: | |
188 | ||
189 | - V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR | |
190 | ||
191 | The user application can subscribe to this event from the ipuX_csiY | |
192 | subdev node. This event is generated by the Frame Interval Monitor | |
193 | (see below for more on the FIM). | |
194 | ||
195 | Cropping in ipuX_csiY | |
196 | --------------------- | |
197 | ||
198 | The CSI supports cropping the incoming raw sensor frames. This is | |
199 | implemented in the ipuX_csiY entities at the sink pad, using the | |
200 | crop selection subdev API. | |
201 | ||
202 | The CSI also supports fixed divide-by-two downscaling indepently in | |
203 | width and height. This is implemented in the ipuX_csiY entities at | |
204 | the sink pad, using the compose selection subdev API. | |
205 | ||
206 | The output rectangle at the ipuX_csiY source pad is the same as | |
207 | the compose rectangle at the sink pad. So the source pad rectangle | |
208 | cannot be negotiated, it must be set using the compose selection | |
209 | API at sink pad (if /2 downscale is desired, otherwise source pad | |
210 | rectangle is equal to incoming rectangle). | |
211 | ||
212 | To give an example of crop and /2 downscale, this will crop a | |
213 | 1280x960 input frame to 640x480, and then /2 downscale in both | |
214 | dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux): | |
215 | ||
216 | media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]" | |
217 | media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]" | |
218 | media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]" | |
219 | ||
220 | Frame Skipping in ipuX_csiY | |
221 | --------------------------- | |
222 | ||
223 | The CSI supports frame rate decimation, via frame skipping. Frame | |
224 | rate decimation is specified by setting the frame intervals at | |
225 | sink and source pads. The ipuX_csiY entity then applies the best | |
226 | frame skip setting to the CSI to achieve the desired frame rate | |
227 | at the source pad. | |
228 | ||
229 | The following example reduces an assumed incoming 60 Hz frame | |
230 | rate by half at the IDMAC output source pad: | |
231 | ||
232 | media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]" | |
233 | media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]" | |
234 | ||
235 | Frame Interval Monitor in ipuX_csiY | |
236 | ----------------------------------- | |
237 | ||
238 | The adv718x decoders can occasionally send corrupt fields during | |
239 | NTSC/PAL signal re-sync (too little or too many video lines). When | |
240 | this happens, the IPU triggers a mechanism to re-establish vertical | |
241 | sync by adding 1 dummy line every frame, which causes a rolling effect | |
242 | from image to image, and can last a long time before a stable image is | |
243 | recovered. Or sometimes the mechanism doesn't work at all, causing a | |
244 | permanent split image (one frame contains lines from two consecutive | |
245 | captured images). | |
246 | ||
247 | From experiment it was found that during image rolling, the frame | |
248 | intervals (elapsed time between two EOF's) drop below the nominal | |
249 | value for the current standard, by about one frame time (60 usec), | |
250 | and remain at that value until rolling stops. | |
251 | ||
252 | While the reason for this observation isn't known (the IPU dummy | |
253 | line mechanism should show an increase in the intervals by 1 line | |
254 | time every frame, not a fixed value), we can use it to detect the | |
255 | corrupt fields using a frame interval monitor. If the FIM detects a | |
256 | bad frame interval, the ipuX_csiY subdev will send the event | |
257 | V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR. Userland can register with | |
258 | the FIM event notification on the ipuX_csiY subdev device node. | |
259 | Userland can issue a streaming restart when this event is received | |
260 | to correct the rolling/split image. | |
261 | ||
262 | The ipuX_csiY subdev includes custom controls to tweak some dials for | |
263 | FIM. If one of these controls is changed during streaming, the FIM will | |
264 | be reset and will continue at the new settings. | |
265 | ||
266 | - V4L2_CID_IMX_FIM_ENABLE | |
267 | ||
268 | Enable/disable the FIM. | |
269 | ||
270 | - V4L2_CID_IMX_FIM_NUM | |
271 | ||
272 | How many frame interval measurements to average before comparing against | |
273 | the nominal frame interval reported by the sensor. This can reduce noise | |
274 | caused by interrupt latency. | |
275 | ||
276 | - V4L2_CID_IMX_FIM_TOLERANCE_MIN | |
277 | ||
278 | If the averaged intervals fall outside nominal by this amount, in | |
279 | microseconds, the V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR event is sent. | |
280 | ||
281 | - V4L2_CID_IMX_FIM_TOLERANCE_MAX | |
282 | ||
283 | If any intervals are higher than this value, those samples are | |
284 | discarded and do not enter into the average. This can be used to | |
285 | discard really high interval errors that might be due to interrupt | |
286 | latency from high system load. | |
287 | ||
288 | - V4L2_CID_IMX_FIM_NUM_SKIP | |
289 | ||
290 | How many frames to skip after a FIM reset or stream restart before | |
291 | FIM begins to average intervals. | |
292 | ||
293 | - V4L2_CID_IMX_FIM_ICAP_CHANNEL | |
294 | - V4L2_CID_IMX_FIM_ICAP_EDGE | |
295 | ||
296 | These controls will configure an input capture channel as the method | |
297 | for measuring frame intervals. This is superior to the default method | |
298 | of measuring frame intervals via EOF interrupt, since it is not subject | |
299 | to uncertainty errors introduced by interrupt latency. | |
300 | ||
301 | Input capture requires hardware support. A VSYNC signal must be routed | |
302 | to one of the i.MX6 input capture channel pads. | |
303 | ||
304 | V4L2_CID_IMX_FIM_ICAP_CHANNEL configures which i.MX6 input capture | |
305 | channel to use. This must be 0 or 1. | |
306 | ||
307 | V4L2_CID_IMX_FIM_ICAP_EDGE configures which signal edge will trigger | |
308 | input capture events. By default the input capture method is disabled | |
309 | with a value of IRQ_TYPE_NONE. Set this control to IRQ_TYPE_EDGE_RISING, | |
310 | IRQ_TYPE_EDGE_FALLING, or IRQ_TYPE_EDGE_BOTH to enable input capture, | |
311 | triggered on the given signal edge(s). | |
312 | ||
313 | When input capture is disabled, frame intervals will be measured via | |
314 | EOF interrupt. | |
315 | ||
316 | ||
317 | ipuX_vdic | |
318 | --------- | |
319 | ||
320 | The VDIC carries out motion compensated de-interlacing, with three | |
321 | motion compensation modes: low, medium, and high motion. The mode is | |
322 | specified with the menu control V4L2_CID_DEINTERLACING_MODE. It has | |
323 | two sink pads and a single source pad. | |
324 | ||
325 | The direct sink pad receives from an ipuX_csiY direct pad. With this | |
326 | link the VDIC can only operate in high motion mode. | |
327 | ||
328 | When the IDMAC sink pad is activated, it receives from an output | |
329 | or mem2mem device node. With this pipeline, it can also operate | |
330 | in low and medium modes, because these modes require receiving | |
331 | frames from memory buffers. Note that an output or mem2mem device | |
332 | is not implemented yet, so this sink pad currently has no links. | |
333 | ||
334 | The source pad routes to the IC pre-processing entity ipuX_ic_prp. | |
335 | ||
336 | ipuX_ic_prp | |
337 | ----------- | |
338 | ||
339 | This is the IC pre-processing entity. It acts as a router, routing | |
340 | data from its sink pad to one or both of its source pads. | |
341 | ||
342 | It has a single sink pad. The sink pad can receive from the ipuX_csiY | |
343 | direct pad, or from ipuX_vdic. | |
344 | ||
345 | This entity has two source pads. One source pad routes to the | |
346 | pre-process encode task entity (ipuX_ic_prpenc), the other to the | |
347 | pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads | |
348 | can be activated at the same time if the sink pad is receiving from | |
349 | ipuX_csiY. Only the source pad to the pre-process viewfinder task entity | |
350 | can be activated if the sink pad is receiving from ipuX_vdic (frames | |
351 | from the VDIC can only be processed by the pre-process viewfinder task). | |
352 | ||
353 | ipuX_ic_prpenc | |
354 | -------------- | |
355 | ||
356 | This is the IC pre-processing encode entity. It has a single sink | |
357 | pad from ipuX_ic_prp, and a single source pad. The source pad is | |
358 | routed to a capture device node, with a node name of the format | |
359 | "ipuX_ic_prpenc capture". | |
360 | ||
361 | This entity performs the IC pre-process encode task operations: | |
362 | color-space conversion, resizing (downscaling and upscaling), | |
363 | horizontal and vertical flip, and 90/270 degree rotation. Flip | |
364 | and rotation are provided via standard V4L2 controls. | |
365 | ||
366 | Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace | |
367 | without motion compensation, and pixel reordering. | |
368 | ||
369 | ipuX_ic_prpvf | |
370 | ------------- | |
371 | ||
372 | This is the IC pre-processing viewfinder entity. It has a single sink | |
373 | pad from ipuX_ic_prp, and a single source pad. The source pad is routed | |
374 | to a capture device node, with a node name of the format | |
375 | "ipuX_ic_prpvf capture". | |
376 | ||
377 | It is identical in operation to ipuX_ic_prpenc, with the same resizing | |
378 | and CSC operations and flip/rotation controls. It will receive and | |
379 | process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is | |
380 | receiving from ipuX_vdic. | |
381 | ||
382 | Like the ipuX_csiY IDMAC source, it can perform simple de-interlace | |
383 | without motion compensation. However, note that if the ipuX_vdic is | |
384 | included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), | |
385 | it's not possible to use simple de-interlace in ipuX_ic_prpvf, since | |
386 | the ipuX_vdic has already carried out de-interlacing (with motion | |
387 | compensation) and therefore the field type output from ipuX_ic_prp can | |
388 | only be none. | |
389 | ||
390 | Capture Pipelines | |
391 | ----------------- | |
392 | ||
393 | The following describe the various use-cases supported by the pipelines. | |
394 | ||
395 | The links shown do not include the backend sensor, video mux, or mipi | |
396 | csi-2 receiver links. This depends on the type of sensor interface | |
397 | (parallel or mipi csi-2). So these pipelines begin with: | |
398 | ||
399 | sensor -> ipuX_csiY_mux -> ... | |
400 | ||
401 | for parallel sensors, or: | |
402 | ||
403 | sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ... | |
404 | ||
405 | for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route | |
406 | to the video mux (ipuX_csiY_mux) before sending to the CSI, depending | |
407 | on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in | |
408 | parenthesis. | |
409 | ||
410 | Unprocessed Video Capture: | |
411 | -------------------------- | |
412 | ||
413 | Send frames directly from sensor to camera device interface node, with | |
414 | no conversions, via ipuX_csiY IDMAC source pad: | |
415 | ||
416 | -> ipuX_csiY:2 -> ipuX_csiY capture | |
417 | ||
418 | IC Direct Conversions: | |
419 | ---------------------- | |
420 | ||
421 | This pipeline uses the preprocess encode entity to route frames directly | |
422 | from the CSI to the IC, to carry out scaling up to 1024x1024 resolution, | |
423 | CSC, flipping, and image rotation: | |
424 | ||
425 | -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> | |
426 | ipuX_ic_prpenc capture | |
427 | ||
428 | Motion Compensated De-interlace: | |
429 | -------------------------------- | |
430 | ||
431 | This pipeline routes frames from the CSI direct pad to the VDIC entity to | |
432 | support motion-compensated de-interlacing (high motion mode only), | |
433 | scaling up to 1024x1024, CSC, flip, and rotation: | |
434 | ||
435 | -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> | |
436 | 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture | |
437 | ||
438 | ||
439 | Usage Notes | |
440 | ----------- | |
441 | ||
442 | To aid in configuration and for backward compatibility with V4L2 | |
443 | applications that access controls only from video device nodes, the | |
444 | capture device interfaces inherit controls from the active entities | |
445 | in the current pipeline, so controls can be accessed either directly | |
446 | from the subdev or from the active capture device interface. For | |
447 | example, the FIM controls are available either from the ipuX_csiY | |
448 | subdevs or from the active capture device. | |
449 | ||
450 | The following are specific usage notes for the Sabre* reference | |
451 | boards: | |
452 | ||
453 | ||
454 | SabreLite with OV5642 and OV5640 | |
455 | -------------------------------- | |
456 | ||
457 | This platform requires the OmniVision OV5642 module with a parallel | |
458 | camera interface, and the OV5640 module with a MIPI CSI-2 | |
459 | interface. Both modules are available from Boundary Devices: | |
460 | ||
461 | https://boundarydevices.com/product/nit6x_5mp | |
462 | https://boundarydevices.com/product/nit6x_5mp_mipi | |
463 | ||
464 | Note that if only one camera module is available, the other sensor | |
465 | node can be disabled in the device tree. | |
466 | ||
467 | The OV5642 module is connected to the parallel bus input on the i.MX | |
468 | internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2. | |
469 | ||
470 | The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2 | |
471 | receiver, and the four virtual channel outputs from the receiver are | |
472 | routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1, | |
473 | vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is | |
474 | also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and | |
475 | OV5640 must not share the same i2c slave address. | |
476 | ||
477 | The following basic example configures unprocessed video capture | |
478 | pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and | |
479 | the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is | |
480 | imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are | |
481 | configured to output 640x480, and the OV5642 outputs YUYV2X8, the | |
482 | OV5640 UYVY2X8: | |
483 | ||
484 | .. code-block:: none | |
485 | ||
486 | # Setup links for OV5642 | |
487 | media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]" | |
488 | media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" | |
489 | media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" | |
490 | # Setup links for OV5640 | |
491 | media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]" | |
492 | media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" | |
493 | media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" | |
494 | # Configure pads for OV5642 pipeline | |
495 | media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]" | |
496 | media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]" | |
497 | media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]" | |
498 | # Configure pads for OV5640 pipeline | |
499 | media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]" | |
500 | media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]" | |
501 | media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]" | |
502 | ||
503 | Streaming can then begin independently on the capture device nodes | |
504 | "ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can | |
505 | be used to select any supported YUV pixelformat on the capture device | |
506 | nodes, including planar. | |
507 | ||
508 | SabreAuto with ADV7180 decoder | |
509 | ------------------------------ | |
510 | ||
511 | On the SabreAuto, an on-board ADV7180 SD decoder is connected to the | |
512 | parallel bus input on the internal video mux to IPU1 CSI0. | |
513 | ||
514 | The following example configures a pipeline to capture from the ADV7180 | |
515 | video decoder, assuming NTSC 720x480 input signals, with Motion | |
516 | Compensated de-interlacing. Pad field types assume the adv7180 outputs | |
517 | "interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf | |
518 | entity at its output pad: | |
519 | ||
520 | .. code-block:: none | |
521 | ||
522 | # Setup links | |
523 | media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" | |
524 | media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" | |
525 | media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" | |
526 | media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" | |
527 | media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" | |
528 | media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" | |
529 | # Configure pads | |
530 | media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" | |
531 | media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" | |
532 | media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]" | |
533 | media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" | |
534 | media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" | |
535 | media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]" | |
536 | ||
537 | Streaming can then begin on the capture device node at | |
538 | "ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any | |
539 | supported YUV or RGB pixelformat on the capture device node. | |
540 | ||
541 | This platform accepts Composite Video analog inputs to the ADV7180 on | |
542 | Ain1 (connector J42). | |
543 | ||
544 | SabreSD with MIPI CSI-2 OV5640 | |
545 | ------------------------------ | |
546 | ||
547 | Similarly to SabreLite, the SabreSD supports a parallel interface | |
548 | OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642 | |
549 | connects to i2c bus 1 and the OV5640 to i2c bus 2. | |
550 | ||
551 | The device tree for SabreSD includes OF graphs for both the parallel | |
552 | OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI | |
553 | CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled. | |
554 | The OV5640 module connects to MIPI connector J5 (sorry I don't have the | |
555 | compatible module part number or URL). | |
556 | ||
557 | The following example configures a direct conversion pipeline to capture | |
558 | from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt | |
559 | can be any format supported by the OV5640. $sensordim is the frame | |
560 | dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can | |
561 | be any format supported by the ipu1_ic_prpenc entity at its output pad: | |
562 | ||
563 | .. code-block:: none | |
564 | ||
565 | # Setup links | |
566 | media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" | |
567 | media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" | |
568 | media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]" | |
569 | media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" | |
570 | media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" | |
571 | # Configure pads | |
572 | media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]" | |
573 | media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]" | |
574 | media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]" | |
575 | media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]" | |
576 | media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]" | |
577 | ||
578 | Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl | |
579 | tool can be used to select any supported YUV or RGB pixelformat on the | |
580 | capture device node. | |
581 | ||
582 | ||
583 | Known Issues | |
584 | ------------ | |
585 | ||
586 | 1. When using 90 or 270 degree rotation control at capture resolutions | |
587 | near the IC resizer limit of 1024x1024, and combined with planar | |
588 | pixel formats (YUV420, YUV422p), frame capture will often fail with | |
589 | no end-of-frame interrupts from the IDMAC channel. To work around | |
590 | this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) | |
591 | when 90 or 270 rotations are needed. | |
592 | ||
593 | ||
594 | File list | |
595 | --------- | |
596 | ||
597 | drivers/staging/media/imx/ | |
598 | include/media/imx.h | |
599 | include/linux/imx-media.h | |
600 | ||
601 | References | |
602 | ---------- | |
603 | ||
604 | .. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf | |
605 | .. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf | |
606 | ||
607 | ||
608 | Authors | |
609 | ------- | |
59a4fcf3 MCC |
610 | |
611 | - Steve Longerbeam <steve_longerbeam@mentor.com> | |
612 | - Philipp Zabel <kernel@pengutronix.de> | |
613 | - Russell King <linux@armlinux.org.uk> | |
e1302912 SL |
614 | |
615 | Copyright (C) 2012-2017 Mentor Graphics Inc. |