If there in an error during a set_fmt, do not overwrite the previous
sizes with the invalid config.
Without this patch, v4l2-compliance ends up allocating 4GiB of RAM and
causing the following OOPs
[ 38.662975] ipu3-imgu 0000:00:05.0: swiotlb buffer is full (sz: 4096 bytes)
[ 38.662980] DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:00:05.0
[ 38.663010] general protection fault: 0000 [#1] PREEMPT SMP