Physical modeling of the pixel binning

Blurring may be caused by an optical system (inherent noise of the camera, diffraction limit, etc.), and the PSF of the imaging device. The former can be modeled as linear space invariant while the the latter is considered as linear space variant³⁸. It is difficult to obtain the exact information about the linear space invariant, so it is usually compensated by the specific algorithms or avoided as much as possible. Besides the linear space invariant, in the process of image reconstruction, the PSF of the imaging device (which can also be regarded as the finiteness of the physical pixel-size) is an important factor for blur, which should be incorporated into the reconstruction procedure. As a complementary interpretation, there is a natural loss of spatial resolution caused by the insufficient sensor density and noise that occurs within the sensor or during transmission. As shown in Fig. 4(b), the spectrum loss will be more serious while the decimation factor increases. Figure 4(b) indicates again that the pixel-size is the main limiting factor of the systems which will determine whether it can directly record the high frequency fringes corresponding to the super-resolution of the samples.

(a) The PSF of the low-resolution sensor. (b) The spatial resolution loss after camera sampling.

In traditional multi-height reconstruction method^23,33, many efforts are made to implement subpixel shift to achieve the super-resolution, but the pixel binning is not taken into account, which does not accord with actual physical process. Thus, involving the process of recording digital images in the reconstruction procedure has drawn attention, and the enhancement in resolution has been validated³¹. The process of recording digital images is a down-sampling process which is usually modeled as a spatial averaging operator [LRPixel=∑ahk2(h=0,1…k2−1) where a _h is the gray value of the super-resolution intensity images, k is a decimation factor] as shown in Fig. 4(a). In the iterative process of the reconstruction, we convolve the estimated intensity of the field |Oij|2 with the PSF of the image sensor³⁸, and then it has the same dimension as the raw measurement.