Effects of Microlens Array Lens Size on Sub-Aperture Images in Light Field Cameras

Effects of Microlens Array Lens Size on Sub-Aperture Images in Light Field Cameras

Pei-Chuen Chiou, Cheng-Han Tsai, Chun-Fu Yang, and Jia-Han Li*

Department of Engineering Science and Ocean Engineering, National Taiwan University, Taiwan

(Received 27 February 2013; Accepted 9 April 2013; Published on line 1 June 2013)
*Corresponding author: jiahan@ntu.edu.tw
DOI: 10.5875/ausmt.v3i2.195

Abstract: The effects of microlens aperture size on 3D imaging quality for a 5 x 5 microlens array in a light field camera are studied by Fourier optics in consideration of pupil sizes and the locations of main lens and each lens in a microlens array. The larger size of the lenses in the microlens array provides higher sub-aperture image resolution. The techniques demonstrated in this paper can be useful to evaluate sub-aperture image performance in a light field camera without requiring large computations.

Keywords: microlens array; light field camera; 3D imaging; Fourier optics

Introduction

Recently, microlens arrays (MLA) have been adapted for a range of applications based on their tiny size, light weight, and suitability for mass production by a range of fabrication techniques. For instance, MLAs can be used to concentrate light [1] for high-efficiency solar cells. MLAs are also widely used in fiber communication systems [2, 3], leveraging their condensed arrangements to provide mass storage capacity. MLAs can also play an important role in 3D cameras, also known as light field cameras or plenoptic cameras [4-10], and the relevant techniques can be also applied to 3D display [10]. Adding an MLA in front of photosensor or CCD allows for the capture of directional light field and depth of field information. Conventional cameras only sum the total light rays striking each point in the image and loses all directional information. Using higher CCD resolutions and state-of-art MLA fabrication techniques, a 3D imaging camera based on light field optics has been demonstrated [9]. However, photosensor image quality depends on the microlens features, including lens size, thickness, and material.

In this paper, we study the effects of MLA aperture size on 3D imaging quality using Fourier transforms with MATLAB. This technique can be easily applied without the need of large-scale computation or complicated ray tracing processes.

Light Field Camera

In Figure 1, the light field camera is composed of a main lens, an MLA, and a photosensor or CCD. The components in the figure are not drawn to scale, and the MLA is actually smaller than the main lens.

The MLA is placed on the imaging plane of the main lens, with the light from the object focused by the main lens. The MLA then sorts the rays of light for the photosensor according to the light directions from the main lens aperture. As shown in Figure 2, all light rays passing through the specific sub-aperture of the main lens are focused on the corresponding photosensor pixels under different microlenses. Thus, each microlens can construct an image from one of the sub-apertures in the main lens. By extracting the related pixels for each microlens, sub-aperture images from different fields of view can be obtained. More detail related to the theory of light field cameras can be found in [9, 10].

Fourier Optics in Light Field Camera

The ray tracing method is often used to simulate light behaviors inside light field cameras [9, 10]. The alternative method is using Fourier optics with some constraints. Although this proposed method cannot show the images taken by the photosensors, the equivalent sub-aperture images can be reconstructed from the information of different fields of view after the MLA. In the light field camera, the main lens system (between the object plane and the Fourier plane) performs a Fourier transform and the MLA system (between the Fourier plane and the imaging plane) performs the other Fourier transform. Suppose that the light field of the object can be expressed as a function ${{U}_{l}}$ $\left( x,y \right)$, then the amplitude distribution, in consideration of the pupil of the main lens, can be shown as Equation (1):

\[{{U}_{l}}'\left( x,y \right)={{U}_{l}}\left( x,y \right)P\left( x,y \right){{e}^{-j\frac{k}{2f}\left( {{x}^{2}}+{{y}^{2}} \right)}},\tag{1}\]

where x and y are space coordinates of the system, k is the wavenumber defined as ${2π}{/λ}$, $λ$ is the wavelength of the incidence light, f is the focal length of the lens, and P is the pupil function which is described by Equation (2):

\[\begin{align} & P\left( x,y \right)=1,\text{ }inside\text{ }the\text{ }lens\text{ }aperture \\ & P\left( x,y \right)=0,\text{ }otherwise \\ \end{align}.\tag{2}\]

Using the Fresnel diffraction formula, the distribution ${{U}_{f}}$ $\left( x,y \right)$ of the field amplitude across the back focal plane of the main lens can be expressed by Equation (3),

\[\begin{align} & {{U}_{f}}\left( {{x}_{f}},{{y}_{f}} \right)= & \frac{{{e}^{j\frac{k}{2f}\left( x_{f}^{2}+y_{f}^{2} \right)}}}{j\lambda f}\int{\int_{-\infty }^{\infty }{{{U}_{l}}'\left( x,y \right){{e}^{j\frac{k}{2f}\left( {{x}^{2}}+{{y}^{2}} \right)}}{{e}^{-j\frac{2\pi }{\lambda f}\left( x{{x}_{f}}+yy{}_{f} \right)}}dxdy}}, \\ \end{align}\tag{3}\]

where ${{x}_{f}}$ and ${{y}_{f}}$ are spatial frequencies.

Substituting Equation (1) into Equation (3), we can obtain

\[\begin{align} & {{U}_{f}}\left( {{x}_{f}},{{y}_{f}} \right)= & \frac{{{e}^{j\frac{k}{2f}\left( {{x}_{f}}^{2}+{{y}_{f}}^{2} \right)}}}{j\lambda f}\overset{\infty }{\mathop{\iint\limits_{-\infty }{{}}}}\,{{U}_{l}}\left( x,y \right)P\left( x,y \right){{e}^{-j\frac{2\pi }{\lambda f}\left( x{{x}_{f}}+yy{}_{f} \right)}}dxdy. \\ \end{align}\tag{4}\]

Based on Fourier optics theory [11], ${{U}_{f}}$ $\left( x,y \right)$ is the Fourier transform of ${{U}_{l}}$ $\left( x,y \right)$ in consideration of pupil size.

The Fourier transform in the MLA is similar to that of the main lens described above, but it needs to consider the position and pupil size for each lens of the MLA. Although this simulation process cannot directly show the real light intensities captured on the photosensors, the image information after the MLA is technically equivalent to the image data information obtained from the Fourier optics transformation by the main lens and the MLA. This image information can be used to reconstruct the object images in the field of views with different directions.

Simulation Results and Discussions

To study the effect of the MLA lens size, we simulate light field photographs in a simplified light field camera system with a 5 × 5 MLA, i.e. N = 5 (Figure 2). In this section, the aperture of the main lens is fixed at 25 mm, and the large and small lens sizes of the MLA are 500 µm and 450 µm, respectively. For convenience, we do not consider the effect of the focal lengths of the main lens and the MLA on image quality. Thus, we separate the main lens aperture into 25 sub-regions according to the total number of microlenses, as shown in Figure 3. Each lens in the MLA records 25 sets of light field information from different the sub-regions of the main lens aperture. The numbers in the cells represent the coordinates in the main lens plane.

In this paper, we consider only a 5 x 5 microlens and a 5 × 5 field of view to demonstrate use of constrained Fourier optics to reconstruct the sub-aperture images following light field camera capture. In Fig. 4, the 5 × 5 images in the different fields of view are taken using an interval of 17 horizontal degrees from -34° to 34° and an interval of 14 longitudinal degrees from -28° to 28° using a traditional camera (SONY $α$-100). These 2D images are used to depict the object information in our simulations, using 25 different perspectives to construct a 3D scene.

Figure 5 shows the Fourier transformed images after the main lens and the MLA for large and small microlenses. The 5 × 5 images in Figure 5 represent the equivalent images after each MLA lens. It should be also noted that the images in Figure 5 computed from the Fourier optics after the main lens and the MLA are not the images captured on the photosensors. The sub-images in Figure 5(b) are more blurred than the sub-images in Figure 5(a) because the smaller lenses of the MLA act as smaller pupils to filter out high spatial frequency information.

By rearranging the locations of each sub-image of Figure 5, the sub-aperture images of different fields of view can be obtained and the results are as shown in Figure 6. Comparing Figures 6(a) and 6(b), larger microlenses in the MLA produce higher sub-aperture image resolution because more light information from different fields of view can be captured by the larger aperture of each lens in the MLA.

Conclusion

This paper demonstrates the effects of MLA aperture size on 3D imaging quality for a 5 x 5 MLA using Fourier optics in consideration of pupil sizes and locations of the main lens and each MLA lens. Other factors such as fabrication methods or material effects will be taken into account in future work. Simulation results show that larger MLA lenses provide higher sub-aperture image resolution. This technique can be easily applied to larger MLAs with more lenses without requiring significant computation time. The simulation results of this simple model can be useful to perform quick evaluations of sub-aperture image performance in light field cameras.

Acknowledgement

Support from the National Science Council of Taiwan through NSC 101-2221-E-002-101 and NSC 101-2221-E-002-099 are gratefully acknowledged.

References

  1. H. J. Nam, S. C. Jeong, D. Y. Jung, Y. K. Kim, K. M. Han, and J. Yi, "Fabrication of textured silicon solar cell using microlens as anti-reflection layer," in Conference on Optoelectronic and Microelectronic Materials and Devices (COMMAD), Sydney, Australia, 2008, pp. 246-248.
    doi: 10.1109/COMMAD.2008.4802137
  2. X. Zeng, J. Plain, S. Jradi, C. Darraud, F. Louradour, R. Bachelot, and P. Royer, "Integration of polymer microlens array at fiber bundle extremity by photopolymerization," Optics Express, vol. 19, no. 6, pp. 4805-4814, 2011.
    Available: http://www.opticsexpress.org/abstract.cfm?URI=oe-19-6-4805
  3. J. Kunde, A. C. Pliska, K. Krasnopolski, S. Grossmann, C. Bosshard, R. Bauknecht, A. Soufiane, R. Krahenbiihl, P. Zaina, T. Ammer, and A. Peterhans, "Novel releasable multi-fiber optical connectivity solution for optical communication using v-grooves and micro-lens arrays," in 1st Electronics Systemintegration Technology Conference, 2006, vol. 1, pp. 268-273.
    doi: 10.1109/ESTC.2006.280009
  4. S. H. Hong and B. Javidi, "Improved resolution 3d object reconstruction using computational integral imaging with time multiplexing," Optics Express, vol. 12, no. 19, pp. 4579-4588, 2004.
    Available: http://www.opticsexpress.org/abstract.cfm?URI=oe-12-19-4579
  5. S. H. Hong and B. Javidi, "Distortion-tolerant 3d recognition of occluded objects using computational integral imaging," Optics Express, vol. 14, no. 25, pp. 12085-12095, 2006.
    Available: http://www.opticsexpress.org/abstract.cfm?URI=oe-14-25-12085
  6. T. E. Bishop and P. Favaro, "The light field camera: Extended depth of field, aliasing, and superresolution," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 972-986, 2012.
    doi: 10.1109/TPAMI.2011.168
  7. F. P. Nava, "Super-resolution in plenoptic cameras by the integration of depth from focus and stereo," in 19th International Conference on Computer Communications and Networks (ICCCN), 2010, pp. 1-6.
    doi: 10.1109/ICCCN.2010.5560019
  8. E. H. Adelson and J. Y. A. Wang, "Single lens stereo with a plenoptic camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 99-106, 1992.
    doi: 10.1109/34.121783
  9. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, "Light field photography with a hand-held plenoptic camera," Stanford Tech Report CTSR, 2005.
    Available: http://graphics.stanford.edu/papers/lfcamera/
  10. X. Xiao, B. Javidi, M. Martinez-Corral, and A. Stern, "Advances in three-dimensional integral imaging: Sensing, display, and applications [invited]," Applied Optics, vol. 52, no. 4, pp. 546-560, 2013.
    Available: http://ao.osa.org/abstract.cfm?URI=ao-52-4-546
  11. J. W. Goodman, Introduction to fourier optics, 3rd ed. Englewood: Roberts and Company, 2005.

Refbacks

  • There are currently no refbacks.


Copyright © 2011-2018 AUSMT ISSN: 2223-9766