The light field in natural scenes

Abstract

This thesis focuses on the properties of light fields with respect to object appearance. More specifically, our interest was mainly directed to the structure and spatial variation of light fields in natural scenes. We approached the structure of light fields by means of spherical harmonics which allows one to divide the complicated spherical functions of local light fields in frequency bands and to analyze those separately. In chapter 2 we empirically studied the variation of different frequencies of light field approximations over natural scenes by means of panoramic photography and found that the low order components show systematic and stable spatial variations whereas the high order components vary rapidly and chaotically over most scenes. We showed how the ‘quality of light’ can be expressed by means of a light vector and a squash tensor which provide a formal mathematical but nevertheless very intuitive way of representation bridging the gap between scientific and artistic understandings of light. In chapter 3 we continued the study of the spatial behavior of light fields more thoroughly considering complicated scenes and focusing on the 2nd order structures which we measured by our custom made device. In Chapter 4 we presented a technique with which the 2nd order descriptions can be recovered for an entire three dimensional scene on the basis of a limited number of measurements and presented a visualization of the structure of light fields by means of light tubes. Chapter 5 was devoted to possible topological structures of light fields. In the Appendix we provided additional examples of methods, measurements and visualization of light fields. In chapter 2 we considered simple scenes such as typical ‘street’, ‘wall’ and ‘forest’ scenes and studied the spatial variation of light fields along the main axes of symmetry of the scenes. The measurements were performed photographically by utilizing panoramic imaging. We described the local light fields in terms of spherical harmonics up to the 10th order and analyzed the qualitative properties and physical meanings of the low order components. We took a first step in a further development of Gershun’s classical work on the light field by extending his description beyond the three-dimensional vector field, towards a more complete description of the light field using tensors. We showed that the three first components, namely the monopole (density of light), the dipole (light vector) and the quadrupole, which we named ‘squash tensor’, suffice to describe a wide range of qualitatively different light fields. The empirical analysis allowed us to conclude that the low order components dominate the structure of most light fields. The low order components are rather constant over the scenes whereas high order components are not. Using simple models, we found a strong relation between the low order components and the geometrical layouts of the scenes. In chapter 3 we presented a new technique to capture the global structure of the light field in terms of spherical harmonics functions. Our custom made device Plenopter allows to perform measurements of light fields up to the second order easily, quickly and with a high dynamic range. Using that device we continued the research presented in chapter 2 by considering measurements across the scenes, along the line orthogonal to the main axis of geometrical symmetry. The measurements clearly indicate that in scenes of similar geometry the light fields demonstrate characteristic variations of the light vector and the squash tensor over the scene. This happened despite the fact that the scenes possessed different reflective properties and even were differently oriented with regard to the primary light sources. In chapter 4 we presented a method for measuring, reconstructing and visualizing the global structures of light fields in finite 3D spaces. We used the Plenopter to measure second order light fields at points over a regular grid and interpolate the spherical harmonics coefficients to calculate the light fields at all points of a closed 3D space. We presented a new way of visualizing the light field in 3D space by means of light tubes which indicate the radiant flux transfer and provide intuitive insights into the global structure of the light field through the entire space of the scene. In chapter 5 we considered possible topological structures of light fields. We studied singular points and showed that all generic topological structures that are possible for 2D vector fields may also occur in the case of light fields. We provided models which showed that light tubes can even be closed. The global structure of the light field may be described by means of the singular points.