The theory of illuminance flow estimation by structure tensors is generalized for oblique viewing of anisotropic texture. Previous theory is revised using general matrix formulations and predictions are compared with results on rendered images.

Light fields [Gershun 1939] of natural scenes are highly complex and vary within a scene from point to point. However, in many applications complex lighting can be successfully replaced by its low order approximation [Ramamoorthi and Hanrahan 2001]. The purpose of this research was to investigate the spatial behavior of light fields in natural scenes. We describe the light fields in terms of spherical harmonics and analyze their qualitative properties. We show that low order approximations of natural light fields vary smoothly and systematically, in accordance with very simple models. This finding has important implications for graphics, visualization, architectural design and scene perception research.

Three dimensional surface corrugations on globally smooth surfaces give rise to brightness modulations of global shading patterns. We study systematic variations of such 3D image texture as a function of illumination and viewing geometry. The 3D texture is especially noticeable near the shadow terminator (for collimated illumination) or near the dark pole (for hemispherical diffuse illumination). We find that a simple micro-facet model, assuming locally Lambertian scattering, suffices to robustly describe texture contrast gradients of a large variety (measured and rendered textures; laboratory and field conditions) in a semi-quantitative manner. Robust statistical measures of the texture allows one to draw inferences concerning the nature of the light field (collimated to diffuse) and of surface roughness parameters, which can be used as input to the simplest BRDF models.

Three human observers estimated the illumination direction for samples of random Gaussian surfaces illuminated by a collimated beam from random directions. These stimuli appear as 'texture' due to shading and shadowing (the surface on the microscale was Lambertian of uniform albedo; thus texture appears only through shading and shadowing). We found that observers were able to estimate the azimuth of the source with remarkable accuracy. In the shading regime (no shadows) the observers committed 180° azimuth errors with 50% probability, whereas in the shadow-dominated regime they were able to avoid this convex/concave confusion to a large extent. They evidently relied on second-order statistics in the shading regime and used an unidentified first-order cue in the shadow regime. The elevations of the source were also estimated with remarkable precision. We attribute this to the statistical homogeneity of the sample which can apparently be exploited by the observers. Likely cues are the fraction of shadowed surface, average intensity and rms contrast. The ability of human observers to estimate the illumination direction from surface texture no doubt contributes to the ability to estimate the light field in scenes, which is a prerequisite to the photometric parsing of scenes (shape from shading, and so forth).

We measured radiance distributions for black lining cloth and copper gauze using the convenient technique of wrapping the materials around a circular cylinder, irradiating it with a parallel light source and collecting the scattered radiance by a digital camera. One family of parallel threads (weave or weft) was parallel to the cylinder generator. The most salient features for such glossy plane weaves are a splitting up of the reflection peak due to the wavy variations in local slopes of the threads around the cylinders and a surface scattering lobe due to the threads that run along the cylinder. These scattering characteristics are quite different from the (off-)specular peaks and lobes that were found before for random rough specular surfaces, The split off-specular reflection is due to the regular structures in our samples of man-made materials. We derived simple approximations for these reflectance characteristics using geometrical optics.

In an image of a scene illuminated by a single source, such as a landscape in sun light, the light field is approximately the same at all locations in the scene. This is apparent from the common direction of cast shadows, the common polarity of illuminated and shaded parts of convex objects, and so forth. Here we concentrate upon the statistics of texture due to 3D surface corrugations. We show that patches of roughly uniform texture reveal the local direction of the illumination. In this way we are able to map the global structure of the "illuminance flow" through simple image processing techniques. We propose a theoretical treatment of texture due to illuminaton of rough surfaces and we present experiments on real textures and scenes. The illuminance flow is a robust indicator of the light field and thus reveals global structure in a scene. It is an important entity for many subsequent inferences from the image such as shape from shading.

We present a theory of image texture resulting from the shading of corrugated (three-dimensional textured) surfaces, Lambertian on the micro scale, in the domain of geometrical optics. The derivation applies to isotropic Gaussian random surfaces, under collimated illumination, in normal view. The theory predicts the structure tensors from either the gradient or the Hessian of the image intensity and allows inferences of the direction of irradiation of the surface. Although the assumptions appear prima facie rather restrictive, even for surfaces that are not at all Gaussian, with the bidirectional reflectance distribution function far from Lambertian and vignetting and multiple scattering present, we empirically recover the direction of irradiation with an accuracy of a few degrees.

We investigate the ability of human observers to judge the direction of illumination from image texture. Photographs of 61 real surfaces were used, taken from the Columbia-Utrecht Reflectance and Texture (Curet) database (http://www.cs.columbia.edu/CAVE/curet). All samples were normally viewed but obliquely illuminated, the elevation of the source being 22.5°, 45.0°, or 67.5°. The illumination was with a collimated, parallel beam. Stimuli were presented in random orientation, and observers had to judge both the elevation and the azimuth of the source. Observers judged the azimuth within approximately 15°, except for the fact that they committed random (with approximately 50% probability) sign flips (180° flips). Connected with this finding is the fact that observers judged the illumination to be from above rather than below in the overwhelming majority of cases, despite the fact that each case occurred with equal probability. The elevation of the illumination can be judged to some extent but is not far above chance level. The data are in good agreement with a simple model that bases the estimate of illumination direction on the second-order statistics of local luminance gradients. This locates the locus of the probable mechanism very early in the visual stream.

We derive the bidirectional reflectance distribution function for a class of opaque surfaces that are rough on a macroscale and smooth on a microscale. We model this type of surface as a distribution of spherical mirrors. Since our study concerns geometrical optics, it is only the aperture of the concavities that is relevant, not the dimension. The three-dimensional problem is effectively transformed into a much simpler two-dimensional one involving the possibly infinitely many reflections in a spherical mirror. We find that these types of surface show very strong backscattering when the pits are deep but forward scattering when the pits are shallow. Such surfaces also show spectral effects as a result of multiple reflections and polarization effects that are due to the orientation of the effective surface. Both this model and the locally diffuse thoroughly pitted surface model [Int. J. Comput. Vision 31, 129 (1999)] are superior to other models in that they allow for an exact treatment for physically realizable surface geometries.

We identify a neglected source of information in the 3D texture in images of articulated surfaces. The texture is especially noticeable near the terminator of the body shadow. We provide a very simple model that allows us to describe the 3D texture contrast in detail. Experiments involving the data from the Curet database, laboratory observations, and informal snapshots reveal that the observations follow the theoretical expectations very closely. The 3D texture allows one to draw inferences concerning the nature of the light field (collimated to diffuse) and of parameters of the surface roughness that are the vital input to the simplest BRDF models, namely the range of slopes of surface micro facets, the distribution of heights relative to the fiducial surface, and the width of the surface protrusions. Thus 3D texture is a very useful cue when rerendering or BRDF estimations are to be attempted.