Light fields [Gershun 1939] of natural scenes are highly complex and vary within a scene from point to point. However, in many applications complex lighting can be successfully replaced by its low order approximation [Ramamoorthi and Hanrahan 2001]. The purpose of this research was to investigate the spatial behavior of light fields in natural scenes. We describe the light fields in terms of spherical harmonics and analyze their qualitative properties. We show that low order approximations of natural light fields vary smoothly and systematically, in accordance with very simple models. This finding has important implications for graphics, visualization, architectural design and scene perception research.@en

We applied fundamental perceptual and physico-mathematical studies to a fast method for luminance remapping of 2D texture maps which enhances perceived surface roughness in comparison with conventional 2D texture mapping. The fundamental physical mechanism underlying the method is the fact that texture contrast increases as the incident illumination tends towards grazing for rough matte surfaces, actually exploding near the shadow edge [Pont and Koenderink 2005]. A psychophysical study by Ho et al. [Ho et al. 2006] confirmed that human observers use texture contrast as a cue for relief-depth or surface roughness. Thus, 2D texture-mapped objects will appear to have a rougher surface if the texture contrast is increased as a function of the local illumination angle. In particular, we increase the bidirectional texture contrast in close accordance with the contrast gradients measured for real objects with rough surfaces. The method presented works well for random textures of locally-matte surfaces if the original texture does not have a contrast that is too high. This modification is in addition to the usual attenuation of the surface irradiance due to the angle of the incident illumination and the computational costs of the technique are similar to that of conventional diffuse shading. This low cost makes it straightforward to implement the technique with real-time shaders which allow interactive rendering on modern graphics hardware.@en

The structure of light fields of natural scenes is highly complex due to high frequencies in the radiance distribution function. However it is the low-order properties of light that determine the appearance of common matte materials. We describe the local light field in terms of spherical harmonics and analyze the qualitative properties and physical meaning of the low-order components. We take a first step in the further development of Gershun's classical work on the light field by extending his description beyond the 3D vector field, toward a more complete description of the illumination using tensors. We show that the three first components, namely, the monopole (density of light), the dipole (light vector), and the quadrupole (squash tensor) suffice to describe a wide range of qualitatively different light fields. In this paper we address a related issue, namely, the spatial properties of light fields within natural scenes. We want to find out to what extent local light fields change from point to point and how different orders behave. We found experimentally that the low-order components of the light field are rather constant over the scenes whereas high-order components are not. Using very simple models, we found a strong relationship between the low-order components and the geometrical layouts of the scenes.@en

The appearance of objects depends on their material, shape, and on the illumination conditions. Conversely, object appearance provides us with cues about the illumination and the material. This so-called inverse problem is basically underdetermined and therefore we expect that material and illumination perception are confounded. To gain insight into the relevant mechanisms, we rendered a set of artificial spheres for vastly different canonical light fields and reflectance functions. We used four physics-based bidirectional reflectance distribution functions (BRDFs) representing glossy, pitted, velvety, and matte material. The six illumination conditions were collimated illumination from four directions, hemispherical diffuse illumination, and fully diffuse (Ganzfeld) illumination. In three sub-experiments we presented pairs of stimuli and asked human observers to judge whether the material was the same, whether the illumination was the same, and for a subset in which either the illumination or the material was the same to judge which of the two was constant. We found that observers made many errors in all sub-experiments. In experiment 2 the illumination direction was chosen at random. Using an interactive interface, we asked human observers to match the illumination direction of a sphere of one of the four materials with that of a Lambertian sphere. We found systematical material-dependent deviations from veridical performance. Theoretical analysis of the radiance patterns suggests that judgments were based mainly on the position of the shadow edge. In conclusion, we found no evidence for 'material constancy' for perception of smooth rendered spheres despite vast quantitative and qualitative differences in illumination and in BRDF between the stimuli. Although human observers demonstrated some 'illumination constancy', they made systematic errors depending on the material reflectance, suggesting that they used mainly the location of the shadow edge. Our results suggest that material perception and light-field perception are basically confounded.@en

We studied image texture due to the shading of corrugated (3D textured) surfaces, which are Lambertian on the micro scale. Our theory applies to physically canonical cases of isotropic Gaussian random surfaces, under collimated illumination. In this investigation we analyze effects of oblique viewing, extending our theory which applied to normal viewing conditions only [5]. The theory for normal views predicts the structure tensors from either the gradient or the Hessian of the image intensity and allows for inferences of the orientation of irradiation of the surface. Even for surfaces that are not at all Gaussian, the BRDF [10] far from Lambertian, with vignetting and multiple scattering present, such inferences of the orientation of irradiation were accurate up to a few degrees. In this paper we derive predictions for oblique viewing conditions, for which the inferences of the irradiation orientation will deviate from the veridical value in a systematic manner, depending on the viewing and illumination directions. Theoretical predictions are compared with empirical data, for rendered and for real rough surfaces, and found to be in good agreement. We discuss issues of scale selection and robustness.@en

The appearance of objects in natural scenes is determined by their reflectance, their 3D texture, their shape and by the nature of the illumination. Results of previous experiments using computer generated images of spheres with different reflectance modes and under different canonical illuminations suggested that perception of reflectance mode and illumination are basically confounded. In the present study we investigate whether the conclusions from the experiments with simplified rendered spheres can be extended to ecologically valid images. We use two sets of photographs of real spheres, the first set is taken from the Dror Database [Dror et al. 2001], with simple reflectance modes but complex natural illumination (e.g. desklamp, lab, foodcourt). The second set is taken from the Utrecht Oranges Database [Pont and Koenderink 2003], with simple canonical illumination but material consisting of both reflectance and 3D texture differences (e.g. orange, golf ball, christmas decoration). We find that, although to a lesser extent, even in images of complex objects, perception of material and illumination are basically confounded. Overall, illumination and material are confounded most when we present rendered spheres that differ only in reflectance mode under simple canonical illumination conditions. Interestingly, adding complex natural illumination containing higher order angular frequencies helps to disambiguate this confound in material judgments, but not in illumination judgments. Most helpful was the addition of 3D texture.@en

Three dimensional surface corrugations on globally smooth surfaces give rise to brightness modulations of global shading patterns. We study systematic variations of such 3D image texture as a function of illumination and viewing geometry. The 3D texture is especially noticeable near the shadow terminator (for collimated illumination) or near the dark pole (for hemispherical diffuse illumination). We find that a simple micro-facet model, assuming locally Lambertian scattering, suffices to robustly describe texture contrast gradients of a large variety (measured and rendered textures; laboratory and field conditions) in a semi-quantitative manner. Robust statistical measures of the texture allows one to draw inferences concerning the nature of the light field (collimated to diffuse) and of surface roughness parameters, which can be used as input to the simplest BRDF models.@en

Current reflectance models for glossy surfaces assume that the reflectance from such objects can be represented by a linear combination of specular and diffuse reflectance functions. This assumption is based on the physics of reflectance of smooth dielectric materials such as plastic and holds quite well for perfectly smooth convex dielectric objects such as billiard balls. In this paper, however, we demonstrate visually and semi-quantitatively that linear combinations of body and interface scattering do not apply to rough surfaces (concavities on a meso scale) and to concave shapes (concavities on a macro scale), because the linearity is destroyed by multiple scattering. Using a physical model of a canonical case of rough opaque surfaces, namely a hemispherical pit, we demonstrate that the mechanism of multiple specular and diffuse reflexes causes a discrepancy between a linear superposition of specular and diffuse cases and the real glossy model. We discuss how the reflectance from locally glossy pitted surfaces may be calculated on the basis of two exact descriptions of the complete bidirectional reflectance distribution functions for locally specular and locally matte "thoroughly pitted surfaces."@en

Three human observers estimated the illumination direction for samples of random Gaussian surfaces illuminated by a collimated beam from random directions. These stimuli appear as 'texture' due to shading and shadowing (the surface on the microscale was Lambertian of uniform albedo; thus texture appears only through shading and shadowing). We found that observers were able to estimate the azimuth of the source with remarkable accuracy. In the shading regime (no shadows) the observers committed 180° azimuth errors with 50% probability, whereas in the shadow-dominated regime they were able to avoid this convex/concave confusion to a large extent. They evidently relied on second-order statistics in the shading regime and used an unidentified first-order cue in the shadow regime. The elevations of the source were also estimated with remarkable precision. We attribute this to the statistical homogeneity of the sample which can apparently be exploited by the observers. Likely cues are the fraction of shadowed surface, average intensity and rms contrast. The ability of human observers to estimate the illumination direction from surface texture no doubt contributes to the ability to estimate the light field in scenes, which is a prerequisite to the photometric parsing of scenes (shape from shading, and so forth).@en

We investigate the ability of human observers to judge the direction of illumination from image texture. Photographs of 61 real surfaces were used, taken from the Columbia-Utrecht Reflectance and Texture (Curet) database (http://www.cs.columbia.edu/CAVE/curet). All samples were normally viewed but obliquely illuminated, the elevation of the source being 22.5°, 45.0°, or 67.5°. The illumination was with a collimated, parallel beam. Stimuli were presented in random orientation, and observers had to judge both the elevation and the azimuth of the source. Observers judged the azimuth within approximately 15°, except for the fact that they committed random (with approximately 50% probability) sign flips (180° flips). Connected with this finding is the fact that observers judged the illumination to be from above rather than below in the overwhelming majority of cases, despite the fact that each case occurred with equal probability. The elevation of the illumination can be judged to some extent but is not far above chance level. The data are in good agreement with a simple model that bases the estimate of illumination direction on the second-order statistics of local luminance gradients. This locates the locus of the probable mechanism very early in the visual stream.

@en

In an image of a scene illuminated by a single source, such as a landscape in sun light, the light field is approximately the same at all locations in the scene. This is apparent from the common direction of cast shadows, the common polarity of illuminated and shaded parts of convex objects, and so forth. Here we concentrate upon the statistics of texture due to 3D surface corrugations. We show that patches of roughly uniform texture reveal the local direction of the illumination. In this way we are able to map the global structure of the "illuminance flow" through simple image processing techniques. We propose a theoretical treatment of texture due to illuminaton of rough surfaces and we present experiments on real textures and scenes. The illuminance flow is a robust indicator of the light field and thus reveals global structure in a scene. It is an important entity for many subsequent inferences from the image such as shape from shading.@en

We present a theory of image texture resulting from the shading of corrugated (three-dimensional textured) surfaces, Lambertian on the micro scale, in the domain of geometrical optics. The derivation applies to isotropic Gaussian random surfaces, under collimated illumination, in normal view. The theory predicts the structure tensors from either the gradient or the Hessian of the image intensity and allows inferences of the direction of irradiation of the surface. Although the assumptions appear prima facie rather restrictive, even for surfaces that are not at all Gaussian, with the bidirectional reflectance distribution function far from Lambertian and vignetting and multiple scattering present, we empirically recover the direction of irradiation with an accuracy of a few degrees.@en

We measured radiance distributions for black lining cloth and copper gauze using the convenient technique of wrapping the materials around a circular cylinder, irradiating it with a parallel light source and collecting the scattered radiance by a digital camera. One family of parallel threads (weave or weft) was parallel to the cylinder generator. The most salient features for such glossy plane weaves are a splitting up of the reflection peak due to the wavy variations in local slopes of the threads around the cylinders and a surface scattering lobe due to the threads that run along the cylinder. These scattering characteristics are quite different from the (off-)specular peaks and lobes that were found before for random rough specular surfaces, The split off-specular reflection is due to the regular structures in our samples of man-made materials. We derived simple approximations for these reflectance characteristics using geometrical optics.@en

Image texture due to 3D surface corrugations ("3D texture") differs fundamentally from image texture due to pigmentation variations of a smooth surface ("flat texture"). Flat texture yields no information concerning the way the surface is illuminated, whereas 3D texture is highly dependent upon the illumination. Question: Is the human observer able to perceive the illumination direction from 3D texture? Method: In order to address this question we extracted over sixty 3D textures due to natural materials from the Curet database. Fiducial illumination directions were 22.5°, 45° and 67.5° from the normal direction, all viewing directions were normal. The samples were clipped to circular discs and were randomly rotated in the picture plane. Observers indicated perceived illumination direction by way of an indicator panel showing a hemispherical boss on a plane, viewed normally to the plane. The simulated illumination direction was under the observer's control. Five observers viewed all samples in six different orientations. Result: We find that the azimuth of the illumination direction can be perceived quite veridically except for the fact that 180° confusions are frequent. The elevation cannot be perceived with great certainty (deviations up to 45° occurring frequently) though the increase of perceived elevation with true elevation is significant. We find a number of interesting variations on this pattern that depend critically upon the type of texture.@en

We derive the bidirectional reflectance distribution function for a class of opaque surfaces that are rough on a macroscale and smooth on a microscale. We model this type of surface as a distribution of spherical mirrors. Since our study concerns geometrical optics, it is only the aperture of the concavities that is relevant, not the dimension. The three-dimensional problem is effectively transformed into a much simpler two-dimensional one involving the possibly infinitely many reflections in a spherical mirror. We find that these types of surface show very strong backscattering when the pits are deep but forward scattering when the pits are shallow. Such surfaces also show spectral effects as a result of multiple reflections and polarization effects that are due to the orientation of the effective surface. Both this model and the locally diffuse thoroughly pitted surface model [Int. J. Comput. Vision 31, 129 (1999)] are superior to other models in that they allow for an exact treatment for physically realizable surface geometries.

@en

We identify a neglected source of information in the 3D texture in images of articulated surfaces. The texture is especially noticeable near the terminator of the body shadow. We provide a very simple model that allows us to describe the 3D texture contrast in detail. Experiments involving the data from the Curet database, laboratory observations, and informal snapshots reveal that the observations follow the theoretical expectations very closely. The 3D texture allows one to draw inferences concerning the nature of the light field (collimated to diffuse) and of parameters of the surface roughness that are the vital input to the simplest BRDF models, namely the range of slopes of surface micro facets, the distribution of heights relative to the fiducial surface, and the width of the surface protrusions. Thus 3D texture is a very useful cue when rerendering or BRDF estimations are to be attempted.

@en

In four experiments, we tested whether haptic comparison of curvature ranging from -4/m to +4/m is qualitatively the same for static and for dynamic touch. In Experiments 1 and 3, we tested whether static and dynamic curvature discrimination are based on height differences, attitude (slope) differences, curvature differences, or a combination of these geometrical variables. It was found that both static and dynamic haptic curvature discrimination are based on attitude differences. In Experiments 2 and 4, we tested whether this mechanism leads to errors in the comparison of stimuli with different lengths for static and dynamic touch, respectively. If the judgments are based on attitude differences, subjects will make systematic errors in these comparisons. In both experiments, we found that subjects compared the curvatures of strips of the same length veridically, whereas they made systematic errors if they were required to compare the curvatures of strips of different lengths. Longer stimuli were judged to be more curved than shorter stimuli with the same curvature. We conclude that similar mechanisms underlie static and dynamic haptic curvature comparison. Moreover, additional data comparison showed that static and dynamic curvature comparison is not only qualitatively, but also quantitatively similar.

@en

We investigated whether haptic comparison of the curvature of strips is influenced by the tilt (the average slope relative to the horizontal) of the curved strips. This particular stimulus manipulation was chosen to decide between two broad ways in which dynamic curvature comparison might be done: on the basis of the attitude (slope) differences over the surfaces or on the basis of the attitude (slope) differences between successively presented surfaces. For this purpose we conducted matching and discrimination experiments in which strips of constant curvature 9 cm long were touched dynamically and the tilts of the strips were varied. The results of the matching experiment showed some influence of tilt: the strips were judged to be slightly more curved if the tilt was nonzero than if it was zero. The results of the discrimination experiment did not show an effect of tilt. Because the attitude (slope) differences over the surfaces were independent of the tilt it is concluded that curvature comparison is based primarily on attitude differences over the curved surfaces.@en

An investigation was undertaken into whether haptic comparison of curvature and of shape is influenced by the length/width ratio of the hand. For this purpose three experiments were conducted to test the curvature matching of curved strips (experiment 1), the matching of cylindrically curved hand-sized surfaces (experiment 2), and the shape discrimination of elliptically curved hand-sized surfaces (experiment 3). The orientation of the stimuli with respect to the fingers was varied. The results of the two matching experiments showed that a given curvature is judged to be more curved when touched along the fingers than when touched across the fingers. The phenomenal flatness along and across the fingers was found to be different and subject dependent. The results of the shape-discrimination experiment showed that the orientation of ellipsoidal surfaces influences the judgments of the shapes of these surfaces. This influence could be predicted on the basis of results of the second matching experiment. It is concluded that similar mechanisms underlie the (anisotropic) perception of curvature and shape. For the major part the trends in the results can be explained by the length/width ratio of the hand and the phenomenal flatnesses.@en

Static haptic discrimination of the curvature of convex, concave, or straight 20-cm-long strips was investigated for nine placements on the hand. In one condition, the strips were touched with the palmar side of the hand, and in the other condition, with the dorsal side. The influence of the lengths of the strips, and thus of contact lengths, was also investigated. For all placements, discrimination was poorer in the dorsal than in the palmar condition, owing to poorer cutaneous resolution on the dorsal side of the hand (the kinesthetic stimulation was the same in both conditions). Thus cutaneous stimulation is important. In both experiments, performance appeared to depend primarily on contact length. Moreover, the discrimination thresholds for all different placements and contact lengths followed the same trend. We conclude that in these experiments the effective stimulus for the discrimination of curved strips is the total difference of local surface attitude - that is, the slope difference over the far ends of the stimulus.@en