MatMix 1.0 is a novel material probe we developed for quantitatively measuring visual perception of materials. We implemented optical mixing of four canonical scattering modes, represented by photographs, as the basis of the probe. In order to account for a wide range of materials, velvety and glittery (asperity and mesofacet scattering) were included besides the common matte and glossy modes (diffuse and forward scattering). To test the probe, we conducted matching experiments in which inexperienced observers were instructed to adjust the modes of the probe to match its material to that of a test stimulus. Observers were well able to handle the probe and match the perceived materials. Results were robust across individuals, across combinations of materials, and across lighting conditions. We conclude that the approach via canonical scattering modes and optical mixing works well, although the image basis of our probe still needs to be optimized. We argue that the approach is intuitive, since it combines key image characteristics in a ''painterly'' approach. We discuss these characteristics and how we will optimize their representations.@en

Materials are omnipresent. Recognizing materials helps us with inferring their physical and chemical properties, for instance if they are compressible, slippery, sweet and juicy. Yet in literature, much less attention has been paid to material perception than to object perception. This dissertation presents studies on a method to systematically measure human visual perception of opaque materials and test the influence of lighting and shape on material perception. In our studies, we applied multiple psychophysical methods such as matching, discriminating, and perceptual scaling to test the visual perception of materials for human observers. Beyond just matte and glossy material variations that were commonly tested in material perception literature, we included in total four canonical material modes to account for a wide range of materials, namely diffuse, asperity, forward, and mesofacet scattering for "matte", "velvety", "specular", and "glittery" material modes, respectively. For the lightings, we included three canonical lighting modes within a spherical harmonics and perception based framework, namely "ambient", "focus", and "brilliance" lighting. Based on the spherical harmonics analysis of the global lighting environment, we were able to quantify the “diffuseness” and “brilliance” of the light maps by using Xia’s diffuseness metric and a novel brilliance metric we proposed. Combining the four material modes and three lighting modes, we presented a canonical set that in combination with optical mixing supports a painterly approach in which key image features could be varied directly. With this method we were able to test and predict light-material interactions using both photographs of the real objects and computer rendered stimuli. We first introduced a new type of non-spherical appearance probe, implementing the painterly approach. Moreover, we developed an interactive interface that integrated the probe for an asymmetric matching task, where observers adjusted sliders to vary each material mode in the probe. The interface was found to be intuitive for inexperienced users and allowed purely visual quantitative measurements. Performances were generally well above chance and robust across experiments and observers, validating the approach. We further developed the material probe and expanded it to allow optical mixing of canonical lighting modes. In a light matching experiment and a 4-category discrimination experiment we found asymmetric perceptual confounds between judgments of material and lighting. Specifically, observers were found to be less sensitive to light changes than to material changes. Moreover, using this canonical approach, we were able to test and predict light-material interactions in two perceptual scaling experiments. To this aim a novel spherical harmonics based metric was introduced for quantifying the "brilliance". Lastly, we compared results from our probing method and results from other psychophysical experimental methods, namely perceptual scaling and discrimination, in which semantic information (for material attributes) was involved. Robust effects of light, shape, and light map orientation were found, in a material dependent way. To conclude, our research mainly contributed to 1) the development of a novel probing method that mixes image features of the proximal stimulus in a fluent manner instead of varying the distal physical properties of the stimuli, plus a validation that it works and that it allows quantitative measurements of material perception and material-lighting interactions; 2) understanding of visual perception of opaque materials and material-light-interactions in a wide ecological variety; 3) a validated model for predicting the material dependent lighting effects for matte, specular, velvet and glittery materials; and 4) the interpretations of the material perception results in a manner relating to shape and light. Our findings can be further applied to many subjects, such as industrial design, education, e-commerce, computer graphics, and future psychophysical studies. @en

We previously presented a systematic optics-based canonical approach to test material-lighting interactions in their full natural ecology, combining canonical material and lighting modes. Analyzing the power of the spherical harmonics components of the lighting allowed us to predict the lighting effects on material perception for generic natural illumination environments. To further understand how material properties can be brought out or communicated visually, in the current study, we tested whether and how light map orientation and shape affect these interactions in a rating experiment: For combinations of four materials, three shapes, and three light maps, we rotated the light maps in 15 different configurations. For the velvety objects, there were main and interaction effects of lighting and light map orientation. The velvety ratings decreased when the main light source was coming from the back of the objects. For the specular objects, there were main and interaction effects of lighting and shape. The specular ratings increased when the environment in the specular reflections was clearly visible in the stimuli. For the glittery objects, there were main and interaction effects of shape and light map orientation. The glittery ratings correlated with the coverage of the glitter reflections as the shape and light map orientation varied. For the matte objects, results were robust across all conditions. Last, we propose combining the canonical modes approach with so-called importance maps to analyze the appearance features of the proximal stimulus, the image, in contradistinction to the physical parameters as an approach for optimization of material communication.@en

To better understand the interactions between material perception and light perception, we further developed our material probe MatMix 1.0 into MixIM 1.0, which allows optical mixing of canonical lighting modes. We selected three canonical lighting modes (ambient, focus, and brilliance) and created scenes to represent the three illuminations. Together with four canonical material modes (matte, velvety, specular, glittery), this resulted in 12 basis images (the "bird set"). These images were optically mixed in our probing method. Three experiments were conducted with different groups of observers. In Experiment 1, observers were instructed to manipulate MixIM 1.0 and match optically mixed lighting modes while discounting the materials. In Experiment 2, observers were shown a pair of stimuli and instructed to simultaneously judge whether the materials and lightings were the same or different in a four-category discrimination task. In Experiment 3, observers performed both the matching and discrimination tasks in which only the ambient and focus light were implemented. Overall, the matching and discrimination results were comparable as (a) robust asymmetric perceptual confounds were found and confirmed in both types of tasks, (b) performances were consistent and all above chance levels, and (c) observers had higher sensitivities to our canonical materials than to our canonical lightings. The latter result may be explained in terms of a generic insensitivity for naturally occurring variations in light conditions. Our findings suggest that midlevel image features are more robust across different materials than across different lightings and, thus, more diagnostic for materials than for lightings, causing the asymmetric perceptual confounds.@en

Photographers and lighting designers set up lighting environments that best depict objects and human figures to convey key aspects of the visual appearance of various materials, following rules drawn from experience. Understanding which lighting environment is best adapted to convey which key aspects of materials is an important question in the field of human vision. The endless range of natural materials and lighting environments poses a major problem in this respect. Here we present a systematic approach to make this problem tractable for lighting–material interactions, using optics-based models composed of canonical lighting and material modes. In two psychophysical experiments, different groups of inexperienced observers judged the material qualities of the objects depicted in the stimulus images. In the first experiment, we took photographs of real objects as stimuli under canonical lightings. In a second experiment, we selected three generic natural lighting environments on the basis of their predicted lighting effects and made computer renderings of the objects. The selected natural lighting environments have characteristics similar to the canonical lightings, as computed using a spherical harmonic analysis. Results from the two experiments correlate strongly, showing (a) how canonical material and lighting modes associate with perceived material qualities; and (b) which lighting is best adapted to evoke perceived material qualities, such as softness, smoothness, and glossiness. Our results demonstrate that a system of canonical modes spanning the natural range of lighting and materials provides a good basis to study lighting–material interactions in their full natural ecology. @en

In previous research we tested visual material perception in matching and discrimination tasks, and found multiple material and lighting dependent interactions. We used four basis surface scattering modes, namely diffuse, asperity, forward, and mesofacet scattering, which we represented by covering a same-shaped 3D object with "matte", "velvety", "specular", and "glittery" finishes, respectively. All four birds were photographed in so-called ambient, focus and brilliance lighting, three canonical modes that are commonly used in lighting design. In the current study, we asked observers to judge the 12 stimuli on 9 material qualities terms that are commonly used in material perception studies, namely "matte", "velvety", "specular", "glittery", "hard", "soft", "rough", "smooth", and "glossy", to test 1) whether the naming of the scattering modes we used is proper; 2) whether certain material qualities can be brought out or eliminated by certain canonical types of illuminations. For each term and each stimulus image, we first asked observers to judge whether it was applicable. If they answered "yes", they were asked to rate the term on a scale from 1 to 7. Three repetitions for 12 stimuli and 9 qualities, made 324 trials per observer. In preliminary results with 9 inexperienced observers, we found that 1) "matte" applied to all materials, while "velvety", "specular" and "glittery" specifically applied to those respective materials; 2) for "specular" and "glossy" we found similar judgments; 3) brilliance light brought out glitteriness, specularity, glossiness and smoothness the best; 4) focus light resulted in a small increase in the velvetiness, softness and roughness ratings compared to those for ambient and brilliance light. In further analysis, we will look into how the key stimuli image features can trigger certain perceived qualities and how to design lighting to optimize appearance. @en