Visual inferences of material changes: color as clue and distraction

Abstract

In a snapshot, a scene consists of things, but across time, the world consists of processes. Some are cyclical, for example, trees changing foliage through the seasons, surfaces getting wet and drying out; others are unidirectional, for example, fruit ripening and then decaying, or dust accumulating on surfaces. Chemical and physical properties of objects provide them with specific surface patterns of colors and textures. When endogenous and exogenous forces alter these colors and textures over time, the ability to identify these changes from appearances can have great utility in judging the composition, state, and history of objects. This short review presents thoughts on studying visual inferences of the properties of materials and their changes, including how to acquire calibrated images of time-varying materials, how to model time-varying appearance changes, how to measure observers’ identification abilities, and how to parse out the perceptual qualities that help or hinder in recognizing materials and their states. For instance, if color information is removed, observers do significantly worse at recognizing materials and their changes, especially for organic materials. The role of color in object and scene recognition is still being debated, so elucidating color’s role in material identification may also help to resolve the wider issue. This review introduces material change as an object of study in human perception and cognition, because the visual traces of changes are integral components of material and object identity. Visually based judgments of materials share the property of propensity with mental inferences, and conscious or unconscious visual imagery may play a role in setting expectancies for object shapes and properties.

MATERIAL APPEARANCE AS CLUE TO HISTORY

In studies of color constancy, each material’s reflectance is assumed to be constant and the task is to identify objects and materials across illumination changes.^1-8 However, material appearances often change due to endogenous physical and chemical processes such as ripening or decay, and exogenous processes such as the effects of exposure to light, heat, water, or dust.^9,10 These changes provide rich and extremely useful cues to history. Look at Figure 1(a) and judge whether this building has recently been painted. When looking at the building, not only do we discern that it has not been painted in some time, we also recognize dirt as the cause of the darker pattern on the façade. Similarly, one can judge that the chain in Figure 1(b) has not recently been tied at this location. The top part of the chain is not informative, but the pattern of rust on the cobblestones gives the game away. How do we figure out the nature of the relevant cues and make inferences that approach veridicality? In some cases, the physics of the situation is simple enough to understand on a näive level.¹¹ Figure 1(c) illustrates how repeated rain can clean some parts of a vertical surface and redeposit dirt on lower parts.

FIGURE 1.

(a) Dirt accumulated on building. (b) Rust from chain on flagstones. (c) Diagram illustrates how dust on a vertical façade is washed down by rain and redeposited as dirt lower on the surface, based on surface geometry and absorption. (Reprinted with permission from Ref 11. Copyright 1996 ACM Press)

In other instances, such as judging the ripeness of bananas from their appearance, the physics may be too complicated, but it may be possible to explain image variations by simple temporal processes. The images of bananas in different stages of ripeness as shown in Figure 2(a), differ in color and spatial patterns, but since there is no reason to think that one part of the banana is inherently different from any other part, it is worth considering whether the spatial variations arise because all points follow the same temporal process but at different rates and from different starting positions. To illustrate such an analysis, the middle panel of Figure 2(b)¹² plots the input of the red sensor of a camera of three points each on images of drying wood, rusting steel, burning wood, decaying apple, and drying paper, acquired at regular intervals during material changes. The bottom panel shows that the three curves for each material can be superimposed on a single curve when they are appropriately compressed or expanded on the time-scale, and placed on the proper segments, by using dynamic time-warping.¹³ These unifying curves represent the dominant process, for example, each part of the banana skin turns yellower as chlorophyll is replaced by carotene. The spatial pattern is assumed to arise from the temporal differences, so a test of the model is to see if random variations in temporal parameters recreate typical spatial pattern changes.

FIGURE 2.

(a) Bananas ripening and decaying (Google Images). (b) Top: Samples from five materials. Middle: Red input over time as the material changes. Bottom: Traces from middle panel superimposed on single curves using dynamic time-warping. (Reprinted with permission from Ref 12. Copyright 2006 Association for Computing Machinery)

In other cases, even though there may be just one cause for the changes, the time-varying spatial patterns are of primary importance. For example, Figure 3(a) shows that the perceptual effects of accumulating dust include changing degrees of translucency, dust shadows, loss of specular highlights, and color mixtures between the dust’s color and diffuse object colors. Similarly, the appearance of a freshly painted surface is highly specular, but as the liquid medium dries, the specular component diffuses out. Figure 3(b) provides schematics of typical light paths through dust, water-color, oil-paint, and water covering a moderately rough solid surface. As time passes, the layer of dust gets thicker, paints get denser, and water evaporates. These processes lead to quite different spatially distributed changes in appearance. The basic physical measurement for material appearance is the bidirectional reflectance distribution function (BRDF), which defines for each point of a surface the fraction of light incident from each direction that is reflected in each direction.¹⁴ For some purposes, it is necessary to measure spectrally-sampled and time-varying variants of the BRDF. To summarize the enormous data constituting BRDFs, with a reasonable number of parameters, a number of phenomenological and physical models have been proposed,^15-19 but there are many details that they are unable to capture.²⁰ Modeling time-varying BRDFs requires more than just making the parameters a function of time, it also requires considering more factors, for example, the diffuse color shifts resulting from paint drying, are affected not just by the colors of the paint and the surface, but also by the thickness of the paint and the absorbance of the material. For particular processes like paints drying and dust accumulating, it has been possible to reproduce appearance changes reasonably well by combining multiple analytic BRDF models and fitting them to particular time-varying perceptual qualities like specularity.²¹

FIGURE 3.

(Top) Images of a glossy teapot and cup as dust accumulates. (Bottom) Simplified schematic of possible light paths through dust, water-color, oil-paint, and clear liquid covering a moderately rough surface. (Reprinted with permission from Ref 21. Copyright 2006 European Association for Computer Graphics)

ACQUIRING IMAGES OF TIME-VARYING MATERIALS

Before figuring out how something is done, it is important to figure out if it can be done, i.e., prior to testing which classes of computational models are relevant to human perception, it needs to be established what kinds of materials and their changes can be recognized by human observers, and whether this can be done without object-shape and time-course cues. Since it is unrealistic to have observers continuously monitor slowly changing real materials, it is necessary to acquire images that sample these changes and that can be presented at experimentally convenient time-scales in realistic, random, or jumbled sequences of altered images. Two considerations are key in capturing time-varying BRDFs: the time samples should be close enough not to miss important variations, and the angular domain should be sampled densely to capture the high-frequency changes due to specularities. Figure 4(a) shows a multilight, multicamera system that has been used to acquire calibrated time-varying measurements of materials.¹² The system consists of an icosahedron dome with 16 calibrated cameras (1300 × 1030 pixels) placed at the vertices, and 150 white LED lights evenly spaced at the edges. This system is capable of acquiring multiple images of a sample placed at the center and lit by combinations of multiple light sources simultaneously. In conditions where the angular sampling of this system is inadequate, but fewer viewpoints and sparser time-sampling would suffice, a smaller number of cameras can be mounted on a robot arm that permits rapid small movements²¹ (Figure 4(b)).

FIGURE 4.

(a) A multi-light, multi-camera system designed to acquire simultaneous images of changing materials from different lighting angles and viewpoints,¹² thus providing fine temporal and angular resolution. (b) Four cameras mounted on robot arms that permit rapid small movements coupled with a light-source that can move inside a semi-circle.²¹ Correction added on May 25 2011, after first online publication. Figure 4 was incomplete and has been replaced.

Table 1 lists 26 materials whose images were acquired along with their natural or speeded changes and Figure 5(a) shows the initial and final images of each sequence.¹² Samples of materials were allowed to undergo either natural changes or the changes were speeded along with the help of artificial aids like heat or chemicals.

TABLE 1.

Twenty-Six Materials and Their Changes

Wood burning	Rock drying	Wood drying
Orange cloth drying	Light wood drying	White felt drying
Quilted paper drying	Cardboard drying	Wet brick drying
Apple core decaying	Wood drying	Green cloth drying
Banana decaying	Steel rusting	Leaf under humid  heat
Patterned cloth  drying	Apple slice decaying	Granite drying
Tree bark drying	Potato decaying	Wood getting  charred
Waffle toasting to  burnt	Bread toasting to  burnt	Cast iron rusting
Copper oxidizing to  patina	Cast iron rusting

FIGURE 5.

(a) Initial and last fronto-parallel images of the 26 material changes listed in Table 1. (b) Same images, but chromatic information has been removed.

MATERIAL IDENTIFICATION BY HUMAN OBSERVERS

To measure baseline performance for the materials in Figure 5, Yoonessi and Zaidi²² presented square fronto-parallel images one at a time on a calibrated monitor. Condition 1 consisted of single initial images, Condition 2 of the initial and last images of the sequence, and Condition 3 of a complete sequence of change images (the time frame did not reflect the speed of the actual process). Observers were allowed to view the images freely for as long as they liked, and then typed the names of their choice for the material and the change.

For studies of human perception, baseline data are interesting only as a way to understand the perceptual qualities underlying the performance. The CIE has proposed color, texture, translucency and gloss as the components of material appearance.²³ This scheme is almost certainly incomplete, but can provide a starting point. In particular, if the chromatic information is removed (Figure 5(b)), it leaves texture, gloss, and translucency essentially unchanged. Hence, comparing performance levels for identifying materials in colored images to performance levels for the achromatic versions of these images can reveal where color is an important cue. After measuring a baseline for material and change identification, a new set of observers repeated all three conditions with achromatic images.²²

Average material identification performance (seven observers) is shown in Figure 6 as percent corrects for the three conditions. With full color information, observers identified 40% of the materials correctly when given only the initial images. The error bars represent variability across observers, and indicate that these qualitative measurements were quite reliable. There were some interesting confusions in the single image condition: 7/7 observers responded clay or tile for orange cloth. Clearly color cues can mislead as well as reveal. In addition, 0/7 observers identified bread correctly. Other frequent confusions were water or stone for metal, stone for wood, and paper and cloth for each other. It is possible that identification of shiny (metal) and textured (bread) materials may be difficult for planar surfaces in the absence of 3D and/or motion information. Since the Gu et al. set¹² includes images of materials from multiple viewpoints and lighting directions, it should be possible to embellish the recognition experiment either by rendering 3D shapes, or by letting observers examine materials, while manipulating the viewpoint and lighting. These manipulations should allow observers to judge glossiness of materials by seeing if specularities move across the surface,²⁴ and to estimate 3D surface texture from the information provided by varying viewpoints and illumination angles.^25-28

FIGURE 6.

(a) Percent correct for material identification for initial images, initial plus final image, and complete sequence of images. (b) Odds ratios comparing performance with and without color information.²²

When both the initial and final images were available, performance went up to 60%, that is, seeing a recognizable change across two states also helped in recognizing the material. Surprisingly, performance was improved less when observers were presented with the whole sequence of changes than when they saw just the pair of initial and final images. The reasons for the performance decline are not clear to us. These could include the confusions caused by the conversion of time into spatial arrangements of images for the experiment, and the possibility that observers are more familiar with the beginning and end of the temporal processes than with the in-between stages.

Figure 6 also addresses the role of color in material identification. Comparing heights of the bars for the gray-scale and color images, reveals that removing color information had a detrimental effect on average performance in all three conditions. The bottom plot shows odds ratios for color over gray-scale conditions, and confirms statistically that material recognition is significantly worse without color cues.

In a study²⁹ where materials were presented as images of 3D objects, and observers were asked to identify broad categories like leather or plastic, identification performance was around 90% even for rapid presentations. The images of flat materials in Figure 5, were missing the 3D information that correlates with glossiness, softness and many other material qualities, and within category confusions, for example, pear for apple, were considered to be incorrect responses. To compare category-based performance across the studies, the 26 materials were informally divided into the five categories in Table 2.

TABLE 2.

Material Categories and Constituents

Organic	Apple, potato, leaf, waffle, banana,…
Wood	All types of woods
Mineral	Rock, marble, granite, brick, …
Metal	Copper, iron,…
Fabric	Cloths, felts, paper, quilt,…

When performance is tabulated for category identification with full color images, performance goes up to 70% for single images of some categories, and even higher for the pair and sequence conditions (Figure 7). Observers can thus recognize some categories of materials well, even for these impoverished stimuli. Metals were the least correctly recognized in static images of flat sheets, even when additional images included rust or patina. Wood and minerals were recognized best in single images, probably because of distinctive surface patterns. Color information improved performance most for organic images.²²

FIGURE 7.

Percent correct identifications for material categories in the three conditions.²²

Color names in almost all languages have historically been intrinsically bound to materials. The earliest use of the word color seems to have been in the 13th century as skin color, the word white probably came from bright or light, the word green comes from the same Germanic root as grow, red shares Greek roots with rust and ruddy, a large number of colors are named after animals (sable), plants (mauve), insects (crimson), stones (turquoise), fruits (orange), foods (chocolate), fabric (linen), metals (silver), and so on. To earlier results that mental representations of typical organic objects can bias color judgments,³⁰ these results add evidence that color is an integral part of the mental representations of material appearance.

CHANGE IDENTIFICATION BY HUMAN OBSERVERS

Since the perception of material changes is a new topic, to give an intuitive feel for the phenomena, it is worth asking readers to look at some image sequences in Figures 8-15, and to try to guess the nature of the material and the change before reading the description. (Note: In the experiment, the images were acquired by calibrated cameras and displayed on calibrated monitors for color fidelity with the real materials, but the reproductions below are unlikely to have the same degree of fidelity.)

FIGURE 8.

(a)–(h) What are these materials? What are the changes they are undergoing? See text for answers.

FIGURE 15.

Based just on visual form, which of these drapes would be predicted to be softer to the touch? (a) Cloth drapes. (b) Limestone ‘draperies’ in King Solomon’s Cave, Mole Creek Karst National Park, Tasmania. (c) Concrete cloth.

The images in Figure 8(a) are of wood being burned, but since there is no time-scale information, observers also reported mold growing on wood, or a dark stain being applied. Figure 8(b) consists of images of a wet granite drying, but observers also reported it as a change of illumination from shade to light. Figure 8(c) seems to be very recognizable images of decaying banana, but a couple of observers reported wood getting charred. The images of a wet fabric drying in Figure 8(d), were sometimes confused as an illumination change, or a fabric being bleached. Wood drying in Figure 8(e) was sometimes confused as bleaching or wood being sanded. Figure 8(f) shows wet felt drying, and the spatial pattern of drying may be responsible for the absence of illumination change reports. The images of a leaf drying in Figure 8(g) were reported correctly by almost all observers despite the lack of shape information, possibly because of the conjunction of typical colors and patterns. Figure 8(h) shows images of bread toasting and eventually burning.

In recognizing material changes, when observers had access only to the initial and final images, they recognized the change on over 60% of the trials (Figure 9). This is similar to material recognition performance levels on the same trials. Again surprisingly, observers were less accurate when shown the full sequence of images. The advantage provided by color cues is evident when comparing results for the colored and achromatic images in the first two panels, and the values of the odds ratios in the last panel. The classes of materials where color information is not helpful are particular candidates for analyses of spatial cues. It would also be interesting to extract spatial patterns for changes such as drying across different classes of materials, and to judge if the changing patterns are diagnostic for material recognition by combining them with colors from other materials.

FIGURE 9.

Percent correct for change identification for initial and last images, and for the whole sequence. Odds ratios compare performances with and without color information.²²

The most common confusions, wet with polished or stained, drying with bleaching or illumination increase, burning with mold, decaying with burning, rusting with dirt/dust accumulating, and patina with mold, reflect natural ‘metamers’, that is, physically distinct processes that generate similar images. Some of these changes would not be ‘metamers’ in real life because they take different amounts of time, others would not be ‘metamers’ if shape and multiview information was also provided, for example, as fruits and vegetables decay, they not only lose the visual cue of glossiness, but also become softer which is reflected in object shape rather than surface pattern, and the texture of charred wood would easily distinguish it from surface patterns formed by dark mold or stain. Consistent and inconsistent conjunctions of surface information with shape or texture could thus be used to provide alternate probes for material perception. For instance, observers could be asked to judge the realism of a sequence of wood images with texture changes consistent with burning but without the corresponding color changes. Such experiments could use the set of images taken from multiple viewpoints and lighting directions.¹²

PARSING MATERIAL IDENTIFICATION INTO PERCEPTUAL COMPONENTS

A major question is whether material appearance can be reduced to a discrete number of perceptual qualities, and whether perception of material changes can be reduced to changes in these qualities. A simple experimental strategy is to remove one or more perceptual qualities and test identification performance. Removing color information²² is easy, but removing other qualities requires identifying image parameters correlated with each property. Besides the work on time-varying materials discussed in the first section, considerable work has been done on rendering classes of materials in static states, and some work on the human perception of material qualities like glossiness and translucency. From this work, two sorts of stimulus analysis strategies can be adapted for experiments with time-varying materials: those based on BRDFs and those based on image statistics.³¹

Figure 10(a) shows spheres rendered from BRDFs of a hundred different materials, demonstrating a large number of perceptual qualities.³² If the BRDF for each of these materials is considered a high-dimensional vector (each measurement forming one element), then linear or nonlinear dimensionality reduction can be applied to obtain a manifold of between 15 and 45 dimensions that efficiently characterizes all the BRDFs. These dimensions are similar to the number of parameters in physics-based BRDF models for textured surfaces.¹⁹ By having observers choose exemplars of perceptual qualities like redness, greenness, blueness, specularness, diffuseness, glossiness, metallic-like, plastic-like, roughness, silverness, gold-like, fabriclike, acrylic-like, greasiness, dustiness, and rubber-like,³² methods like support vector machines³³ can be used to generate linear functions that correspond to variation along the percepts in the low-dimensional manifold. Even though this method does not lead to a list of independent qualities, as some sets of qualities above lead to correlated percepts, the BRDF subspace correlates can be used to manipulate the values of particular qualities (Figure 10(b)). Images that increase or decrease certain qualities can then be used in experiments to identify the qualities critical for recognizing material changes.

FIGURE 10.

(a) Spheres made from a 100 different materials, demonstrating a large number of perceptual qualities: redness, greenness, blueness, specularness, diffuseness, glossiness, metallic-like, plastic-like, roughness, silverness, gold-like, fabriclike, acrylic-like, greasiness, dustiness, rubber-like, and others. (b) Simulations of increases in redness, silver, gold, and specularity from left to right, in first to fourth row, respectively. (Reprinted with permission from Ref 32. Copyright 2003 Association for Computing Machinery)

The earliest analysis of human perception of material appearance, seems to have been Helmholz’s demonstration that the percept of luster was due to stereoscopic luminance conflict.³⁴ However, it is only recently that substantial work has begun on human material perception, and most of it has concentrated on isolated visual properties like gloss, which is an important clue to the state of many materials, for example freshness of fruits and vegetables. A pioneering approach explored the role of image statistics in material appearance,³⁵ and was successful in identifying skewness of the image histogram as a particular correlate of glossiness. However, histograms do not reflect spatial structures of images, for example scrambling pixels leaves the histogram unchanged but completely changes appearance, and it is possible that in some cases geometric information is also required to properly judge the glossiness of a surface.³⁶ In addition, change in organic objects may alter some geometric properties. These problems may explain why histogram skewness was found to be insufficient to predict changes of glossiness in images of vegetable decay.³⁷ Despite these caveats the ability to control material qualities in an image by just manipulating the histogram is invaluable for experimental purposes.

A more observer-based approach showed that apparent gloss of spheres in achromatic images could be well represented in a 2D space by multidimensional scaling of dissimilarities in perceived gloss. The dimensions were then related to perceptual primitives like perceived lightness.³⁸ More direct approaches for discovering perceptual primitives underlying material qualities, have related perceived brightness and contrast to perceived albedo^39,40 and transparency.^41,42 For instance, thin neutral density filters are defined by the physical values of reflectivity and inner-transmittance. Observers found it easy to manipulate either of the physical values to match the perceived transparency of filters differing in both values. At the match points, observers were shown to equate perceived contrast, which turned out to correlate perfectly with total transmittance through the filter.⁴¹ The abstracted quality of perceived transparency is thus 1D in a space that includes perceived contrast as one of the dimensions. However, none of the metrics proposed in the literature were able to predict perceived contrast,⁴¹ which itself may be multidimensional in other spaces. Consistent with this view, a meticulous analysis of translucency,⁴³ combining psychophysics with graphical simulations of subsurface light transport, elucidated the complexity of the percept, showing that translucency was enhanced by highlights, the sign of the correlation between intensity and color saturation, decrease in object size, perceived contrast, blur, and back-lighting. If image statistics are used, they are from selected regions, especially the edges of the object.

One way to go beyond simple image statistics is to exploit the relative success in computer vision of statistics of local oriented multi-scale filters in identifying scenes, objects, and materials.^44-46 However, even when a number of these methods were combined in an optimal learning algorithm and tested on an extensive database, the model performed only about half as well as human observers.²⁹ As was done for the BRDF data, it may be worthwhile forming vectors from the outputs of these filters, reducing them to low-dimensional manifolds, and then using observer ratings of material qualities to estimate functions that covary with subjective impressions. These functions could then be used in image manipulations for psychophysical experiments.

MATERIAL HISTORY AS CUE FOR OBJECT IDENTIFICATION

Research on human perception of material appearance provides a necessary complement to object recognition studies.⁴⁷ Picture the following objects in your mind, one at a time: a coat, a brown coat, a brown plastic raincoat, a brown wool coat, a wet brown plastic raincoat, a wet brown wool coat. Notice that material description can be an inherent part of object identity, hence from the second object on, picturing the coat requires envisioning material qualities, such as the softness of wool versus the relative stiffness of plastic, and the difference in appearance of more versus less absorbent wet materials. Material identification is thus an integral part of object perception, especially when objects have to be used for particular purposes like warmth or waterproofing. Deciphering the physical cues that signal qualities and states of materials, and understanding how these cues are acquired and processed by the visual system, will require combinations of computational, psychophysical, and neural techniques. Given the newness of the field, it is premature to aim for a consensus about areas and methods of study, so this review has highlighted the diversity of work on this topic.

Appearance related investigations can use analytical strategies that try to identify the roles of individual factors by subtracting the effects of others, or synthesizing strategies that try to identify the effects of factors by adding them on top of other factors. Figure 8 demonstrates an analytic approach that removes the effects of object identity and shapes to isolate perceptual factors. However, removing 3D and motion information and using artificial lighting conditions may be removing critical clues, since object shape can influence material inferences,^48,49 motion of reflections is a critical cue to shininess,²⁴ and natural lighting provides useful variations.⁵⁰ A clever synthetic strategy for judging material identity of properties, is to use different objects made of the same materials or similar objects made of different materials²⁹ (Figure 11). An interesting informal observation, that will need to be tested, is that in some cases where materials are used in ‘un-natural’ ways, for example, when paper is made stiff enough to form a chair, or a plaster cast is made to look like a soft robe, observers are often able to identify both the material and the artifice (Figure 12).

FIGURE 11.

(a) Similar objects made of different materials. (b) Different objects made of similar materials.

FIGURE 12.

(a) Paper chair; (b) plaster robe.

This review introduces material change as an object of study in human perception and cognition. A primary motivation is that the visual traces of changes are integral components of material and object identity. The material of a shiny sphere is more likely to be recognizable if the sphere has a history embodied as scratches, dents, etc. because paint would show chipping, plastic would show scratches, and metal would show dents. For example, it is difficult to judge if the three shiny spheres in Figure 13 are glass or metal, wood or stone, and plastic or mud from left to right. (Note: they are metal, wood, and mud, respectively) On the other hand, the three weathered spheres in Figure 14 are easy to discern as metal, wood, and mud because of tarnish, chipping, and cracking, respectively.

FIGURE 13.

What are the materials of these shiny spheres? (a) Glass or metal? (b) Wood or stone? (c) Plastic or mud?

FIGURE 14.

What are the materials of these older spheres? (a) Glass or metal? (b) Wood or stone? (c) Plastic or mud?

In understanding the visual parsing of material history, physics-based models are likely to be useful, as the visual system has been shown to exploit physics of situations for many different perceptual inferences.^39,41,51,52 The perceptual system, however, can have its own internal logic independent of physical laws. As discussed earlier, material change ‘metamers’ arise naturally when different physical processes generate similar images. Consequently, when image changes are correlated with physical models, multiple models will have to be considered in many circumstances. On one hand, observers can perceive transparency and translucency in conditions that are physically incompatible with the phenomena,^43,53 especially in images with isoluminant colors, and on the other, observers do not always use available information that would simplify an identification task.^7,52,54

Possibly the most important quality of visually based judgments of materials, is that they share the property of propensity with mental inferences like judging the personality of a person. For example, surmising properties such as soft, stiff, brittle, dull, rancid, sticky, or slippery from visual information, in essence makes predictions for visual appearances in other states of the material, and for the outputs of other sensory modalities and motor actions applied to the material. For example, a rectangular sheet of cloth stretched on a level surface could have a similar geometrical shape to a thin slab of limestone. If sufficient image clues are present about the identity of the materials, observers would accept the image in Figure 15(a) as a probable alternate state of the material, based on the usual pliability of cloth. Based on usual inferences about the rigidity of stone, however, observers are likely to evidence considerable surprise at the ‘folded’ forms in Figure 15(b). The limestone ‘draperies’ are surprising, because it is difficult to imagine spontaneous processes of dripping and erosion that lead to similar forms as folding. Similarly, observers assume that the folds in Figure 15(c) imply that the material is flexible and other folds are equally probable, so it is surprising to realize through touch that the cloth is as stiff as concrete and the folds permanent. These ‘surprises’ suggest that conscious or unconscious visual imagery plays a role in material inferences from object shapes.⁵⁵ By measuring the degree of expectancy from one state of material to another, the property of propensity can be exploited to decouple visual inferences of material qualities from verbal and semantic connotations, and link them to physical operations.