Human flavor perception: Application of information integration theory

Abstract

The perception of flavor arises from the combination of inputs from several sensory modalities, especially gustation (taste proper) and olfaction (the primary source of flavor qualities). Both the perception of intensity of suprathreshold flavorants and, notably, the detection of weak flavorants are consistent with a rule of additivity. Thus, the detectability, d′, of mixtures of the gustatory flavorant sucrose and the olfactory flavorant vanillin approximates the additive sum of detectabilities of the two components, within a model that assumes pooled noise in the flavor system that derives from both modalities. When gustatory and olfactory flavorants are presented in isolation, however, under conditions that encourage or permit selective attention to one modality or the other, it may be possible to filter out the noise associated with the unattended modality, and leading thereby to a rule of vector summation.

The perception of flavor provides a superb example of multisensory processing. Foods and beverages taken into the mouth produce flavor percepts that commonly reflect the integration of, and perhaps also interactions among, outputs from at least three different sensory channels: gustation (or taste proper), olfaction, and somatosensation.

The lion's share of flavor quality comes typically from the olfactory sense, stimulated when air-borne molecules pass from the mouth, retronasally through the nasopharynx, to the olfactory mucosa. The gustatory sense – taste proper – contributes when molecules stimulate receptors in the tongue and oral cavity, providing the qualities of sweet, sour, salty, bitter, and savory or umami (characterized by the taste of monosodium glutamate, or MSG). And the somatosensory system contributes to flavor perception in several ways: through proprioceptors in the jaw and mechanoreceptors in the oral cavity, which give information about texture; through warm, cool, and heat receptors in the tongue and oral cavity, which give information about temperature; and through nociceptors, which mediate pungency and spiciness. Further, the sight of food (e.g., its color) and the sounds produced while chewing may also contribute to the overall perception of flavor (e.g., Koza, Cilmi, Dolese, & Zellner, 2005).

How does the perception of flavor, and in particular the ability to detect weak flavorants and to perceive the intensity of stronger ones, depend on their multisensory components? Research on flavor perception has emphasized unisensory rather than multisensory integration, mainly in the gustatory system (see McBride & Anderson, 1990). Multisensory research has focused on the integration of flavor information from the gustatory and olfactory modalities, and several lines of evidence converge in support of the hypothesis that information about multisensory flavor intensity often combines additively – that when gustatory and olfactory signals merge to produce flavor percepts, the detectability or perceptual magnitude of the gustatory and olfactory components combines linearly.

Given a flavor mixture containing a gustatory component and an olfactory component, we may write a general linear equation for intensity processing as:

$$I_F=I_g+I_o+N_{g,o}$$ (1)

where I_F is the overall intensity signal of the flavor, I_g and I_o are the intensity signals produced by the gustatory and olfactory stimulus components, respectively, incorporating any weighting coefficients, and N_g,o is the internal noise in the flavor system. In principle, Equation 1 could apply both to the detectability of very weak gustatory-olfactory flavors and to the perception of intensity of suprathreshold flavors.

Additivity in the perception of suprathreshold flavor intensity

Evidence so far of additivity in flavor perception comes largely from studies of suprathreshold gustatory-olfactory mixtures: Murphy, Cain, and Bartoshuk (1977), Murphy and Cain (1980), McBride and Anderson (1990; see also McBride, 1993), and Cerf-Ducastel and Murphy (2004) all reported findings that are reasonably consistent with linear additivity; but see, however, Garcia-Medina (1981) and Hornung & Enns (1986). Anderson's (1982) approach to information integration provides an appropriate analytic framework: In each study, subjects rated flavor mixtures constructed by combining each of n concentrations of a gustatory flavorant with each of m concentrations of an olfactory flavorant, producing factorial plots that were reasonably parallel.

Two features of these findings are noteworthy. First, in all three studies, the concentration series of gustatory and olfactory flavorants included values of “zero”, so the stimulus matrix included pure water (zero-zero combination), to which, on average, the subjects gave substantial non-zero ratings. Presumably, the ratings of the flavor intensity of water provide at least a rough measure of the psychological magnitude of internal noise, N_g,o. And second, in every case, the mean ratings of the gustatory components were substantially greater than those of the olfactory components. Importantly, additivity emerged despite the disparities in mean perceived intensity, which are known to produce strong contextual effects (e.g., Rankin & Marks, 2000) and which presumably varied in degree across the studies. By implication, stimulus context may differentially affect the weightings of the gustatory and olfactory components without affecting the principle of additivity. Recently, we confirmed this implication by asking subjects to rate the perceived flavor intensity of mixtures of sucrose (a sweet gustatory flavorant) and citral (a lemon-like olfactory flavorant) in different contextual conditions: In one condition the concentrations of citral were relatively low and those of sucrose high, while in another condition the concentrations of citral were high and those of sucrose low. An information-integration analysis suggests additivity in each condition, with context exerting adaptation-like effects on scale values or weightings (Marks, Burger, & Chakwin, 2007).

Additivity in the detection of weak flavorants

Additivity also characterizes the detection of mixtures of weak gustatory and olfactory flavorants. The detectability of flavorants is ultimately limited by the magnitude of the sensory noise in the flavor system – that is, by N_g,o. In applying additive models to the detection of weak flavorants, we follow the spirit of Anderson's (1982) application of information integration to Receiver Operating Characteristics (ROCs) by obtaining measures of output such as hits, false alarms, or d′. Because d′ equals the ratio of each intensity signal, I, to the noise, N Equation 1 implies that the detect-abilities of the gustatory component, $d_g^{\prime }$, the olfactory component, d′_o and their mixture, $d_F^{\prime }=d_{g,o}^{\prime }$, are given, respectively, by

$$d_g^{\prime }=I_g∕N_{g,o}$$ (2a)

$$d_o^{\prime }=I_o∕N_{g,o}$$ (2b)

$$d_F^{\prime }=(I_g+I_o)∕N_{g,o}=d_g^{\prime }+d_o^{\prime }$$ (2c)

To test the additivity implied by Equation 2c it is useful to obtain, for each subject, a measure of d′ for several combinations of gustatory and olfactory concentration within a factorial design. But this is an exceedingly time-consuming process, especially in the chemical senses, where subjects must rinse thoroughly between stimuli and where stimuli are presented at a relative slow pace, around two per minute. Such considerations constrain the number of possible stimulus presentations within a single experimental session. As a practical matter, rather than use a cumbersome yes/no design to obtain hits, false alarms, and ROCs for each subject, we chose instead the much more efficient two-alternative forced choice (2AFC) method, where percentage correct can be converted directly to d′ (Macmillan & Creelman, 2005). In 2AFC, on each trial the subject tastes two samples of solution in succession (rinsing before each). One sample has only water, the other water plus a flavorant, and the subject must indicate which of the two samples had the flavorant. Initial tests of the additive model examined integration of the gustatory flavorant sucrose with the olfactory flavorant vanillin (preliminary reports appear in Elgart & Marks, 2006; Marks, Elgart, & Ashkenazi, 2006).

Figure 1 shows the factorial plot of average data obtained from eight subjects tested repeatedly, over a period of several months, on a 3 × 3 matrix representing mixtures of sucrose and vanillin. The data are reasonably consistent with additivity. The interaction term is not significant [F(2, 14) = 1.4] – an outcome commensurate with findings on perceived intensity of suprathreshold flavor mixtures already discussed. Taken together with the suprathreshold data, the present results paint a relatively simple picture of flavor processing, in which the intensity signals from the gustatory and olfactory channels combine linearly, presumably in a central neural region such as orbitofrontal cortex.

Figure 1.

Results of Experiment 1. The detectability (d′) of nine flavor mixtures constructed by combining three concentrations each of the gustatory flavorant sucrose and the olfactory flavorant vanillin. The concentrations of sucrose, S1, S2, and S3, and the concentrations of vanillin, V1, V2, and V3, were determined individually for each subject to produce 65%, 75%, and 85% correct responses in a sequential two-alternative forced choice. On each trial the subject chooses the observation interval that contained a flavorant.

Additivity versus summation: Role of selective attention

As pleasing as these simple results may be, data reported by Ashkenzi and Marks (2004) suggest a complication to the story. Ashkenazi and Marks used a 2AFC method to ask, under a variety of conditions, how well subjects can attend selectively to either the gustatory or the olfactory component of sucrose-vanillin mixtures. If there is no “cross-talk” between the gustatory and olfactory channels, and if people can attend selectively to signals on either modality, then the ability to detect a weak gustatory or olfactory flavorant should be unaffected by the addition of a flavorant in the other channel. When attending to sucrose, for example, subjects should detect sucrose just as well with and without the presence of vanillin. Although that study did not aim to investigate additivity per se, baseline conditions in those experiments provide data that are pertinent, in that the subjects were asked, in different sessions, to detect sucrose alone, vanillin alone, or sucrose-vanillin mixtures.

Importantly, the mixtures that Ashkenazi and Marks (2004) tested were not constructed by factorial combination of concentrations. Instead, in the summation design, the experimenters combined equipotent concentrations of sucrose and vanillin – concentrations that gave equivalent levels of detectability (equal values of d′). Thus, the results gave measures of forced-choice detectability of six concentrations of sucrose, six concentrations of vanillin, and six sucrose-vanillin mixtures, each set of six measured in separate block of trials. Different kinds of stimuli were not intermixed within a session.

Figure 2 replots results obtained by Ashkenazi and Marks (2004). The figure gives the detectability (d′, calculated from percentage correct in 2AFC) of each of the six sucrose-vanillin mixtures as a function of the average detectability (d′) of the sucrose and the vanillin presented alone. In a simple additive (summation) model, d′ for the mixture should be twice the average d′ of the components. The data of Figure 2 deviate markedly from a model of simple summation given by Equation 2c, but can be fitted well by a vector equation of the form

$$d_F^{\prime }=\sqrt{d_g^{\prime 2}+d_o^{\prime 2}}$$ (3)

Figure 2.

Detectability (d′) of mixtures of sucrose and vanillin as a function of the average detectability of the components. At each level, the two flavor components were roughly equal in detectability. The straight line gives the prediction of the vector model: the detectability of the mixture is 1.414 (square root of 2) times the detectability of the unmixed components. Data from Ashkenazi and Marks (2004).

This outcome is consistent with various models (Green & Swets, 1966; Fidell, 1970) proposed to account for the integration of information from independent channels, under the assumption that not only are the channels (here, gustation and olfaction) stochastically independent (uncorrelated noise in the two channels), but also under appropriate conditions the subjects can attend selectively to signals in each channel – or, equivalently, that the subjects may optionally attend to either sensory modality or both.

With stochastically independent sources of noise in the two channels (modalities), we may decompose overall noise as

$$N_{g,o}^2=N_g^2+N_o^2$$ (4)

where $N_g^2$ and $N_o^2$ are the variances of the gustatory and olfactory components, respectively. If subjects attend selectively to the gustatory and olfactory channels when detecting unmixed flavorants, then the detectability of sucrose is limited by noise N_g and the detectability of vanillin by noise N_o, while the detectability of gustatory-olfactory flavorants is limited by noise N_g.o. Under these circumstances, when the gustatory and olfactory flavorants are matched in detectability, d′_g = d′_o, as in Ashkenazi and Marks (2004) Equation 3 follows from Equation 4. In essence, if subjects are able to “filter out” the noise from the irrelevant channel when detecting unmixed flavorants, then the unmixed flavorants will be more detectable than they will be when subjects attend to both channels. This is essentially the noise-reduction model of selective attention (e.g., Pashler, 1998). See Marks and Wheeler (1998) for an application of the noise-reduction model to selective attention in detecting weak gustatory stimuli.

Assuming that subjects are capable of attending selectively to the gustatory or olfactory channel when detecting weak flavorants, and that the channels are stochastically independent, this line of reasoning leads to several predictions: First, the mathematical rule of flavor summation/addition depends on whether the experimental paradigm encourages attentional selection. Selective attention is possible when unmixed and mixed flavorants are tested in separate blocks of trials, or perhaps when each flavorant is cued on each trial within a mixed series (but see Ashkenazi & Marks, 2004, for limitations). When trials containing unmixed flavorants (gustatory or olfactory) and mixtures of flavorants (gustatory plus olfactory) are interspersed within a single test session, however, subjects are likely to attend throughout the session to both channels – that is, to attend fully rather than selectively. There are costs and benefits associated with attending selectively versus attending fully. Selective attention enhances the ability to detect a particular subset of stimuli, but at the cost of depressed sensitivity to unattended stimuli. Full or pooled attention may maximize the detection of a wide range of possible stimuli, but does not maximize the detection of any particular kind of stimulus. The vector Equation 3 may apply under conditions of selective attention, whereas the additive Equation 1 would apply under conditions of full attention.

We tested these implications by measuring the detectability of sucrose, vanillin, and sucrose-vanillin mixtures in two additional experiments. Interleaved among the experimental sessions of Experiment 1 were sessions of Experiment 2, in which we asked the same eight subjects to detect either pure sucrose (sucrose alone, with no vanillin, V0) or pure vanillin (vanillin alone, with no sucrose, S0). That is, Experiment 2 measured sensitivity to pure sucrose and to pure vanillin in conditions (blocked trials) that encouraged selective attention to each flavorant. By interleaving the sessions of Experiment 2 among those of Experiment 1, we minimized the possibility that the sensitivity of the gustatory or olfactory system might differ systematically across the two experiments. This precaution makes it possible to compare measures of sensitivity obtained with three concentrations each of pure sucrose and pure vanillin (Experiment 2) to measures of sensitivity to sucrose-vanillin mixtures containing the same stimulus concentrations (Experiment 1).

After Experiments 1 and 2 were completed, four of the eight subjects returned to participate in Experiment 3, which contained two conditions. In the first condition, we asked the subjects to detect each of three concentrations of sucrose, which could be unmixed (no vanillin, V0) or mixed with the lowest concentration of vanillin (V1). In the second, complementary, condition we asked the subjects to detect three concentrations of vanillin, which also could be unmixed (no sucrose, S0) or mixed with the lowest concentration of sucrose (S1). Although both Experiment 2 and Experiment 3 expand the design of Experiment 1 to include pure sucrose and pure vanillin, we anticipate that the results of Experiment 3 but not Experiment 2 will conform to the prediction of an additive model. The conditions of Experiment 3 should encourage subjects to attend fully to both channels, gustation and olfaction. The combined gustatory and olfactory noise should limit equivalently the detection of pure sucrose, pure vanillin, and their mixture. Adding a small amount of vanillin to pure sucrose or a small amount of sucrose to pure vanillin should, therefore, increase d′.

The conditions of Experiment 2, however, should encourage subjects to attend selectively to gustation when pure sucrose is presented but to attend selectively to olfaction when pure vanillin is presented. Gustatory noise alone should limit the detection of sucrose, and olfactory noise alone should limit the detection of vanillin. The mixtures to which the resulting measures of d′ are compared, however, are obtained under conditions that encourage full attention, that is, attention to both gustation and olfaction, where the overall noise is greater. In the case of Experiment 2, the vector model (Equation 3) predicts that the detectability of sucrose can actually be greater when subjects detect pure sucrose, attending selectively in Experiment 2, than when they detect the same amount of sucrose to which a small amount of vanillin is added (Experiment 1). Similarly, the model predicts that the detectability of vanillin in Experiment 2 can be greater than the detectability, in Experiment 1, of the same vanillin to which a small amount of vanillin is added.

The results shown in Figure 3 are broadly consistent with these predictions. When trials containing pure sucrose or pure vanillin were interspersed with trials containing mixtures, thereby encouraging subjects to attend to both channels throughout each session, then the addition of weak vanillin or sucrose to pure sucrose or vanillin improved detectability, as the additive model predicts. But when trials of pure sucrose and trials of pure vanillin were blocked, affording subjects the opportunity to attend selectively to whichever modality was stimulated, and thus to avoid the noise inherent in the unstimulated modality, then the detectability of the pure flavorants was relatively greater.

Figure 3.

Detectability (d′) of sucrose and vanillin, separately and mixed. The open symbols in the left panel show the detectability of three sucrose concentrations (S1, S2, S3) (a) presented alone (V0) and blocked, so subjects could attend just to sucrose, and (b) mixed with weak vanillin (V1), so that subjects attended to both sucrose and vanillin. The filled symbols in the left panel show corresponding measures of detectability of three concentrations of unmixed vanillin (V1, V2, V3) (a) presented alone (S0) and blocked so subjects could attend just to vanillin, and (b) mixed with weak sucrose (S1), so that subjects attended to both vanillin and sucrose. The right panel shows the detectability of the stimuli when V0 and V1 trials were intermixed (open symbols) and when S0 and S1 trials were intermixed, so that subjects always attended to both sucrose and vanillin (filled symbols).

Conclusion

The models advanced here make several assumptions about multisensory processing of flavor. In particular, Equation 4 assumes that the two channels are stochastically independent. Given that gustatory and olfactory signals pass through several stages of peripheral processing before combining centrally, it is certainly plausible to assume that noise is independent in the periphery of the two modalities. Functionally, however, the assumption of stochastic independence is at best an approximation, as it implies that there is no noise at all introduced at or beyond the point in flavor processing where the gustatory and olfactory signals merge. That is, the model assumes that all of the noise is “peripheral”, none of it “central”. Central noise could originate in those regions of the brain where multisensory information about flavor is integrated, likely candidates being the insula, the anterior cingulate cortex, and the orbitofrontal cortex (e.g., Small, Voss, Mak, Simmons, Parrish, & Gitelman, 2004).

When food is taken into the mouth, sensory information from three modalities, gustation or taste proper, olfaction, and somatosensation, is integrated into a coherent perception of flavor. To a first approximation, the integration of flavor intensity information from gustation and olfaction can be characterized by a rule of linear addition, at least when subjects attend fully to information on both modalities. The integration of both gustatory and olfactory flavor information with somatosensory information is likely to be more complex. Changing the temperature of a food or beverage delivered to the mouth, for example, not only modifies the thermal information (thermal sensation) but also changes the concentration of olfactory molecules entering the airspace; similarly, changing a food's viscosity not only modifies the perceived texture but also can affect biophysical interactions between gustatory molecules and gustatory receptors. So it may be difficult to manipulate somatosensory information without simultaneously modifying the gustatory and/or olfactory components. While teasing apart all of these processes is challenging, one of the virtues of information integration theory is that it provides a conceptual framework to evaluate the rich complexity of perceptual processing of intensity information.