A brain-derived metric for preferred kinetic stimuli

Abstract

We here address the question of whether there is any correlation between subjective preference for simple configurations within a specific visual domain such as motion and strength of activity in visual areas in which that domain is emphasized. We prepared several distinctive patterns of dots in motion with various characteristics and asked humans to rate them according to their preference, before and while scanning the activity in their brains with functional magnetic resonance imaging. For simplicity, we restricted ourselves to motion in the fronto-parallel plane. Moving patterns produced activity in areas V1, V2, the V3 complex (V3, V3A, V3B) and V5, but only in areas V5, V3A/B and parietal cortex did the preferred kinetic patterns produce stronger activity when compared with the non-preferred ones. In addition, preferred patterns produced activity within field A1 of medial orbito-frontal cortex (mOFC), which is not otherwise activated by kinetic stimuli. Hence, for these areas, stronger neural activity correlated with subjective preference. We conclude that configurations of kinetic stimuli that are subjectively preferred correlate with stronger activity within early visual areas and within mOFC. This opens up the possibility of more detailed studies to relate subjective preferences to strength of activity in early visual areas and to relate activity in them to areas whose activity correlates with the subjective experience of beauty.

2. Introduction

In the work reported here, we address within a specific and restricted context a more general question of whether there are any definable characteristics of stimuli that render them more attractive, or at any rate preferable. The question has of course been theoretically addressed many times before in artistic speculation, though within a much broader context. Characteristics such as harmony, proportion and symmetry have at various times been considered to be attributes of beautiful works, but without a general consensus. This is perhaps not surprising; attributes such as harmony or proportion are difficult to define for all works that are apprehended as beautiful except in terms of the perceiver. Even the extent to which easily definable properties such as symmetry or proportion, at least for visual objects, are characteristic of beautiful works has been much debated [1]. Within vision, what constitutes proportion or symmetry in one category of visual stimuli (e.g. objects) cannot be easily translated to other attributes (e.g. colour or motion). One way around this difficulty is to concentrate on a single visual attribute, such as visual motion, and enquire whether there are any characteristics or configurations that, for human observers, make some kinetic patterns preferable to others and, if so, whether we can account for this preference in neural terms. Basing ourselves on the functional specialization of the visual brain for different visual attributes [2–4], among which is a specialization for visual motion [5–7], we asked whether there are any particular patterns of dots in motion that stimulate visual areas known to contain directionally selective cells preferentially. Of these, the V5 complex (MT+) is the most prominent, although other areas, such as those comprising the V3 complex (V3, V3A and V3B), which are also dominated by a magnocellular input [3,8], have been shown to contain directionally selective cells [9] or to be responsive to motion in human functional magnetic resonance imaging (fMRI) experiments, though less robustly than V5 [10–17].

3. Material and methods

3.1. Ethics statement

Informed written consent was obtained from all participants and the University College London (UCL) Research Ethics Committee approved the study.

3.2. Subjects

Nineteen healthy subjects (10 males; minimum age 21, maximum age 56, mean age 32) were recruited through advertisements and from the UCL psychology subject pool; three of them were excluded after being scanned because their rating data had not been correctly recorded. None of our participants was an artist and all had normal or corrected-to-normal vision.

3.3. Stimuli

Stimuli were generated using Processing (www.processing.org) and then passed to Cogent 2000 and Cogent Graphics (www.vislab.ucl.ac.uk/cogent.php) for playback. Subjects were shown and asked to rate the stimuli twice: once during a visit to the laboratory one or more days before scanning, when the experiment was also explained to them, and once in the scanner.

In the pre-scanning session subjects sat in a darkened room at a fixed distance from a cathode ray tube computer display. They rated the patterns using a computer keyboard. The responses in this part of the study were on an 8-level scale.

Stimuli consisted of eight patterns of moving white dots on a black background, designed with the expectation that some patterns would be preferred to others. The patterns were generated algorithmically using trigonometric functions or particle systems [18], and were matched for the number of dots and their linear velocities at the five velocity levels used. In the pre-scanning experiments, subjects sat at a distance of 60 cm from the display and each dot of the display subtended 0.5° of visual angle. In the scanner, subjects were positioned 55 cm from the display, which had a width of 31 cm. The actual area used to display the stimuli was adjusted so that each subject was able to see the entire field of dots. Each individual stimulus contained 192 dots. The speed of the dots’ motion in the pre-scanning sessions was varied in five steps, corresponding to 2.86, 5.73, 8.59, 11.46 and 14.32 deg s⁻¹, while speeds in the scanning session varied based on how the stimuli were scaled down for display. The mean dimensions of the area in which the stimuli were shown after individual adjustment were 29 × 23°. A single dot subtended a visual angle of approximately 0.17°, the size reduction being necessary to make the entire stimulus area visible to the subject in the scanner. The stimuli are available for viewing at www.vislab.ucl.ac.uk/kinetic_beauty_movies. We emphasize that the two patterns preferred by the majority of subjects had different characteristics and both differed in their characteristics from a pattern preferred by one of the other subjects (see below and §5).

Two subjects were given two stimulus sessions in the scanner owing to a recording failure (and only data from the second session was used for them); the remaining subjects were given one session. The session began with a 26 s period with no stimulus on the screen. Brain volumes recorded during this period were discarded from subsequent analysis to allow T₁ equilibration effects to subside. The stimulus sequence began after this period. A block design was used to plan the timing of the stimuli. The patterns were shown in 16 s epochs followed by a 5 s inter-stimulus interval during which the subject was asked via a text prompt to rate the preceding pattern in terms of preference, on a scale of 1 (least preferred) to 4 (most preferred), by pressing a key. Subjects were not given any guidance as to what aspects of the stimuli they should base their ratings on. It was made clear to them that they should rate each stimulus on its own merits rather than relative to the other stimuli and that if they did not have a strong preference for any of the stimuli, they should rate them all neutrally. Once this was done, the screen turned to a mid-grey until the onset of the next epoch. In total, there were 45 epochs, each pattern was shown five times at each of the different speeds (see above). Additionally, there were five epochs during which a static arrangement of dots was shown to provide a baseline of activity for the subsequent analysis. The patterns were ordered using a pseudo-random system which ensured that there were no occasions on which the same pattern was shown in two adjacent epochs.

3.4. Scanning details

Scanning was done in a 1.5 T Siemens Magneton Sonata MRI scanner fitted with a head volume coil (Siemens, Erlangen, Germany) to which an angled mirror was attached, allowing subjects to view a screen onto which stimuli were projected using a liquid crystal display projector.

An echo-planar imaging sequence was applied for functional scans, measuring blood oxygen level-dependent (BOLD) signals (echo time TE = 50 ms, repeat time TR = 90 ms, volume time = 4.32 s). Each brain image was acquired in a descending sequence comprising 48 axial slices, each 2 mm thick, with an interstitial gap of 1 mm and a voxel resolution of 3 mm, covering nearly the whole brain. After functional scanning had been completed, a T₁ modified driven equilibrium Fourier transform anatomical scan was performed in the sagittal plane to obtain a high-resolution structural image (176 slices per volume, constant isotropic resolution of 1 mm, TE = 3.56 s, TR = 12.24 s). During scanning, subjects’ eye gaze position, heart rate and respiration were recorded.

3.5. Analysis

Data were prepared for analysis in SPM5 [19] using the procedure described by Zeki & Romaya [20]. The onsets and durations of the patterns were modelled as boxcar functions. Head movement parameters calculated from the realignment pre-processing step were included in the model as regressors of no interest. Stimulus functions were convolved with the default SPM5 canonical haemodynamic response function and entered into a linear convolution model (for each subject). Impulse functions convolved with the haemodynamic response were added to the generalized linear model to account for activity related to keypresses. Speed of motion was included as a parametric modulator of no interest in the models. The ratings given by subjects during scanning were included as a modulator. Maximum-likelihood estimates of the associated parameters were then taken to the second (between-subject) level for random effects inference, using the summary statistic approach [21]. This involved taking contrasts or mixtures of parameter estimates summarizing condition-specific effects in each subject and creating statistical parametric maps of unpaired t-statistics.

The following contrasts were generated: voxels where viewing a moving pattern produced a greater BOLD response than a stationary one; voxels with a BOLD response proportional to the rating given in the scanner; voxels with a greater BOLD response for patterns rated 4 than for patterns rated 1 in the scanner.

4. Results

4.1. Determination of the preferred characteristics of the kinetic stimuli from ratings provided

Subjects rated stimuli according to their preferences on a scale of 1–8 during a pre-scanning visit and 1–4 while in the scanner. The rating was given immediately after each stimulus was viewed. Patterns 2 or 5 were the first preference for 13 out of 16 subjects and the second preference for 15 out of 16; hence they were significantly more preferred by subjects (table 1). There were two departures from these preferences, which are significant for the interpretation of results: one subject had a preference for pattern 4, which was not preferred by the others, while another subject did not have a marked preference for any of the patterns. Patterns 2 and 5 differed from one another in smoothness of motion, uniformity of dot distribution and coherence of motion. Pattern 4 also differed from the other patterns, including patterns 2 and 5, on these measures (see electronic supplementary material for a full table of pattern characteristics and an explanation of their calculation).

Table 1.

The eight patterns and the mean rating (from 1–4; 4 being the most preferred) given to each by subjects when responding in the scanner (n = 75).

pattern	mean rating	s.d.
1	1.9	0.75
2	3.4	0.80
3	1.9	0.86
4	2.6	0.67
5	3.2	0.84
6	1.8	0.94
7	2.4	0.81
8	1.9	0.82

4.2. Imaging study

In table 2 we report activations that were significant at p < 0.05 for family-wise error rate (FWE), with a Bonferroni correction for multiple comparisons and trend-significant activations at p_FWE < 0.1, corrected (in italics). We also report one small volume correction (SVC) activation based on previous results.

Table 2.

Activation sites. Activations shown are significant at p_FWE < 0.05 or p < 0.1 (in italics) after Bonferroni correction for multiple comparisons. k_E is the cluster size in voxels. Coordinates are given in MNI space.

contrast	brain areas	pFWE	kE	X	Y	Z
motion > static	calcarine sulcus and surroundings, dorsal and ventral V2 and V3	0	572	21	−96	12
	lV5	0	103	−36	−60	9
	rV5	0	129	48	−57	3
modulated with rating	lV5	0	193	−51	−69	3
	rV3B	0	76	36	−87	−6
	left parietal cortex	0.009	44	−33	−36	45
	right superior parietal lobule	0	130	21	−57	63
	rV5	0.006	48	48	−78	3
	left superior parietal lobule	0	94	−18	−57	63
	right superior frontal sulcus	0.074	27	21	−6	54
liked > disliked (rated 4 > rated 1)	ventral rV3B	0.002	51	36	−84	−6
	lV5	0.001	54	−48	−75	3
	left occipital/parietal	0.096	22	−21	−90	30

4.2.1. Activation of visual motion-related cortex

As expected, the contrast moving dots > stationary dots led to strong activity in the V5 complex [22,23] and also to a large area of activation in the occipital lobe, which includes the upper and lower lips of the calcarine sulcus (corresponding to area V1), as well as dorsal and ventral V2; the activation also included areas of the V3 complex (V3, V3A and V3B; table 2 and figure 1a). All these areas are known to have directionally selective cells [9,24] or to be responsive to motion [25–27]. There was no significant activation of parietal cortex or of medial orbito-frontal cortex (mOFC). There was a trend-significant activation in the right superior frontal sulcus.

Figure 1.

Activation sites. (a) Contrast motion > static (background threshold p_uncorr < 0.0001, cluster threshold k_E > 0, horizontal section at z = 5). (b) Visual cortical areas at which activity was parametrically modulated by rating (background threshold p_uncorr < 0.001, cluster threshold k_E > 0, horizontal section at z = 3). (c) Cortical areas from the contrast patterns rated 1 > patterns rated 4 (background threshold p_uncorr < 0.001, cluster threshold k_E > 0, horizontal section at z = 0). (d) Parietal cortex activations that correlated with rating (as in (b); background threshold p_uncorr < 0.001, cluster threshold k_E > 0, horizontal section at z = 63).

4.2.2. Parametrically modulated response

A parametric analysis of the relationship between the BOLD signal and the declared preferences for the moving stimuli showed that cortical activity was positively correlated with subjective preference within the right and left areas V5 and right V3A/B, which are not readily distinguishable from one another even in retinotopic studies (they correspond to LO1 and LO2 in the terminology of Larsson & Heeger [16]; table 2 and figure 1c) and bilaterally within the parietal cortex (figure 1d), which is also responsive to motion [28,29] and has been implicated in perceptual organization generated by motion or other cues [30,31]. It also led to activation of field A1 of mOFC ((−15, 52, −12) Montreal Neurological Institute (MNI) space with a peak level significance (FWE corrected) of 0.034) when we used an SVC of 18 mm radius, centred on the coordinates ((−3, 41, –8) MNI space) from a previous study of beauty [25]. Speed of motion was not related to the preference expressed. There were no significant activations for second order or higher expansions of the rating.

A parametric analysis for the one subject who preferred pattern 4 to patterns 2 and 5 showed that cortical activity correlated positively with subjective preference within his V5 and within the right V3A/B, but not in the parietal cortex. For the subject who had rated most patterns equally, there were no areas in which strength of activity was proportional to preference, although the pattern of activity in this subject's brain was similar to that of other subjects in the contrast motion > static.

4.2.3. Preferred > non-preferred (4 > 1)

A contrast of preferred more than non-preferred led to the activation of left V5 and right V3A/B, and trend-significant activation in left occipito-parietal cortex (table 2 and figure 1c).

We could not detect any significant activations in the contrast non-preferred more than preferred.

5. Discussion

Our purpose in this study was to begin an enquiry into the relationship between declared (subjective) preferences for simple visual stimuli on the one hand and brain activation on the other, concentrating specifically on early visual areas. The functional specialization of the visual brain [2–4] allowed us to restrict our enquiry to one domain, that of visual motion. Although any visual or indeed cortical area in which strength of activity correlated with strength of subjective preference would have been of interest, we were especially interested in areas containing directionally selective cells or ones that, in the human, respond strongly to visual motion stimulation. Of these, the most prominent is a set of motion-sensitive visual areas comprising V5 and its satellites (the V5 complex or MT+ [26,27]) and other visual areas such as V3, V3A and V3B, which, though having lower concentrations of directionally selective cells, are nevertheless dominated by a magnocellular input and give robust responses to visual motion stimulation [13,32–34]. As our sole concern was whether it is possible to relate the strength of cortical activity in early visual areas with subjective preference, we did not think it useful, for the present study, to try and subdivide the active areas further with retinotopic mapping or to search for subdivisions or groupings within them for which responses correlate with preference, although both approaches will be of interest for future studies.

5.1. Strength of activation and motion-sensitive areas of the cortex

Even though directionally selective cells are most prominently concentrated in area V5 and its satellites (the V5 complex), several other visual areas have been described as containing them. These include areas V1, V2, V3, V3A and V3B, all of which have been responsive to stimulation with visual motion in human fMRI experiments, to varying degrees [12]. Of these, it is area V5 that has been the most extensively studied, with results that are now generally agreed upon. Chief among these are that its cells are overwhelmingly responsive to motion, that the great majority are directionally selective and that most are indifferent to both the colour and the form of the stimulus [5,8,35]. Indeed, many respond optimally to a spot or spots moving in the appropriate direction. It is almost certain that the activity of neurons in V5 is at the basis of the perception of motion in both monkey and human [22,36,37], with the behavioural, psychometric functions and the physiological, neurometric ones from V5 cells in monkeys being almost identical [38]. These characteristics make it relatively easy to prepare visual stimuli that activate V5 strongly. In this study, we opted for patterns of dots in motion in the fronto-parallel plane, known to activate human V5 [30,32], with patterns of incoherently moving dots being more potent activators than a pattern of coherently moving ones [32,39], presumably because more directionally selective cells are activated with incoherently moving patterns.

The areas that were prominently active in the contrast motion more than static were not identical to those in which there was relationship between BOLD signal and preference. In particular, areas V1 and V2 showed no activity related to preference. Activity related to preference was seen only in the V3 complex, V5, parietal cortex and field A1 of mOFC (neither of the latter two areas having shown activity in the contrast motion > static). While the directionally selective cells of V1 respond to component motion of stimuli and those of V5 to their direction of motion [40], the directionally selective cells of areas V2 and V3 have been rather less extensively studied. Area V3 is less directionally selective than V5 in both monkey and human [9,13], but has been found to be responsive to visual motion stimulation in several human studies. Areas V3A and V3B (which are not easily distinguishable from one another even with retinotopic mapping [16,41]) are located dorsally in the brain but represent both upper and lower quadrants [42], also contain directionally selective cells [9] and have been reported, in the human, to be more responsive to motion than V3, though not as responsive as V5 [10,13]. Cells responsive to more complex types of motion, including optic flow patterns, have also been described in both monkey [43,44] and human [28,45] parietal cortex. The latter cortex has in fact been subdivided on the basis of the preferred type of motion [28], but, in the absence of detailed retinotopic mapping, we cannot be certain of which subdivision to allocate our activity to within the parietal cortex. On the basis of MNI coordinates, we would place it as lying closest to IPS3.

That areas outside of V3A and V3B should have lower concentrations of directionally selective cells and be less responsive to simple planar motion than V5 implies, of course, that the preference for particular kinetic configurations may be dictated by a relatively small proportion of cells within it. This would not be surprising. Based on their studies of V5, Shadlen & Newsome [46] estimated that 100 neurons may be the fundamental signalling units of the cortex. This, and the fact that parts of the V5 complex may be more responsive to coherent and others to incoherent motion [23], points to the need for more detailed future studies using techniques such as multi-voxel pattern analysis, which would allow us to determine whether such cells form groupings within areas V5 and V3A/B.

It is difficult to predict from what little is known about the characteristics of motion-selective cells in the activated areas (apart from V5) if there are any particular patterns of moving dots, between coherently moving ones at one end and incoherently moving ones at the other, that are more effective in activating areas containing directionally selective cells than other patterns and, if so, whether they are also the ones that are preferred by human subjects. The notion that such cortical areas might have evolved in response to, and in preference for, particular patterns of motion such as optical flow or biological motion is plausible; this in turn makes it plausible to suppose that the preferred patterns would not only evoke greater activity in V5 and other cortical areas with directionally selective cells, but also lead to the patterns themselves being preferred subjectively.

5.2. Preferred kinetic patterns and physical characteristics

In this study, we have shown that, of the visual areas that contain directionally selective cells and respond strongly to visual motion, certain kinetic patterns with definable characteristics, and which are subjectively preferred, evoke more powerful activity only in V5, the areas of the V3 complex and in the parietal cortex, compared with patterns with other definable characteristics or ones lacking them. There are two important issues to address in this context: one is that the recorded subjective preference and the observed accompanying stronger cortical activation in the V3 complex, in V5 and in the parietal cortex are related to preference and not to physical characteristics of the stimulus. The subject who preferred pattern 4 and thus differed from the majority also showed a correlation of cortical activity with subjective preference, even though his preferred kinetic pattern had different physical characteristics from the ones preferred by the majority. As well, another subject who had rated all the stimuli, with their different characteristics, equally had no parametric relation between cortical activity and declared preference. Although all but two of the patterns had some element of grouping of the kinetic dots, which may have suggested objects to some subjects, these groupings differed in size and the extent of the screen occupied, and yet subjects preferred some groupings over others, and one subject preferred pattern 4, which did not exhibit groupings. This makes it very unlikely that grouping dictated preference. All this fortifies the conclusion reached from the other results here, namely that the relationship we observed is indeed between declared subjective preference and cortical activity, rather than with specified characteristics, although of course each subject preferred stimuli with definable characteristics.

5.3. The role of attention

It is worth considering next the extent to which our results could be accounted for by attention. Attentional load enhances the strength of activity in V3 and V5 [47,48]. Yet this enhancement is always accompanied by a pattern of cortical activity that includes not only parietal but frontal cortex as well [49–53], and hence a pattern of cortical activity that was not observed in our study. Nor was there activity in V1 or V2, the activity in both of which is modulated by attention [54,55]. Evidence suggests that attentional mechanisms in the parietal cortex are stimulus-driven, whereas the frontal cortex exerts a top-down attentional influence [56,57]. Because the activity we observed is restricted to the parietal cortex, we assume that it is only stimulus-driven. The difficulty in separating attention and preference in a stimulus-driven context, in addition to the fact that all our stimuli had the same number of dots moving at the same speeds and covering the same extent of the field of view, makes it unlikely that the results we observed are owing to attention towards spatial location or to top-down attentional influences from the frontal cortex.

It is to be noted that we did not ask subjects to rate the stimuli according to how beautiful they felt they were; the kind of simple kinetic stimuli that we used are not obviously characterized as beautiful by many, even though kinetic art that remotely echoes our stimuli (e.g. the mobiles of Alexander Calder where movement is emphasized and form and colour are de-emphasized) has given aesthetic satisfaction to writers and poets [58]. For this study, preference seemed to us to be a better and more secure guide of subjective satisfaction, although we are aware that a study based on preference may end up as a prelude to studying those characteristics in visual stimuli that are experienced as beautiful (see below). It is thus encouraging to note that there was activity in field A1 of mOFC [25], even though it could only be elicited with the use of an SVC based on our previous study of this field. Activity in mOFC has been reported to correlate with the experience of beauty, but no study has ever reported it to respond to moving stimuli, leading us to conclude that the activity that we report there is related to preference alone.

It is evident that there is potentially an infinite number of kinetic stimuli that we could have prepared and some could have led to even stronger preferences, perhaps leading even to aesthetic preferences, and stronger activity than what we report. This, however, would have been a very demanding task and would not, in any way, have improved upon the conclusion that we reach here, namely that some kinetic patterns are preferred over others, that there is no constant characteristic of the stimuli that makes them preferable to all subjects, and that preferred kinetic stimuli lead to stronger activity within the motion-sensitive area V5, within the V3 complex of areas and in the parietal cortex. As well, we are aware that variations in the part of the visual field in which the stimuli are presented, as well as variations in the density and duration of dots, could have led to variations in preference, as they do in discrimination thresholds [59]. But to make the study more manageable, we instead opted for a limited number of patterns, and used these to detect preferences and to correlate preferences with the strength of activity in early visual areas.

The results given here edge us closer to understanding the relationship between identifiable characteristics of a kinetic visual stimulus and its aesthetic rating. Whether the approach we have used in this study can, with modifications, be used for other types of visual stimulus, in the domains of form and colour, for example, remains to be seen. For the moment, we restrict ourselves to the conclusion that characteristics of kinetic stimuli can be identified that, when part of the kinetic work, make it preferable over others lacking these characteristics, and that the preference for these stimuli correlates with their capacity to elicit a stronger response from visual areas that are strongly activated by moving visual stimuli as well as from the field A1 of mOFC.