“What” and “how”: Evidence for the dissociation of object knowledge and mechanical problem-solving skills in the human brain

Abstract

Patients with profound semantic deterioration resulting from temporal lobe atrophy have been reported to use many real objects appropriately. Does this preserved ability reflect (i) a separate component of the conceptual knowledge system (“action semantics”) or (ii) the operation of a system that is independent of conceptual knowledge of specific objects, and rather is responsible for general mechanical problem-solving skills, triggered by object affordances? We contrast the performance of three patients—two with semantic dementia and focal temporal lobe atrophy and the third with corticobasal degeneration and biparietal atrophy—on tests of real object identification and usage, picture-based tests of functional semantic knowledge, and a task requiring selection and use of novel tools. The patient with corticobasal degeneration showed poor novel tool selection and impaired use of real objects, despite near normal semantic knowledge of the same objects’ functions. The patients with semantic dementia had the expected deficit in object identification and functional semantics, but achieved flawless and effortless performance on the novel tool task. Their attempts to use this same mechanical problem-solving ability to deduce (sometimes successfully but often incorrectly) the use of the real objects provide no support for the hypothesis of a separate action-semantic system. Although the temporal lobe system clearly is necessary to identify “what” an object is, we suggest that sensory inputs to a parietal “how” system can trigger the use of objects without reference to object-specific conceptual knowledge.

A major goal of cognitive neuroscience is to understand how knowledge about the world, for both objects and words, is represented and organized in the brain. With regard to object knowledge, suppose that you were handed a familiar object such as a watering can and asked to do two different tasks. In response to “Please tell me all about this object,” you would presumably say something like “It’s a watering can, it’s made of metal, you put water in it and then use it to pour water onto plants,” etc. In response to “Please demonstrate how this object is used,” you would presumably grasp it by the handle and tip it sideways to gesture how you would water something. Evidence from neurological patients suggests that these two aspects of object knowledge may be independently vulnerable to focal brain disease or injury. That is, one class of patients may succeed in producing a normal conceptual explanation of a watering can, but be unable to organize appropriate actions to and with it, despite having no significant sensory impairment or limb weakness. Conversely, it is claimed that another class of patients, who fail to name familiar objects, fail to produce correct semantic descriptions of them, and even fail to perform nonverbal tests of semantic knowledge (such as grouping objects from the same semantic category in a picture-sorting test), can succeed well in using these same objects appropriately.

This dissociation might be taken as evidence for two separable subsystems of learned representations for familiar objects: object semantics vs. action semantics. According to this viewpoint, the apparently normal usage of objects observed in patients with a profound loss of general semantic knowledge would reflect the fractionation of a “multimodal semantic system” of the type first advocated by McCarthy and Warrington (1). A very similar position was adopted by Heilman and colleagues (2–4) to explain the apparently specific loss of knowledge of tool action and usage (ideational or conceptional apraxia) in patients with vascular lesions or Alzheimer’s disease. In a more recent formulation of a multimodal framework, Lauro-Grotto et al. (5) proposed that object-specific action procedures constitute part of the visual semantic system preferentially accessed by objects and pictures. This position would predict a close correlation between appropriate object usage and performance on tests of visual knowledge (matching an object to its recipient, location, and other objects of related usage, etc.).

An alternative view is that the apparent selective preservation of action semantics in the face of degraded general conceptual knowledge of the relevant objects might be explicable in terms of a rather different neural system responsible for general mechanical abilities, which can be triggered by the visual and/or tactile properties of an object. On this view, it often would be possible to demonstrate appropriate responses even to previously unfamiliar objects by general reasoning about what their physical properties “afford” in the way of action (6). Damage to this system, which is linked to dorsal-visual processing (7), has been proposed as the cause of apraxia in some patients with parietal lobe damage. Goldenberg and Hagmann (8) have suggested that such mechanical problem solving, or “direct inference of function from structure” (ref. 8, p. 581), is independent of frontal lobe executive function and is not, therefore, simply a component of general problem-solving capability. Based on observations of a patient with a profound loss of semantic knowledge because of temporal lobe damage after herpes encephalitis, who was asked to gesture in response to visually presented objects, Sirigu et al. (9) likewise postulated the existence of a parietal lobe-based system of nonsemantic, sensorimotor representations or schemas that may be triggered by object affordances.

If preserved mechanical problem-solving abilities triggered by affordances can explain the observed instances of normal, everyday-object use by patients with impaired conceptual knowledge, one would not need to postulate the existence of a separate set of semantic representations for action. To investigate this contentious issue, we contrasted the performance of three patients representing two very different disorders on naturalistic tests of real-object usage and naming, tests of visually based object knowledge, and a mechanical problem-solving task involving the selection and use of novel tools. Two of the patients had a diagnosis of semantic dementia, a form of frontotemporal dementia associated with focal temporal lobe atrophy; the third had a diagnosis of corticobasal degeneration with bilateral parietal lobe atrophy.

We hypothesized that the patients with semantic dementia would show normal object-specific usage only for objects yielding evidence of some retained conceptual knowledge on other semantic tests. For objects on which these patients failed in the additional semantic tests, we predicted that they would use them in a plausible fashion, but not necessarily in accordance with their conventionally correct usage. The degree of correct usage should be determined by the specificity of the object’s affordances. Furthermore, and importantly, our hypothesis requires that the patients with semantic dementia have preserved mechanical problem-solving skills in a task involving selection and use of novel tools. By contrast, we expected the patient with corticobasal degeneration to reveal defective performance in both the novel tool assessment of mechanical problem solving and object usage, despite having normal semantic knowledge about the familiar objects.

Patients with semantic dementia are characterized by progressive degradation of semantic memory or conceptual knowledge. Although the earliest and most prominent deficits are in the domain of language, with loss of receptive and especially expressive vocabulary, all abilities requiring access to conceptual knowledge are increasingly compromised with disease progression; thus, longitudinal assessments almost invariably reveal degraded knowledge of objects and familiar people as well as words. In contrast to the semantic deficits, other aspects of cognition—including episodic and working memory, syntactic and phonological aspects of speech, visuospatial and perceptual skills, and frontal “executive” abilities—are strikingly well preserved (10–14). Structural (magnetic-resonance) brain imaging reveals highly focal atrophy of the anterior (polar) and inferolateral portions of the temporal lobe; the atrophy is typically bilateral but asymmetrical and often more severe in the left temporal lobe (15). Functional imaging studies in normal subjects also have highlighted the left inferolateral temporal region in semantic processing of both words and pictures of objects (16–18).

The syndrome of semantic dementia offers a uniquely informative opportunity to evaluate a putative action-semantic system: although such patients have significantly degraded conceptual knowledge about common objects, they have been reported to use these objects normally in the everyday setting of their own homes (5, 19, 20). The only assessment, to date, of this specific question in a patient with semantic dementia suggested that conceptual knowledge of objects is not necessary to support their correct usage, but the patient was tested at a stage at which semantic memory was only moderately impaired and no attempt was made to compare knowledge and usage of the same items (21).

The syndrome of corticobasal degeneration, which recently has attracted increasing attention, in many ways constitutes the mirror image to semantic dementia. Patients present with progressive difficulty in using real objects because of profound limb apraxia in the absence of substantial weakness, ataxia, or sensory loss. Gait disturbance and Parkinsonian features are also typically present, and alien limb phenomena may occur. Language and general cognitive abilities are reported to be normal, although semantic knowledge has not been investigated systematically. In keeping with the clinical picture, the pathological features are pronounced in the basal ganglia and parietal lobes and, in some cases, involve the frontal lobes, whereas the temporal lobes are spared (22–24).

Case 1, DJE, a 62-year-old, right-handed former builder, presented with a 4-year history of progressive word-finding difficulties followed by a relentless decline in comprehension. At the time of the examination, DJE’s communicative impairments rendered conversation nearly impossible (Examiner: “Do you have any hobbies?” Patient: “What are hobbies?”). His day-to-day memory was well preserved. He had become an avid devotee of jigsaw puzzles and remained a keen gardener and competent golfer. On formal testing he showed a severe and pervasive semantic impairment on word-and-picture-based tests, affecting all conceptual categories tested (animals, fruit, household items, vehicles, and tools), but performed normally on tests of visuoperceptual and spatial ability that do not require specific semantic knowledge (e.g., figure–ground discrimination, fragmented letters, etc.). Whole-brain three-dimensional MRI scanning showed severe bilateral, but asymmetrical, focal temporal lobe atrophy involving the polar and inferolateral cortex, particularly on the left side, with sparing of the hippocampus. Of note is that the frontal and parietal lobes appeared normal.

Case 2, IF, a 66-year-old, right-handed ex-accountant, presented with a virtually identical history of progressive decline in expressive and receptive language ability, with severe impairment of semantic memory, but preservation of visuospatial abilities and practical skills. Compared with DJE, IF showed even more severely impoverished language output. An MRI also showed striking bilateral but asymmetrical (L > R), anterolateral temporal lobe atrophy.

Case 3, FL, a 75-year-old, right-handed woman, presented with a 3-year history of difficulty in using familiar objects, resulting in a severe deficit in everyday activities. For example, she was unable to use a knife and fork, to dress, or to write. Her left hand also wandered and groped at objects without conscious control (alien or anarchic limb phenomena). In striking contrast to this severe apraxia, FL’s language and memory functions were unaffected. In formal language testing she was moderately dysarthric but showed no evidence of any linguistic impairment. She was still reading novels and was able to give a detailed account of their content. Neurological examination showed a hypokinetic syndrome with asymmetric limb rigidity. MRI scanning revealed bilateral parietal lobe atrophy with sparing of the frontal cortex and, importantly, of medial and lateral temporal lobe structures.

The experimental test battery was based on 20 real objects and colored photographs of the same objects plus 12 novel tools in a mechanical problem-solving task (8). On each trial of this latter task, the subject must select the appropriate one of three novel tools for lifting a wooden cylinder (with a special feature matched to the appropriate tool) out of a socket. For example see Fig. 3.

Figure 3.

Examples from the novel tool task (8). The test consists of a set of six cylinders and six tools. On each trial, one cylinder is placed in a socket and a selection of three tools is placed beside the socket. The patient is asked to select the tool best suited to pick up the cylinder and then to demonstrate its usage. Two points are awarded for a correct first choice and one point is awarded for a self-correction (chance score, therefore, = 6).

Case 1, DJE, was unable to name any of the 20 familiar objects, although seven of his naming responses indicated partial knowledge about the object in question. On the picture-based tests of functional semantic knowledge of objects, his performance did not exceed chance level on any of the three conditions (see Fig. 1): matching a familiar object (i) to its typical location, (ii) to the typical recipient of its action (illustrated in Fig. 2), and (iii) to another item with similar usage. When handling the 20 real objects, he was never clumsy or hesitant, and when asked to demonstrate their use, he made no errors of spatial orientation or perseveration; but only seven (35%) were used appropriately. For each of the other 13 familiar objects, he demonstrated a use that was incorrect but largely compatible with the object’s physical properties (e.g., he carefully removed each match from the matchbox, commenting that they looked like “little pencils” and holding them as if to write; he used the nail clippers to demonstrate lifting something with the attached chain). There was striking overlap between the specific objects that DJE was able to use correctly and those of which he apparently had some residual conceptual knowledge: of the seven objects used correctly, six were those for which his naming responses demonstrated partial knowledge, with only one specifically correct usage from the set of objects where he gave no appropriate information in naming.

Figure 1.

Performance of the 3 patients (DJE and IF, semantic dementia and temporal lobe atrophy; FL, corticobasal degeneration and parietal lobe atrophy) and a group of 10 age- and education-matched controls on (i) naming of 20 familiar objects, (ii) three tests of functional semantic knowledge involving the matching of a picture of each of the same objects to either (a) its typical location (such as kitchen, bathroom, or workshop), (b) the typical recipient of its action (see Fig. 2), and (c) another item used for the same purpose (n = 20 per test, chance = 20/60); (iii) demonstrating the usage of the 20 items; and (iv) performance on the novel tool task (8). The pale blue shading represents the range of scores of the 10 controls on each task.

Figure 2.

Example of the test of functional semantic knowledge in which the subject is asked to select which of the top three items “you typically use the lower one with.” The test consists of 20 target items with two preceding practice items.

Case 2, IF, also was unable to name any objects, and his naming errors indicated partial knowledge for only two. He scored at chance on all of the pictorial tests of functional and associative semantic knowledge. Unlike case 1, however, IF succeeded in demonstrating normal usage of nine of the 20 real objects, including the two with partial knowledge on naming. Analysis of the responses revealed that this “correct” performance occurred either for objects with a transparent relationship between physical structure and usage (e.g., scissors: he inserted fingers into the holes then tentatively separated the fingers) or as a result of trial and error [for example, when handed the pencil sharpener he inspected it with no apparent recognition; subsequently, when given a pencil, he first inserted the pencil into the hole the wrong way (pointed end) up and held vertically, then tried it with the correct—pointed end—in the sharpener (still held vertically), and eventually turned it to the horizontal and tried turning the pencil].

In the novel tool task designed to test mechanical problem solving, both DJE and IF scored flawlessly (12/12) and their responses were as fast and fluent as those of normal controls (see Fig. 1).

Case 3, FL, with corticobasal degeneration, scored within the normal range on both the object-naming and picture-based tests of visual semantic knowledge. She succeeded in demonstrating the correct use of 12 of 20 of the common objects, which, although slightly better than either of the patients with semantic dementia, is very poor, especially in the context of preserved object knowledge. Furthermore, even on the 12 correctly used objects, her movements were hesitant and unconfident. She was very impaired on the mechanical problem-solving task involving the selection of the correct novel tool (7/12: chance responding would give a score of 6/12 because there is a score range of 0 to 2 per item). This poor performance cannot be explained on the basis of FL’s impaired motor skill, because the subject is not required to use the novel tool correctly (although DJE and IF always did so) but can score simply by indicating the appropriate tool.

In keeping with previous anecdotal reports, the semantic-dementia patients showed some preservation of real object usage. In the case of DJE, there was a close correlation between items with evidence of partially retained knowledge (based on naming responses) and those eliciting item-specific correct use of the real objects. IF correctly used nine objects, despite naming none and producing circumlocutions indicating partial knowledge of only two; but his correct responses almost never had the immediate quality of a normal person’s object use but, rather, indicated a gradual problem-solving approach.

In terms of the contrasting hypotheses outlined above, it seems highly unlikely that the partial preservation of object usage represents the normal functioning of a separate action-semantic system: both patients with semantic dementia performed at chance on all three picture tests of functional semantic knowledge, which gives no support to the proposal of Lauro-Grotto et al. regarding the grounding of object use in a visual semantic system (5); IF’s apparently normal usage of some objects in the absence of explicit knowledge typically was derived by reliance on visual (and perhaps tactile) affordance with a trial-and-error, problem-solving approach; and both of these patients performed flawlessly on the novel tools task.

We suggest, therefore, that the semantic dementia cases were drawing on a “reasoning” system that is independent of any object-specific semantic representations. This hypothesis is supported by the findings in patient FL, who showed the converse pattern attributable to disruption of this visuomotor reasoning system. Her performance in object use also was severely impaired, and in a fashion that cannot be ascribed merely to motor problems, despite the fact that her knowledge about the objects was easily within the normal range. We cannot exclude the possibility that a separate action-semantic system exists but can be accessed only after an item is correctly identified, although this seems a less parsimonious explanation.

The strikingly different patterns exhibited by these cases are germane to understanding the properties of the nonsemantic visuomotor system in humans. The initial characterization of the two principal visual-processing streams was in terms of the “what” and “where” pathways (25). The importance of the ventral (occipitotemporal) stream for successful object recognition, categorization, and naming remains undisputed and is confirmed further by the syndrome of semantic dementia as exemplified here by DJE and IF. The dorsal (occipitoparietal) stream, however, now is viewed by some investigators more in terms of its role in the on-line guidance of motor functions that are computed according to position, axis length, and orientation of objects in space, rather than the simple location of objects in space (26, 27). It has been argued that this system may function automatically without recourse to other brain systems, although its role in everyday object use has yet to be specified. Key experiments in a patient who had destruction of the ventral stream with preservation of the superior parietal regions (28) entailed simple, manual tasks such as letter posting and object grasping, which the patient performed normally although she was unable to match oriented lines correctly. Milner and Goodale (26, 28) have claimed that a system that integrates ventral- and dorsal-stream processing is necessary for skilled and appropriate object use: “… a higher-level praxis system needs to have access to the products of the ventral stream’s processing, so that it can then ‘instruct’ the relevant visuomotor systems.” Milner and colleagues also have argued that the dorsal system, by itself, should enable efficient, though not necessarily precisely appropriate, object usage (29).

The findings of the present study suggest a refinement to the hypothesis regarding two streams of visual processing. In addition to dorsally based processes that permit accurate object location and reaching, there are clearly more sophisticated processes that facilitate the plausible—if not always correct—manipulation and usage of objects. It seems likely that parietal lobe areas are responsible for the transformation of spatial representations of attended objects into the motor coordinate frame for action, which entails an element of mechanical problem solving. Outputs from parietal lobe to premotor cortex then presumably translate these representations into specific motor plans.

In conclusion, we found no evidence to support the hypothesis of an object-specific, action-semantic system that is likely to be spared and that can support the use of objects when conceptual knowledge is disrupted. We argue instead that, when semantically impaired patients are observed to use objects (for which they have degraded knowledge) “appropriately,” this is attributable to a parietal lobe system specialized for visuomotor interaction with the environment, which may be triggered by the visual, and perhaps tactile, affordances of objects. Although this system presumably is tuned by learning and experience with objects in general, it does not rely on object-specific knowledge but, rather, enables mechanical problem solving and, hence, the efficient use of objects (whether familiar or not) in a manner consistent with their physical properties. The apraxic disorders of patients with parietal lobe pathology may be attributable to disruption of this hypothesized problem-solving ability. Competent conventional use of objects depends on additional conceptual knowledge for which inferotemporal brain structures appear to be critical.

Note

We acknowledge that our assessment of real-object use by patients with semantic dementia was not designed to maximize the probability of correct responding, in that (i) the stimuli (e.g., hammers, pencil sharpeners, bottle openers) were exemplars belonging to the experimenters rather than the patient’s own familiar versions of these objects, and (ii) the objects were presented in isolation rather than in their typical, visually rich and relevant contexts [see Snowden et al. (20) for evidence that object use by a patient akin to DJE and IF was facilitated significantly both for own vs. other’s exemplars and for appropriate vs. neutral context]. An understanding of the mechanisms underlying this benefit, though an important issue in its own right, is a different issue from the one addressed here. Normal individuals have no difficulty in recognizing and using unfamiliar exemplars of everyday objects. Furthermore, despite the benefit from familiarity of specific exemplar and/or context, we have many anecdotal examples of patients with semantic dementia misusing their own familiar objects in their own homes.