Real action in understanding language

Dawn Wiseman

In order to better grasp how humans process language, Roberto de Almeida (Psychology) went to IKEA.

He wasn't interested in how customers figure out the company's range of odd product names like barnslig, driv and ektorp (a rug, light fixture and couch respectively), but rather in its prefab, well-lit kitchens. “We went with a film crew at 7 a.m., so we could film the events needed for our research.”

De Almeida's Psycholinguistic and Cognition Lab focuses largely on understanding how information gleaned from verbs, and verbs in sentences, structures our overall comprehension of meaning.

“Verbs are the key elements in sentences,” he explained. “They tell us who did what to whom.”

De Almeida and his research team are interested in what fundamental processes take place in the brain when it is understanding language. “It's looking at how we put words and pieces of words together and how they are conveyed into units of thought.”

One of the key areas of psycholinguistic research has been language deficits brought on by brain injury or disease. To date, this research has focused largely on nouns, and how trauma to the brain can affect language knowledge related to people, places or things. “Some stroke victims lose the ability to name natural objects, like trees, but can still name man-made objects, like tables,” said de Almeida. “We were wondering if there are similar dissociations with verbs.”

Graduate student Forouzan Mobayyen worked with ten Alzheimer's patients. She exposed them to causative verbs (like opening, closing, melting or freezing, that describe a change in state), motion verbs (like running, walking, crawling, or hopping) and perception verbs (like hearing, seeing, smelling, or touching), “and found that patients did indeed have difficulty in understanding some types of events more than others.” For Alzheimer sufferers perception events, those associated with the senses, seemed to be the hardest to grasp.

So why the film crew?

“Up to now,” explained de Almeida, “most research has been done by showing subjects static pictures and line drawings.” This method is not ideal for the dynamic events described by verbs. “If we want subjects to actually name the event, a more valid way of assessing verb knowledge is for them to see the action in full motion.”

The films made at IKEA have been used by de Almeida and graduate student Julia Di Nardo to examine how context affects the understanding of what is being said. “In some instances our brains can actually anticipate the meaning of a sentence before it is finished based on the visual context in which it is spoken.”

In one study, subjects were asked to watch three movies of a cook preparing a cake. While the video in each is slightly different, the audio remains virtually the same, and describes the need to “break” (a causative verb) or “inspect” (a perception verb) the eggs. “In one movie, the cook mixes the cake without visual reference to the eggs,” said de Almeida. “In another, he moves away from the eggs, and in the third he moves towards the eggs.”

When the verb was causative, subjects watching the third scene actually moved their eyes to the eggs in advance of the eggs being named by the narrator. “It's just a peek at the object on which the cook will act,” said de Almeida, “but they are clearly anticipating meaning, albeit under very constrained circumstances.”

The Psycholinguistic and Cognition Lab is undertaking collaborative work with researchers at the University of Alberta, UQAM, Brock, City University of New York (CUNY), Université de Montréal, and McGill. For in formation or to participate: alcor.concordia.ca/~coglab/coglab_experiment.html, or ext. 2210.