Methods: A Brief Survey

(1)

Methods: A Brief Survey

Eva M. Fernández [email protected]

ABRALIN 24FEB05

S

NP VP

S NP VP

…Tree-Building is Really Happening?

• Is syntactic structure psychologically real?

even though it’s abstract…

 and not at all present in the signal?

• A simple test: RSVP task (see slides from 22FEB05)

RSVP Paradigm

Center-screen, word-by-word display Timing: N ms per word (here: N = 500)

Sentence-recall task

S NP VP

What Information is Used?

• In building (recovering) syntactic structure, the information used includes:

LEXICAL INFORMATION

MORPHO-SYNTACTIC INFORMATION

PHONOLOGICAL INFORMATION (including prosody)

SEMANTIC INFORMATION

GRAMMAR

S NP VP

Three Operations of the Syntax

• Building simple structure…

“Mary often speaks about Joe.”

* “Mary often speak about Joe.”

• Combining simple sentences into complex ones…

“Mary told Joe something.” + “Joe had his shoelace untied.”

Æ “Mary told Joe he had his shoelace untied.”

• Moving elements of sentences around…

“Mary told Joe what”

Æ “What did Mary tell Joe __ ?”

S NP VP

It follows that…

• If the grammar is consulted during sentence processing, we should expect that the system will dislike:

UNGRAMMATICAL sentences

COMPLEX sentences

sentences involving lots of MOVEMENT

 dislike = take longer to process?

be unable to process?

just howcould one measure the parser’s tastes?

(2)

S NP VP

A Potential Problem

• You might reject a grammatical sentence because it’s prescriptively bad:

Where’s the library at?

Who did Joe meet at the party?

Mary’s love life doesn’t concern you and I.

Me and you shouldn’t be talking about Mary’s affairs.

This is historic times. (GWB) etc.!

S NP VP

Another (More Important) Problem

• You might reject a grammatical sentence because it’s hard (or impossible!) to process:

Mary put the candy on the table in her mouth.

When Madonna sings the song is always a hit.

The son of Paraoh’s daughter looked at himself in the mirror.

The cat the dog the boy walked bit meowed.

The machine covered with paper plates handles with chrome.

etc.!

S NP VP

Processing Difficulty: Why?

• Why does the parser dislike some sentences more than others?

Because of the listener’s

inadequate knowledge of language?

Because of sentence processing routines that have gone wrong, that have applied incorrectly?

S NP VP

Processing Routines Defined

• Mechanisms that operate during production and perception:

in constant contact with the grammar

based on working-memory limitations

Minimal Attachment: build the simplest structure Late Closure: attach locally

Minimal Chains: posit the fewest filler-gap dependencies

S NP VP

Psychological Reality, Again...

• How can you tell that the processing routines are being followed?

Observe their work in action…

• Physics: which will fall faster, 1kg of feathers or 1kg of lead?

• Psycholinguistics: which linguistic stimulus will be understood more easily and faster,

one with a Minimal Attachment violation or one without?

S NP VP

Forster & Chambers, 1973 (described in Forster, 1979):

stimuli: letter sequences were either words or orthographically legal non-words, e.g., thamon

tasks: (i) naming (“pronounce ASAP”) (ii) lexical decision

results: lexical decision times (608 ms)

> naming times (508 ms)

(3)

S NP VP

Forster, 1974 (also described in Forster, 1979):

• stimuli: grammatical, ungrammatical, non-word sentences

1. The scouts annoyed the lady 2. The bicycle the calculated cognac 3. The plane gleashed the passengers

• tasks: (i) are all items in input familiar words?

(ii) is sentence meaningful?

• results: “no significant difference between decision times for the two tasks, and indeed for some types of sentences, the sentence task was slightly faster than the lexical task” (p. 30)

S NP VP

Organization of Language Processor and GPS (after Forster, 1979)

LEXICO N

MESSAGE PROCESSOR

SYNTACTIC PROCESSOR

LEXICAL

PROCESSOR GralPROBLEM SOLVER GralCONCEPTUAL KNOWLEDGE

INPUT FEATURES

DECISION OUTPUT

S NP VP

Ambiguity

• At almost any point during structure-building, the parser has multiple options (LOCAL AMBIGUITY!)

S

NP VP

V NP

We knew Ann’s date…

S

NP VP

V S

We knew Ann’s date…

We knew Ann’s date, Joe. We knew Ann’s date would embarrass her at the party.

S NP VP

Multiple Local Ambiguities…

Have the soldiers marched into the barracks…

… ?

… , please!

… had their supper?

… take their boots off, would you?

S NP VP

Ambiguity…

• Not to be confused with vagueness:

“I guess the movie was interesting.”

• Occurs when a lexical string has two possible structures, GLOBALLY:

“The bird is ready to eat.”

“Visiting relatives can be a real nuisance.”

“Joe said Mary called him yesterday.”

“Joe saw Mary with the telescope.”

“Joe saw a dog next to a kitten with an orange sweater.”

• or LOCALLY:

“We knew Ann’s date … , Joe.

… would embarrass her at the party.

… into the barracks?

S NP VP

When there’s an ambiguity…

• How does the parser go about choosing among the alternatives?

 Does the parser notice all of the alternatives?

If so, are all of the alternatives kept active?

A debate in the literature, indeed, but the consensus:

THE PARSER NOTICES THE STRUCTURAL ALTERNATIVE THAT’S EASIEST TO BUILD, following strategies like:

Minimal Attachment: build the simplest structure Late Closure: attach locally

(4)

S NP VP

AN EXPERIMENT

The legionnaires marched into the desert and searched for the nearest oasis.

The legionnaires marched into the desert surprised the Persian forces.

MY HYPOTHESIS…

2^ndsentence is harder:

violates Minimal Attachment let’s see how!

S

VP

marched into the desert NP

the legionnaires

conj

and

VP VP

searched for the nearest

oasis

marched into the desert S

NP

the legionnaires

NP

marched into the desert

S

VP

surprised the Persian forces Uh - oh!

S NP VP

THE EXPERIMENTAL METHOD

• Hypothesis testing

you have an idea about how structures are built by the sentence processor

you test it by designing an experiment

you make predictions about how the experiment will come out

• you think about what it will mean if the experiment comes out the “wrong way”

• this will be easier if you use BINARY COMPARISONS

S NP VP

PARTICIPANTS

• never too many?

• “clones” of each other

• selection criteria:

language history, education history, sex, handedness, age, reading ability, etc.

• controlling:

use background questionnaire

use behavioral criteria

S NP VP

MATERIALS

• never too many?

• instantiating the contrast you’re interested in

• normed in whatever relevant way

• interspersed among distractor items

never too many?

usual filler-target ratio, 2-1 or 3-1

• pseudo-randomized lists

• featuring binary comparisons (unless absolutely necessary)

(5)

S NP VP

PROCEDURE

• “speeded classification tasks”:

“the subject of the experiment is presented with some item of linguistic input, which must be classified according to some experimenter-defined criterion”

• try several types! (ingenuity is essential)

• don’t be fooled by technology:

just because it costs more doesn’t mean it’s more effective

just because everyone says it’s online doesn’t mean it taps the phase of processing you want (e.g., how do you know that information hasn’t flowed through a “later” processor before the GPS makes a decision?)

S NP VP

QUESTIONNAIRES

• a grammaticality / acceptability judgment task

• On a scale of 1-7, indicate how acceptable you think each of these sentences are.

1 = perfectly acceptable

7 = perfectly awful

• measure: most frequent response type

1 2 3 4 5 6 7

S NP VP

ASSETS & LIABILITIES of questionnaire procedures?

• flexible

• low-tech lab

• test 100 people at a time!

• off-line: metalinguistic awareness, prescriptive rules

• subjects might engage in undesirable behavior:

strategies, looking back, etc.

S NP VP

RSVP

• rapid serial visual presentation

words presented center-screen

500 msec for each word

• presentation rate could be varied

task is to remember entire sentence and repeat or write down

measure: accuracy of recall

S NP VP

SELF-PACED READING

• subject controls pace of presentation

constraints on speed determined by experimenter:

• time-outs, instructions

• materials presented in a series of “chunks”

word-by-word, phrase-by-phrase, clause-by-clause

• different types of displays

centered, incremental, moving window

• measure: reading time

(6)

S NP VP

SPR, wd-by-wd: measure

desert the

into marched lgnnaires

The

forces Persian

the surprised

oasis nearest the

for searched and

desert the

into marched lgnnaires

The

S NP VP

SPR, wd-by-wd: measure

into the desert The legionnaires marched

the Persian forces surprised

for the nearest oasis and searched

into the desert The legionnaires marched

S NP VP

KEEPING THE PARTICIPANTS HONEST

• asking questions

after every trial

after every N trials

never?

• excluding error-prone subjects

• providing instant feedback

• the speed-accuracy trade-off:

the faster you respond, the more likely you’ll make errors

--- --- ---- --- --- ---- --- --- - ---.

The student told the professor that everyone hated a lie.

Who told a lie?

the student the professor Did the professor lie?Did the student lie?

correct 2359

S NP VP

SAME-DIFFERENT

SENTENCE-MATCHING

The legionnaires marched into the desert and searched for the nearest oasis.^U The legionnaires marched into the desert and searched for the nearest oasis.

(7)

U

The alligator with the sharp teeth inspected the rifle.

The alligator with the sharp teeth inspected his rifle.

U

Everyone at the party knew Ann’s date had made a fool of himself.

Everyone at the party knew Ann’s mate had made a fool of himself.

S NP VP

EYE-TRACKING: fine-grained

• visual span:

9 letters, + periphery

• focus & move:

eyetracker records focus

• measurements:

first fixations

regressions

total reading

See how the eye moves across the page in normal reading:

http://gandalf.psych.umn.edu/~gellab/mrchips/chips2d.html animation by Steve Mansfield & Tim Klitz

S NP VP

Eyetracking: challenges

• a more “on-line” method than SPR?

the problem of time during saccades

the problem of as much time as necessary for a given item

the problem of peripheral vision

• more limitations

length of items limited by size of display

costly equipment

bite-bar

laborious analyses

S NP VP

EYETRACKING: coarse-grained

“Real World” paradigm

Head-mounted eyetrackers permit examining how the immediate visual context can affect sentence processing.

Easy to use, with adults... and children

Evidence of interactive system for language processing?

(See Garrett, 2000, for discussion.)

(8)

S NP VP

BRAIN IMAGING

• “distinct brains systems associated with several different structural domains…: phrase structure, inflection, movement and binding, and lexical semantics” (Garrett, 2000, p. 43)

• semantic versus syntactic task priorities:

different electrode locations

different timing

• early responses to syntax:

inflexible, localized in classic language processing areas

• later responses to syntax:

more sensitivity to interpretative factors

more broadly distributed