nlp.ppt

What is NLP?

• _{Natural Language Processing (NLP)}

– _{Computers use (analyze, understand, generate)}

natural language

– _{A somewhat applied field}

• _{Computational Linguistics (CL)}

– _{Computational aspects of the human language}

faculty

Why Study NLP?

• _{Human language interesting & challenging}

– _{NLP offers insights into language}

• _{Language is the medium of the web} • _{Interdisciplinary: Ling, CS, psych, math} • _{Help in communication}

– _{With computers}

– _{With other humans}

Goals of NLP

•

_{Scientific Goal}

– _{Identify the computational machinery}

needed for an agent to exhibit various forms of linguistic behavior

•

_{Engineering Goal}

– _{Design, implement, and test systems that}

Applications

• _{speech processing: get flight information or book a}

hotel over the phone

• _{information extraction: discover names of people and}

events they participate in, from a document

• _{machine translation: translate a document from one}

human language into another

• _{question answering: find answers to natural}

language questions in a text collection or database

• _{summarization: generate a short biography of Noam}

General Themes

• _{Ambiguity of Language}

• _{Language as a formal system}

Ambiguity of language

• _Phonetic

– _{[raIt] = write, right, rite}

• _Lexical

– _{can = noun, verb, modal}

• _Structural

– _{I saw the man with the telescope}

• _Semantic

– _{dish = physical plate, menu item}

Language as a formal system

• _{We can treat parts of language formally}

– _{Language = a set of acceptable strings}

– _{Define a model to recognize/generate language}

• _{Works for different levels of language}

(phonology, morphology, etc.)

• _{Can use finite-state automata, context-free}

Rule-based & Statistical Methods

• _{Theoretical linguistics captures abstract}

properties of language

• _{NLP can more or less follow theoretical}

insights

– _{Rule-based: model system with linguistic rules} – _{Statistical: model system with probabilities of}

what normally happens

The need for efficiency

• _{Simply writing down linguistic insights isn’t}

sufficient to have a working system

• _{Programs need to run in real-time, i.e., be}

efficient

– _{There are thousands of grammar rules which}

might be applied to a sentence

• _{Use insights from computer science}

– _{To find the best parse, use chart parsing, a form of}

Preview of Topics

1.Finding Syntactic Patterns in Human Languages: Lg. as Formal System

2.Meaning from Patterns

3.Patterns from Language in the Large

4.Bridging the Rationalist-Empiricist Divide 5.Applications

The steps in NLP

• Morphology: Concerns the way words are built up from

smaller meaning bearing units. (come(s),co(mes))

• _{Syntax: concerns how words are put together to form}

correct sentences and what structural role each word has.

• _{Semantics: concerns what words mean and how these}

meanings combine in sentences to form sentence meanings.

• Pragmatics: concerns how sentences are used in

different situations and how use affects the interpretation of the sentence.

• _{Discourse: concerns how the immediately preceding}

The Problem of Syntactic Analysis

• _{Assume input sentence S in natural language L} • _{Assume you have rules (grammar G) that}

describe syntactic regularities (patterns or structures) found in sentences of L

Example 1

S  NP VP VP  V NP

VP  V

S

VP

he

NP

V

slept

NP  I NP  he V  slept V  ate V  drinks

Parsing Example 1

• _S  NP VP

• _VP  V NP

• _VP  V

• _{NP  I} • _NP  he

• _V  slept

• _V  ate

More Complex Sentences

• _{I can fish.}

• _{I saw the elephant in my legs.}

• _{These sentences exhibit ambiguity}

• _{Computers will have to find the acceptable or}

Example 2

S

NP VP

Pronoun _{V NP}

I can

S

NP VP

Pronoun _{V NP}

I can

S

NP VP

Pronoun _{Aux V}

I can _fish

S

NP VP

Pronoun _{Aux V}

Example 3

• _{NP  D Nom}

• _{Nom  Nom RelClause}

• _{Nom  N}

• _{RelClause  RelPro VP} • _{VP  V NP}

• _{D  the} • _{D  my} • _{V  is} • _{V  hit} • _{N  dog} • _{N  boy}

• _{N  brother} • _{RelPro  who}

S NP VP Nom D Nom RelClause RelPro VP V who NP

hit the dog is my brother

the N boy S NP VP Nom D Nom RelClause RelPro VP V who NP

hit the dog is my brother

the

N

Topics

1.Finding Syntactic Patterns in Human Languages

2. Meaning from Patterns

3.Patterns from Language in the Large

4.Bridging the Rationalist-Empiricist Divide 5.Applications

Meaning from a Parse Tree

• _{We want to understand}

– _{Who does what?}

• _{the canner is me, the action is}

canning, and the thing canned

is fish.

• e.g. Canning(ME, FishStuff)

• This is a logic representation of meaning

S

NP VP

Pronoun _{V NP}

N

I can

fish S

NP VP

Pronoun _{V NP}

N

I can

fish

We can do this by

• associating meanings with lexical items in the tree

Meaning from a Parse Tree (Details)

• _{Let’s augment the}

grammar with

feature constraints

• _S  NP VP

– <S subj> =<NP>

– <S>=<VP>

• _VP V NP

– _{<VP> = <V>}

– <VP obj> =<NP>

S

NP VP

Pronoun _{V NP}

N

I can

fish S

NP VP

Pronoun _{V NP}

N I can fish *2:[pred: Canning] [subj: *1 pred: *2 obj: *3]

*1[sem: ME]

*3[sem: Fish Stuff] [pred: *2

Grammar Induction

• _{Start with a tree bank = collection of parsed}

sentences

• _{Extract grammar rules corresponding to parse trees,}

estimating the probability of the grammar rule based on its frequency

P(Aβ | A) = Count(Aβ) / Count(A)

• _{You then have a probabilistic grammar, derived from}

a corpus of parse trees

• _{How does this grammar compare to grammars}

created by human intuition?

Empirical Approaches to NLP

• _{Empiricism: knowledge is derived from experience} • _{Rationalism: knowledge is derived from reason}

• _{NLP is, by necessity, focused on ‘performance’, in that}

naturally-occurring linguistic data has to be processed

– Have to process data characterized by false starts, hesitations,

elliptical sentences, long and complex sentences, input in a complex format, etc.

• _{The methodology used is corpus-based}

– linguistic analysis (phonological, morphological, syntactic, semantic, etc.) carried out on a fairly large scale

Which Words are the Most Frequent?

Common Words in Tom Sawyer (71,730 words), from

The Annotation of Data

• _{If we want to learn linguistic properties from}

data, we need to annotate the data

– _{Train on annotated data}

– _{Test methods on other annotated data}

• _{Through the annotation of corpora, we}

Topics

1.Finding Syntactic Patterns in Human Languages

2.Meaning from Patterns

3.Patterns from Language in the Large

4.Bridging the Rationalist-Empiricist Divide

5. Applications

Application #1: Machine Translation

• _{Using different techniques for linguistic}

analysis, we can:

– _{Parse the contents of one language}

– _{Generate another language consisting of the same}

MT Approaches

Interlingua

Semantics Semantics

Syntax Syntax

Morphology Direct

Syntactic Transfer

MT Using Parallel Treebanks

hit the dog is my brother

the N boy S NP VP Nom D Nom RelClause RelPro VP V who NP

hit the dog is my brother

the

N

Application #2: Understanding a Simple Narrative (Question Answering)

Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her.

1. When did the running occur?

Yesterday.

2. When did the twisting occur?

Yesterday, during the running.

3. Did the pushing occur before the twisting?

Question Answering by Computer (Temporal Questions)

Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her.

09042005 09052005 run twist ankle d u ri n g fi n is h es o r d u ri n g before

push _befo

re d u ri n g

1. When did the running occur?

Yesterday.

2. When did the twisting occur?

Yesterday, during the running.

3. Did the pushing occur before the twisting?