23 Lecture Machine Translation
CS 6320
What is MT
Translating text in a foreign language (f) into English (e) Done by computer Human languages Types/Applications Rough translation on opendomain You can still understand the meaning You use it as a draft Postediting Accurate translation in closeddomain I.e.: Weather forecasts
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
2
MT is hard
Languages are different Morphology Different syntactic structure ADJNOUN / NOUNADJ English: a red apple French: une pomme rouge SVO/SOV/VSO SVO: English, French, German, Mandarin SOV: Japanese, Hindi VSO: Irish, Arabic, Biblical Hebrew
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
3
MT is hard
Syntactic differences Marking (where you mark relation)
Headmarking: the man's house Dependent: az ember haza (Hungarian) the man househis Manner and direction of motion Verbframed: The bottle exited floating (la botella salio flotando) Satelliteframed: The bottle floated out.
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
4
MT is hard
Different ways to represent namedentities Dates: MMDDYY (AmEnglish) DD/MM/YY (Br English) YYMMDD (Japanese)
Different reference systems for dates Idioms Phrases noncompositional meaning
go with the flow kick the bucket call it a day Words cultural context chicken
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
5
MT is hard
Functional words Determiners Missing: Chinese With gender: French (un/une) Pronouns Verbal infliction English: very simple Spanish: complicated (o = I, as = you, a = he/ she/it, amos = we, an = they) Pronoun might be missing
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
6
MT is hard
Pronouns Different sets in different languages
Subject pronouns Singular 1st person 2nd person 3rd person je tu il elle on I you he, it 3 she, it one Plural nous vous ils elles we you they they
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
7
MT is hard
Verb tenses Lexical issues Onetomany word mapping informatique = computer science Granularity Kinship: Japanese & Chinese: younger brother / older brother Food Eskimo snow Missing concepts
8
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
MT is hard
Lexical issues Complex mapping between words and concepts
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
9
Approaches for MT
Based on level of analysis The MT pyramid:
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
10
Direct Translation
Direct translation Works on input string Disadvantages: Problems with word order Variants Wordforword Examplebased Statistical Statistical techniques: Best results
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
11
Transfer approaches
Steps Analysis Transfer Generation Transfer Contrastive knowledge Syntactic transfer: Alter parse tree Rearrange nodes Insert nodes Delete nodes Lexical transfer
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
12
Syntactic transfer
Advantages Deals well with wordorder problems Disadvantages Like direct transfer, is languagepair dependent Syntactic mismatch in languages Example: The bottle floated into the cave. Spanish: La botella entro a la cuerva flotando (The bottle entered the cave floating)
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
13
Interlingua
Assign a logic form/meaning representation to a sentence John must not go = OBLIGATORY(NOT(GO(JOHN))) John may not go = NOT(PERMITTED(GO(JOHN))) Generate English sentence out of the meaning representation Advantages: For n languages, you need to write only n interpreters and n generators (not n2 like for those that use contrastive knowledge) Disadvantages Difficult to define a single logic form Lexical issues ontologies Preserve ambiguity
14
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
ExampleBased
Fundamental idea: Most of the time people do not translate by doing deep linguistic analysis of a sentence They translate by decomposing the sentence into fragments, translating each of the fragments and then composing those properly Advantages Higher quality because of usage of fragments of human translation Disadvantages May limit coverage depending on the size of the example database
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
15
Examplebased
Example:
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
16
Statistical MT
Get the most probable translation How do we define most probable?
P (e | f )
What do we want?
^ e = arg max P(e | f )
Hard, if not impossible, to compute directly Sourcechannel model use Bayes rule
e
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
17
Statistical MT
Applying Bayes rule:
^ e = arg max
e
P ( f | e ) P ( e) = arg max P( f | e) P(e) P( f ) e
P(f|e) = translation model probability Assign higher probability to sentences to pair of sentences <f,e> that have the same meaning Estimated ("trained") using bilingual corpora P(e) = language model probability Assign higher probability to fluent/grammatical sentences Estimated using monolingual corpora I.e.: trigram language model
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
18
Statistical MT
Another way to see it:
^ e = arg max faithfulness ( f , e) fluency (e)
e
Faithfulness (translation model) Preserve meaning Preserve concepts Preserve relationship between concepts Fluency (language model) The output IS really English and not something like "Him, me, friends. Very good." instead of "We are very good friends".
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
19
Statistical MT
Graphically:
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
20
Statistical MT
Advantages Deals well with lexical ambiguity Deals with well idioms (some implementations) Data driven and language independent > minimal human effort switching to another pair of languages can be done very fast Can be created for any language pair if you have enough data Disadvantages Doesn't explicitly deal well with syntax i.e.: SOV SVO pair of languages
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
21
Translation model
Cannot compute directly Example: f = Ces gens ont grandi, vcu et oeuvr des dizaines d'annes dans le domaine agricole. e = Those people have grown up, lived and worked 1 many years in a farming district. e = I like bungee jumping off high bridges. 2 Try counts:
count ( f , e) P ( f | e) = count (e)
Assumes you already have the translation in the training corpus
22
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
Translation model
Thus you need to break down the formula Introduce an intermediary (hidden variable): a = alignment
P ( f | e) = a P ( a , f | e)
Alignment between intermediary words in the sentence pair
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
23
Alignment
P(a,f|e) still cannot be computed directly Let's give an approximation:
P (a, f | e) = t ( f j | ei )
m
t(fj|ei) can be calculated using counts! More, in the sum that defines P(a,f|e) there will be a term that dominates the sum Thus, P(a,f|e) can be approximated with:
j =1
P ( f | e) = max a P (a, f | e)
Viterbi alignment
^ a = arg max a P (a, f | e)
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
24
Simple SMT
This is an approximation of IBM Model 1 IBM Model 1 also uses string length probability
Problems Doesn't model right the fertility When a word is aligned to more than one word When a word is aligned to zero words Doesn't model word reordering at all (distortion) Bad for idioms it is not able to model efficiently the translation of a phrase in f into a phrase in e Some of the problems are corrected in IBM Model 4
25
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
Decoding
Search problem Like in Automatic Speech Recognition A* Informed depthfirst search Beam search Informed breathfirst search Pruning: discard hypothesis with very low probability Merging: merge two hypothesis that will generate the same output Need future cost estimation
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
26
Loglinear models
Instead of sourcechannel model, define the probability:
^ e = arg max( m hm (e, f , a ))
e,a m =1
m
hm features m weights Sourcechannel model can be cast to this model h (e,f,a) = log P(a,f|e) 1
h2(e,f,a) = log P(e)
Problems: Determine m
Maximum entropy Minimum error rate training
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
27
Align parallel corpora
EM algorithm Implemented in GIZA++
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
28
Align parallel corpora
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
29
Align parallel corpora
GIZA++ outputs Viterbi alignment for f>e and e>f Improve: heuristics
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at Dallas
30
Phrasebased SMT Pharaoh
Idea: divide f into phrases and translate each phrase individually
Models Language model: computes P(e) ngrams Translation model: computes the probability to translate a phrase in f into a phrase in e defined in a translation table Distortion model penalize phrase reordering Word penalty penalize short sentences
2005 Dan I. Moldovan, Human Language Technology Research Institute, The University of Texas at ...