Noun phrase transla1on 51 dan jurafsky step 5 extracng

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: •  EuroParl: h~p://www.statmt.org/europarl/ •  A parallel corpus extracted from proceedings of the European Parliament. •  Philipp Koehn. 2005. Europarl: A Parallel Corpus for Sta1s1cal Machine Transla1on. MT Summit •  Around 50 million words per language for earlier members: •  Danish, Dutch, English, Finnish, French, German, Greek, Italian, Portuguese, Spanish, Swedish •  5 ­12 million words per language for recent member languages •  Bulgarian, Czech, Estonian, Hungarian, Latvian, Lithuanian, Polish, Romanian, Slovak, and Slovene •  LDC: h~p://www.ldc.upenn.edu/ •  Large amounts of parallel English ­Chinese and English ­Arabic text Dan Jurafsky Step 2: Sentence Alignment E ­F Bitext The old man is happy. He has fished many 1mes. His wife talks to him. The fish are jumping. The sharks await. Kevin Knight example El viejo está feliz porque ha pescado muchos veces. Su mujer habla con él. Los 1burones esperan. Sentence Alignment Algorithm: •  Segment each text into sentences •  Extract a feature for each sentence •  length in words or chars •  number of overlapping words from some simpler MT model •  Use dynamic programming to find which sentences align Dan Jurafsky Sentence Alignment: Segment sentences 1.  The old man is happy. 2.  He has fished many 1mes. 3.  His wife talks to him. 4.  The fish are jumping. 5.  The sharks await. 1.  El viejo está feliz porque ha pescado muchos veces. 2.  Su mujer habla con él. 3.  Los 1burones esperan. Dan Jurafsky Sentence Alignment: Features per sentence plus dynamic programming 1.  The old man is happy. 2.  He has fished many 1mes. 3.  His wife talks to him. 4.  The fish are jumping. 5.  The sharks await. 1.  El viejo está feliz porque ha pescado muchos veces. 2.  Su mujer habla con él. 3.  Los 1burones esperan. Dan Jurafsky Sentence Alignment: Remove unaligned sentences 1....
View Full Document

This document was uploaded on 02/14/2014.

Ask a homework question - tutors are online