lecture2-dictionary-handout-6-per

Eachsuchextendedbiwordisnowmadeaterminthe diconary

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: c)on to Informa)on Retrieval Other
stemmers
 Sec. 2.2.4 Introduc)on to Informa)on Retrieval Language‐specificity
   Other
stemmers
exist,
e.g.,
Lovins
stemmer

   Many
of
the
above
features
embody
transforma)ons
 that
are
 
   hZp://www.comp.lancs.ac.uk/compu)ng/research/stemming/general/lovins.htm   Single‐pass,
longest
suffix
removal
(about
250
rules)
   Language‐specific
and
   Oben,
applica)on‐specific
   Full
morphological
analysis
–
at
most
modest
 benefits
for
retrieval
   These
are
“plug‐in”
addenda
to
the
indexing
process
   Both
open
source
and
commercial
plug‐ins
are
 available
for
handling
these
   Do
stemming
and
other
normaliza)ons
help?
   English:
very
mixed
results.
Helps
recall
but
harms
precision
   opera)ve
(den)stry)
⇒ oper
   opera)onal
(research)
⇒ oper
   opera)ng
(systems)
⇒ oper
   Definitely useful for Spanish, German, Finnish, …   30% performance gains for Finnish! 25
 Sec. 2.2 Introduc)on to Informa)on Retrieval 26
 Introduc)on to Informa)on Retrieval Dic)onary
entries
–
first
cut
 ensemble.french 時間.japanese MIT.english These may be grouped by language (or not…). More on this in ranking/query processing. mit.german guaranteed.english entries.english sometimes.english FASTER
POSTINGS
MERGES:
 SKIP
POINTERS/SKIP
LISTS
 tokenization.english 27
 Sec. 2.3 Introduc)on to Informa)on Retrieval Recall
basic
merge
   Walk
through
the
two
pos)ngs
simultaneously,
in
 )me
linear
in
the
total
number
of
pos)ngs
entries
 28
 Sec. 2.3 Introduc)on to Informa)on Retrieval Augment
pos)ngs
with
skip
pointers
 (at
indexing
)me)
 128 41 2 4 8 41 2 3 8 2 8 4 8 41 1 2 3 8 48 11 64 17 128 21 Brutus 31 Caesar If the list lengths are m and n, the...
View Full Document

This document was uploaded on 02/26/2014.

Ask a homework question - tutors are online