Robust A Retrieval Engine for Proximal and Structural Search
Katsuya Masuda Takashi Ninomiya Yusuke Miyao Tomoko Ohta Junichi Tsujii Department of Computer Science, Graduate School of Information Science and Technology, University of Tokyo, Hongo 7-3-1, Bunkyo-ku, Tokyo 113-0033, Japan CREST, JST (Japan Science and Technology Corporation) Honcho 4-1-8, Kawaguchi-shi, Saitama 332-0012, Japan {kmasuda,ninomi,yusuke,okap,tsujii}@is.s.u-tokyo.ac.jp
1
Introduction
In the text retrieval area including XML and Region Algebra, many researchers pursued models for specifying what kinds of information should appear in specied structural positions and linear positions (Chinenyanga and Kushmerick, 2001; Wolff et al., 1999; Theobald and Weilkum, 2000; Clarke et al., 1995). The models attracted many researchers because they are considered to be basic frameworks for retrieving or extracting complex information like events. However, unlike IR by keywordbased search, their models are not robust, that is, they support only exact matching of queries, while we would like to know to what degree the contents in specied structural positions are relevant to those in the query even when the structure does not exactly match the query. This paper describes a new ranked retrieval model that enables proximal and structural search for structured texts. We extend the model proposed in Region Algebra to be robust by i) incorporating the idea of rankedness in keyword-based search, and ii) expanding queries. While in ordinary ranked retrieval models relevance measures are computed in terms of words, our model assumes that they are dened in more general structural fragments, i.e., extents (continuous fragments in a text) proposed in Region Algebra. We decompose queries into subqueries to allow the system not only to retrieve exactly matched extents but also to retrieve partially matched ones. Our model is robust like keyword-based search, and also enables us to specify the structural and linear positions in texts as done by Region Algebra. The signicance of this work is not in the development of a new relevance measure nor in showing superiority of structure-based search over keyword-based search, but in the proposal of a framework for integrating proximal and structural ranking models. Since the model treats all types of structures in texts, not only ordinary text structures like title, abstract, authors, etc., but also semantic tags corresponding to recognized named entities
or events can also be used for indexing text fragments and contribute to the relevance measure. Since extents are treated similarly to keywords in traditional models, our model will be integrated with any ranking and scalability techniques used by keyword-based models. We have implemented the ranking model in our retrieval engine, and had preliminary experiments to evaluate our model. Unfortunately, we used a rather small corpus for the experiments. This is mainly because there is no test collection of the structured query and tag-annotated text. Instead, we used the GENIA corpus (Ohta et al., 2002) as structured texts, which was an XML document annotated with semantics tags in the led of biomedical science. The experiments show that our model succeeded in retrieving the relevant answers that an exact-matching model fails to retrieve because of lack of robustness, and the relevant answers that a nonstructured model fails because of lack of structural specication.
2 A Ranking Model for Structured Queries and Texts
This section describes the denition of the relevance between a document and a structured query represented by the region algebra. The key idea is that a structured query is decomposed into subqueries, and the relevance of the whole query is represented as a vector of relevance measures of subqueries. The region algebra (Clarke et al., 1995) is a set of operators, which represent the relation between the extents (i.e. regions in texts). In this paper, we suppose the region algebra has seven operators; four containment operators (, , , ) representing the containment relation between the extents, two combination operators ( , ) corresponding to and and or operator of the boolean model, and ordering operator (Q) representing the order of words or structures in the texts. For convenience of explanation, we represent a query as a tree structure as
Figure 1: Subqueries of the query [book] ([title] retrieval)
shown in Figure 1 1 . This query represents Retrieve the books whose title has the word retrieval. Our model assigns a relevance measure of the structured query as a vector of relevance measures of the subqueries. In other words, the relevance is dened by the number of portions matched with subqueries in a document. If an extent matches a subquery of query q, the extent will be somewhat relevant to q even when the extent does not exactly match q. Figure 1 shows an example of a query and its subqueries. In this example, even when an extent does not match the whole query exactly, if the extent matches retrieval or [title]retrieval, the extent is considered to be relevant to the query. Subqueries are formally dened as following. Denition 1 (Subquery) Let q be a given query and n1 , ..., nm be the nodes of q. Subqueries q1 , ..., qm of q are the subtrees of q. Each qi has node ni as a root node. When a relevance (qi , d) between a subquery qi and a document d is given, the relevance of the whole query is dened as following. Denition 2 (Relevance of the whole query) Let q be a given query, d be a document and q1 , ..., qm subqueries of q. The relevance vector (q, d) of d is dened as follows: (q, d) = (q1 , d), (q2 , d), ..., (qm , d) A relevance of a subquery should be dened similarly to that of keyword-based queries in the traditional ranked retrieval. For example, TFIDF, which is used in our experiments in Section 3, is the most simple and straightforward one, while other relevance measures recently proposed in (Robertson and Walker, 2000) can be applied. TF value is calculated using the number of extents matching the subquery, and IDF value is calculated using the number of documents including the extents matching the subquery. While we have dened a relevance of the structured query as a vector, we need to sort the documents according to the relevance vectors. In this paper, we rst map a vector into a scalar value, and then sort the documents
1
In this query, [x] is a syntax sugar of x Q /x .
C%7#$514A21B " 8 6 # v q
#" # 0)(' &$" ED3
t q
C86$54@21B " # 1 #
r s %#$" '(3 w x
%#$" 00F#$" ! ! 3
A0 3
`@XgfeU dba R V c V u q #$" ! y q 9087$5432 " 6 # 1 # 1
0#)(' &%$" ! " #
A x
p q ` X R V YSWU T R P H FSQIG %@3 h i
according to this scalar measure. Three methods are introduced for the mapping from the relevance vector to the scalar measure. The rst one simply works out the sum of the elements of the relevance vector. Denition 3 (Simple Sum)
m
sum (q, d) =
i=1
(qi , d)
The second represents the rareness of the structures. When the query is A B or A B, if the number of extents matching the query is close to the number of extents matching A, matching the query does not seem to be very important because it means that the extents that match A mostly match A B or A B. The case of the other operators is the same as with and . Denition 4 (Structure Coefcient) When the operator op is , or Q, the structure coefcient of the query A op B is: scAopB = C(A) + C(B) C(A op B) C(A) + C(B)
and when the operator op is or , the structure coefcient of the query A op B is: scAopB = C(A) C(A op B) C(A)
where A and B are the queries and C(A) is the number of extents that match A in the document collection. The scalar measure sc (qi , d) is then dened as
m
sc (q, d) =
i=1
scqi (qi , d)
The third is a combination of the measure of the query itself and the measure of the subqueries. Although we calculate the score of extents by subqueries instead of using only the whole query, the score of subqueries can not be compared with the score of other subqueries. We assume normalized weight of each subquery and interpolate the weight of parent node and children nodes. Denition 5 (Interpolated Coefcient) The interpolated coefcient of the query qi is recursively dened as follows: ic (qci , d) l where ci is the child of node ni , l is the number of children of node ni , and 0 1. This formula means that the weight of each node is dened by a weighted average of the weight of the query and its subqueries. When = 1, the weight of each query is normalized weight of the query. When = 0, the weight of each query is calculated from the weight of the subqueries, i.e. the weight is calculated by only the weight of the words used in the query. ic (qi , d) = (qi , d) + (1 )
ci
1 2 3 4
([cons]([sem]G#DNA domain or region)) (inQ([cons]([sem](G#tissue G#body part)))) ([event]([obj]gene)) (inQ([cons]([sem](G#tissue G#body part)))) ([event]([obj]Q([sem]G#DNA domain or region))) (inQ([cons]([sem](G#tissue G#body part)))) ([event]([dummy]G#DNA domain or region)) (inQ([cons]([sem](G#tissue G#body part))))
Table 1: Queries submitted in the experiments
3
Experiments
In this section, we show the results of our preliminary experiments of text retrieval using our model. Because there is no test collection of the structured query and tagannotated text, we used the GENIA corpus (Ohta et al., 2002) as a structured text, which was an XML document composed of paper abstracts in the eld of biomedical science. The corpus consisted of 1,990 articles, 873,087 words (including tags), and 16,391 sentences. We compared three retrieval models, i) our model, ii) exact matching of the region algebra (exact), and iii) not-structured at model. In the at model, the query was submitted as a query composed of the words in the queries in Table 1 connected by the and operator ( ). The queries submitted to our system are shown in Table 1, and the document was sentence represented by sentence tags. Query 1, 2, and 3 are real queries made by an expert in the eld of biomedicine. Query 4 is a toy query made by us to see the robustness compared with the exact model easily. The system output the ten results that had the highest relevance for each model2 . Table 2 shows the number of the results that were judged relevant in the top ten results when the ranking was done using sum . The results show that our model w...