Web Enhanced Page Classification
Xiaoguang Qi
Background Utilizing features of neighbors
Using fielded features
Problem definition
Classification A set of labeled data is used to train a classifier which can be applied to label future examples.
Web page classification The process of assigning a web page to one or more predefined category labels. aka. web page categorization
Why important?
Web page classification is essential to Improving quality of retrieval Maintaining web directories Helping question answering systems Many more ...
Why important? (Cont.)
Tradition text classification approaches don't perform well on web pages An experiment on dmoz ODP dataset
12 topical categories 228,000 training documents, 12,000 for testing
Na Bayes: 35% error rate ve Support vector machine: 27% error rate
Text is not enough
Text is not enough (Cont.)
Textual features could be missing, misleading, or unrecognizable.
Web page temporarily unavailable, robot exclusion,
picture, flash, frame, etc ...
There could be too much irrelevant text.
Advertisement, navigational panel, spam follow-ups,
etc ...
Solution?
Besides text, web pages have other features. Make use of them!
On-page features
On-page features (Cont.)
Using on-page features
HTML tags structural info embedded in HTML documents Golub and Ardo [2005]
Assign significance indicators for different HTML tags Title, headings, metadata, and main text
Kwon and Lee [2000, 2003]
Divide all the HTML tags into three groups Assign each group an arbitrary weight
Using on-page features (Cont.)
Summarize then classify Classify web pages based on their summarization [Shen et al., 2004].
URL A web page can be reasonably classified just based on its URL! [Kan, 2004; Kan and Thi, 2005]
Using on-page features (Cont.)
Visual analysis Most approaches focus on text, ignoring visual info. Sometimes, it might be more expensive than using text. Analyze a web page's visual layout, represent the recognized parts in an adjacency graph, apply heuristic rules on the graph [Kovacevic et al., 2004].
Sibling
?
Parent Target page Child
Spouse
Using features of neighbors
Directly incorporate text from parent or child into the target page It does more harm than good [Chakrabarti et al., 1998; Ghani et
al., 2001; Yang et al., 2002].
Parent and child pages == useless? No! We can make it useful! Using a subset of parent/child pages (not all of them) Using a portion of content of parent/child pages (not full content)
Using features of neighbors (Cont.)
Use a neighbor's feature only if its content is similar enough to the target page [Oh et al., 2000]. Use anchor text, the surrounding text of anchor text
[Attardi et al., 1999; Furnkranz, 1999; 2001; Sun et al., 2002; Glover et al., 2002].
Sibling pages are more useful than parent/child pages
[Chakrabarti et al., 1998; Slattery & Mitchell, 2000; Qi & Davison, 2006].
Using features of neighbors (Cont.)
Labels [Chakrabarti et al., 1998; Slattery & Mitchell, 2000; Calado et
al., 2003]
Human-generated, accurate Only a small portion of the web is labeled
Partial content [Glover et al., 2002; Cohen 2002]
Full content
Using artificial links
Besides hyperlinks, we can create other types of links! Content similarity [Kurland and Lee, 2005; 2006] Pages co-occur in top results of a query Co-occur + clicked [Shen et al., 2006] ...
Background Utilizing features of neighbors
Using fielded features
Outline of Neighboring algorithm
Four types of neighbors are used.
Parent, child, sibling, spouse.
Information is linearly combined.
Weights are adjusted through experiments.
Both page content and human labeling are
considered. Human labeling is used whenever available.
A web page and its neighbors
Sibling
?
Parent Target page Child
Spouse
Topic distribution vectors
Arts Business Sports
(0.7, 0.2, 0.1)
A probability distribution vector v p (v p,1 , v p, 2 ,, v p,i ,, v p,n ) Each component v p ,i is the normalized probability of the page p being in the category ci
v4
v1
v5
Sibling
v9
v1 0
v2
Parent
v0
?
v1 4
v1 5
Child
v3
v 6 v7 v8
Target page
v1 1
Spouse
v1 6
v1 2 v1 3
Neighboring pages
Target page
......
Page level weighting
Grouping Parent Child Sibling Spouse
Group level weighting Topic vector of neighbors Weighting between target page and neighbors Final topic vector
Bias on labeled pages
A (1,0,0) A Sibling B A
(1,0,0)
A Parent
?
Target page Child
(0,0,1) A web directory
C
A Spouse
......
Bias on labeled pages (Cont.)
Relative contributions of labeled pages and those unlabeled. Any classifier produces an approximation to the desired human labeling. Use human judgment whenever it is available. Decisions of classifiers are down-weighted.
v p ' v p * w( p) if p is labeled 1 where w( p) if p is not labeled (0 1)
Counting the multiple paths
Sibling
?
Parent Target page Child
Spouse
Tuning the page level bias
0.95 0.90 Weighted path Unweighted path
Accuracy
0.85 0.75 0.80 0.70 0.65 0.60 0 0.5 1 1.5 2
2.5
3
3.5
4
Weighted path works better than unweighted path. if 1 Human labeling is important. p is labeled v p ' v p * w( p) where w( p) (0 1) if p is not labeled
Neighboring pages
Target page
......
Page level weighting
Grouping Parent Child Sibling Spouse
Group level weighting Topic vector of neighbors Weighting between target page and neighbors Final topic vector
Bias among neighboring groups
Parent Child Sibling Spouse
*
*
1 2 3
*
4
*
vp" pParent
pParent
+
vneighbors
vneighbors 1
3 0 i 1,
pSibling w( p)
1, i 1,2,3,4
w( p) vp" pSibling
2
vp" pChild
pChild
4
w( p) vp" pSpouse
pSpouse
w( p)
i 1
4
i
Individual contribution of the four types of neighbors
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Parent Child Sibling Spouse
Accuracy
Sibling pages contribute the most.
Weight of sibling pages
0.92 0.90
Accuracy
0.88 0.86 0.84 0.82 0.80 0.3 0.4 0.5
(0.2, 0.2, 0.4, 0.2)
Parent
0.6 Weight of sibling
Child
0.7
0.8
Sibling
0.9
Spouse
1
Combining neighbors with target page
A combined vector: a weighted average of the topic vector of the
target page and that of the neighboring pages
The category corresponding to the major component is chosen as
the classification result
v vtarget (1 ) vneighbors, where 0 1
Topic vector of the target page: Computed by a textual classifier Aggregated topic vector of the neighboring pages
: combining the neighbors with the target page
Experimental setup
Dataset
Open Directory Project (ODP) A web directory, September 2004, contains 0.6 million categories, 4.4 million web pages. We used 12 out of the 17 top-level categories. Training: 19,000 documents * 12 categories Tuning: 500 documents * 12 categories Test: 500 documents * 12 categories Three disjoint sets of documents randomly
selected from ODP dataset Plus 6.5 million neighboring documents
Experimental setup (Cont.)
Textual classifiers
A Na bayes classifier (Rainbow) ve
A SVM classifier ( SVM Light )
Experimental result of labeled ODP dataset
0.95 0.90 0.85
Accuracy
0.80 0.75 0.70 0.65 0.60
Na ve bayes SVM K+C IO-bridge Neighboring Neighboring + NB + SVM
Textual classifiers ~70%
Prior approaches <80%
Neighboring Alg. ~90%
With and without human labeling
0.95 0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5 Na Bayes ve Neighboring (without labels) Neighboring (with labels)
Neighboring alg. still has a fairly good performance w/o labeling. Prior approaches can't perform well.
Accuracy
Section summary
Our Neighboring algorithm
is able to improve the accuracy of common textual
classifiers from 70% to more than 90%. can improve accuracy even when neighboring pages are unlabeled.
While all neighbor types can contribute, sibling
pages are the most important.
Background Utilizing features of neighbors
Using fielded features
Motivation
Most existing classification work uses web page as a whole.
Should all the content on a web page be treated the same? In retrieval, BM25f [Robertson et al., 2004] is a successful fielded extension to BM25 [Robertson & Walker., 1994]. In classification, information from different parts of web pages should also have different importance. E.g., anchor text should be of extra value.
F-Neighbor Algorithm
A fielded extension to Neighboring Algorithm. Break up web pages (target page, as well as its neighbors) into text fields. Combine them linearly with different weights. Perform classification on the combined representation.
Text fields
Title of the target page; Full text of the target page;
Titles of parent, child, sibling, and spouse pages (as four separate fields);
Full text of parent, child, sibling, and spouse pages;
Anchor text (referring to target) on parent page; Surrounding text of anchor text on parent pages (referred to as "extended anchor text").
Text representation
Each field is represented by a TFIDF vector .
d fi , i 1,, K
The combined representation of a target document is the weighted combination of all the associated fields.
d comb w fi d f i
i 1 K
where w fi 1
i 1
K
The weights will be determined through experiments.
Parameter tuning
Exploring the full K-dimensional parameter space is expensive!
Divide and conquer! Lower layer: tune weights within each neighbor type Upper layer: tune weights among neighbor types
Lower-layer optimization results
oEmphasizing titles
0.80 0.70 0.60 0.50
benefits all neighbor types.
oSignificant
Accuracy
0.40 0.30 0.20 0.10 0.00
improvement for parent, child, and spouse.
oMarginal benefit
target parent child sibling spouse
0
0.1
0.2 0.3
0.4
0.5 0.6
0.7 0.8
0.9
1
for target and siblings.
Weight of title
0.95 0.90 0.85
Accuracy
0.80 0.75 0.70 0.65 0.60 SVM IO-bridge Neighboring SVM F-Neighbor
Section summary
We can further improve performance based on Neighboring algorithm. Fielded information is useful in web page classification.