Unsupervised Transfer Classification
Download
Report
Transcript Unsupervised Transfer Classification
Unsupervised Transfer Classification
Application to Text Categorization
Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong
Michigan State University
Overview
Introduction
Related Work
Unsupervised Transfer Classification
Problem
Definition
Approach & Analysis
Experiments
Conclusions
Introduction
Classification:
supervised learning
semi-supervised learning
supervised
What if No label
information is available?
semi-supervised
impossible but not with some
additional information
unsupervised classification
Introduction
Unsupervised transfer classification (UTC)
a collection of training examples and their assignments
to auxiliary classes
to build a classification model for a target class
auxiliary class 1
auxiliary class K
….
conditional probabilities
No Labeled
training examples
target class
prior
Introduction: Motivated Examples
Image Annotation
auxiliary
classes
grass
target
classes
Social Tagging
google phone verizon
apple
sky
sun
water
1
0
0
?
1
0
0
?
1
1
0
?
1
1
0
?
0
1
1
?
0
1
1
?
0
0
1
?
0
0
1
?
/
/
/
?
/
/
/
?
How to predict an annotation word/social tag that does not appear
in the training data ?
Related Work
Transfer Learning
transfer
knowledge from source domain to target domain
similarity: transfer label information for auxiliary classes
to target class
difference: assume NO label information for target class
Multi-Label Learning, Maximum Entropy Model
Unsupervised Transfer Classification
Data for
auxiliary
class
Examples
assignments to auxiliary classes
Auxiliary Classes
Class
Information
Prior probability
Goal
target class
conditional probabilities
target class label
target classification model
Maximum Entropy Model (MaxEnt)
Favor uniform
distribution
Feature statistics computed
from conditional model
: the jth feature function
Feature statistics computed
from training data
Generalized MaxEnt
Equality constraints
With a large probability
Inequality constraints
Generalized MaxEnt
Generalized MaxEnt
is unknown for target class
How to extend generalized MaxEnt to
unsupervised transfer classification ?
Unsupervised Transfer Classification
Estimating feature statistics of target class from those
of the auxiliary classes
Unsupervised Transfer Classification
Build up Relation between Auxiliary Classes and
Target Class
Independence
Assumption
Unsupervised Transfer Classification
Estimating feature statistics for the target class
by regression
Feature Statistics for
Auxiliary Classes
Class
Information
Feature Statistics
for Target Class
Unsupervised Transfer Classification
Dual problem
: function of U; definition can be found in paper
Consistency Result
With a large probability
The optimal dual solution The dual solution
using the label information obtained by the
proposed approach
for the target class
Experiments
Text categorization
Data sets: multi-labeled data
Protocol: leave one-class out as the target class
Metric: AUC (Area under ROC curve)
Experiments: Baselines
cModel
cLabel
predict the assignment of the target class for training examples by
linearly combining the labels of auxiliary classes
train a classifier using the predicted labels for target class
GME-avg
train a classifier for each auxiliary class
linearly combine them for the target class
use generalized maxent model
compute the feature statistics for the target class by linearly combining
those for the auxiliary classes
Proposed Approach: GME-Reg
Experiment (I)
Estimate class information from training data
Experiment (I)
Estimate class information from training data
Compare to the classifier of the target class
learned by supervised learning
1500
2500
Experiment (II)
Obtain class information from external sources
Datasets: bibtex and delicious
bibsonomy www.bibsonomy.org/tagsbibtex
ACM DL www.portal.acm.orgbibtex
deli.cio.us www.delicious.com/tagdelicious
Experiment (II)
Comparison with Supervised Classification
650
1000~1200
Conclusions
A new problem: unsupervised transfer classification
A statistical framework for unsupervised transfer
classification
based
on generalized maximum entropy
robust estimate feature statistics for target class
provable performance by consistency analysis
Future Work
relax
independence assumption
better estimation of feature statistics for target class
Thanks
Questions ?