Unsupervised Transfer Classification

Download Report

Transcript Unsupervised Transfer Classification

Unsupervised Transfer Classification
Application to Text Categorization
Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong
Michigan State University
Overview



Introduction
Related Work
Unsupervised Transfer Classification
 Problem
Definition
 Approach & Analysis


Experiments
Conclusions
Introduction

Classification:
supervised learning
 semi-supervised learning

supervised

What if No label
information is available?

semi-supervised
impossible but not with some
additional information
unsupervised classification
Introduction

Unsupervised transfer classification (UTC)
a collection of training examples and their assignments
to auxiliary classes
 to build a classification model for a target class

auxiliary class 1
auxiliary class K
….
conditional probabilities
No Labeled
training examples
target class
prior
Introduction: Motivated Examples
Image Annotation
auxiliary
classes
grass
target
classes
Social Tagging
google phone verizon
apple
sky
sun
water
1
0
0
?
1
0
0
?
1
1
0
?
1
1
0
?
0
1
1
?
0
1
1
?
0
0
1
?
0
0
1
?
/
/
/
?
/
/
/
?
How to predict an annotation word/social tag that does not appear
in the training data ?
Related Work

Transfer Learning
 transfer
knowledge from source domain to target domain
 similarity: transfer label information for auxiliary classes
to target class
 difference: assume NO label information for target class

Multi-Label Learning, Maximum Entropy Model
Unsupervised Transfer Classification
Data for
auxiliary
class
Examples
assignments to auxiliary classes
Auxiliary Classes
Class
Information
Prior probability
Goal
target class
conditional probabilities
target class label
target classification model
Maximum Entropy Model (MaxEnt)
Favor uniform
distribution
Feature statistics computed
from conditional model
: the jth feature function
Feature statistics computed
from training data
Generalized MaxEnt
Equality constraints
With a large probability
Inequality constraints
Generalized MaxEnt
Generalized MaxEnt
is unknown for target class
How to extend generalized MaxEnt to
unsupervised transfer classification ?
Unsupervised Transfer Classification

Estimating feature statistics of target class from those
of the auxiliary classes
Unsupervised Transfer Classification

Build up Relation between Auxiliary Classes and
Target Class
Independence
Assumption
Unsupervised Transfer Classification

Estimating feature statistics for the target class
by regression
Feature Statistics for
Auxiliary Classes
Class
Information
Feature Statistics
for Target Class
Unsupervised Transfer Classification

Dual problem
: function of U; definition can be found in paper
Consistency Result
With a large probability
The optimal dual solution The dual solution
using the label information obtained by the
proposed approach
for the target class
Experiments




Text categorization
Data sets: multi-labeled data
Protocol: leave one-class out as the target class
Metric: AUC (Area under ROC curve)
Experiments: Baselines

cModel



cLabel



predict the assignment of the target class for training examples by
linearly combining the labels of auxiliary classes
train a classifier using the predicted labels for target class
GME-avg



train a classifier for each auxiliary class
linearly combine them for the target class
use generalized maxent model
compute the feature statistics for the target class by linearly combining
those for the auxiliary classes
Proposed Approach: GME-Reg
Experiment (I)

Estimate class information from training data
Experiment (I)


Estimate class information from training data
Compare to the classifier of the target class
learned by supervised learning
1500
2500
Experiment (II)


Obtain class information from external sources
Datasets: bibtex and delicious



bibsonomy www.bibsonomy.org/tagsbibtex
ACM DL www.portal.acm.orgbibtex
deli.cio.us www.delicious.com/tagdelicious
Experiment (II)

Comparison with Supervised Classification
650
1000~1200
Conclusions


A new problem: unsupervised transfer classification
A statistical framework for unsupervised transfer
classification
 based
on generalized maximum entropy
 robust estimate feature statistics for target class
 provable performance by consistency analysis

Future Work
 relax
independence assumption
 better estimation of feature statistics for target class
Thanks
Questions ?