Transcript TransferLearning

Transfer Learning
•
•
•
•
Motivation and Types
Functional Transfer Learning
Representational Transfer Learning
References
Transfer Learning
• The goal is to transfer knowledge gathered from previous experience.
• Also called Inductive Transfer or Learning to Learn.
• Example: Invariant transformations across tasks.
Motivation Transfer Learning
Motivation for transfer learning
Similar to self-adaptation: once a predictive model is
built, there are reasons to believe the model will cease to
be valid at some point in time.
The difference is that now source and target domains can
be completely different.
Traditional Approach to Classification
DB1
Learning
System
DB2
Learning
System
DBn
Learning
System
Transfer Learning
DB1
DB2
Source
domain
DB new
Target
domain
Learning
System
Learning
System
Knowledge
Learning
System
Transfer Learning
Scenarios:
1. Labeling in a new domain is costly.
DB1 (labeled)
Classification of Cepheids
DB2 (unlabeled)
Classification of LPV
Transfer Learning
Scenarios:
2. Data is outdated. Model created with one survey but
a new survey is now available.
Survey 1
Learning
System
Survey 2
?
Types of Transfer Learning
Figure obtained from Brazdil, et. Al. Metalearning: Applications to Data Mining, Chapter 7, Springer, 2009.
Transfer Learning
•
•
•
•
Motivation and Types
Functional Transfer Learning
Representational Transfer Learning
References
Functional Transfer: Multitask Learning
Left
Output nodes
Internal nodes
Input nodes
Straight
Right
Functional Transfer in Neural Networks
Given example X, compute the output of every node until
we reach the output nodes:
Output nodes
Internal nodes
Input nodes
Example X
Compute sigmoid
function
Train in Parallel with Combined Architecture
Figure obtained from Brazdil, et. Al. Metalearning: Applications to Data Mining, Chapter 7, Springer, 2009.
Transfer Learning
•
•
•
•
Motivation and Types
Functional Transfer Learning
Representational Transfer Learning
References
Knowledge of Parameters
Assume prior distribution of parameters
Source
domain
Target
domain
Learn parameters and adjust prior distribution
Learn parameters using the source prior
distribution.
Assume Parameter Similarity
P(y|x) = P(x|y) P(y) / P(x)
Parameter Similarity
Task A  Parameter A
Task B 
Parameter B ~ A
Assume hyper-distribution with low variance.
Knowledge of Parameters
Find coefficients ws using SVMs
Find coefficients wT using SVMs
initializing the search with ws
Feature Transfer
Feature Transfer:
Source
domain
Target
domain
Shared representation across tasks
Minimize Loss-Function( y, f(x))
The minimization is done over multiple tasks (multiple regions on Mars).
Feature Transfer
Identify common
Features to all tasks
Meta-Searching for Problem Solvers
Coded divided into pieces
Add pieces of code from previous tasks
New Solution
Start a new solution from scratch
Exploitation: Maximize reward
vs
Exploration: Maximize long-term success.
Transfer Learning in Robotics
First Task
Learn to keep the ball away from the opponent.
Second Task
Learn to score the opponent.
Instance Transfer Learning
Instance Transfer:
Source
domain
Filter samples
New program called
TrAdaboost
Target
domain
Learning
System
Larger target dataset
Instance Transfer Learning
New program called
TrAdaboost
• The main idea is to have a methodology to deal with a changing distribution.
• Examples in the source domain that look as belonging to a diff. distribution
are discarded.
• Examples in the source domain that look similar to the target domain are added
to the training set.
Boosting
1
1
1
1
1
1
DB
1
1
1 2
1
1 1
1
2 2
Incorrectly classified
instances  increase weight
2
2
Boosting
1
1
1
2
1 11
1
DB
22
2
2
2
Combine all hypotheses to produce final weighted function:
w1 f1 + w2 f2 + … + wn fn
Automatic Instance Transfer
Boosting
Incorrectly classified
instances  decrease weight
Source
domain
Target
domain
Learning
System
(Boosting)
Incorrectly classified
instances  increase weight
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Automatic Instance Transfer
Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007
Transfer Learning
•
•
•
•
Motivation and Types
Functional Transfer Learning
Representational Transfer Learning
References
References
Sinno Jialin Pan and Qiang Yang. A Survey on Transfer Learning.
IEEE Transactions on Knowledge and Data Engineering,
22(10):1345-1359, Oct. 2010
Brazdil, P. et. al. Metalearning: Applications to Data Mining.
Springer, 2009.
Dai, W., Boosting for Transfer Learning, Proceedings of ICML 2007.
Video on transfer learning
http://www.youtube.com/watch?v=9ChVn3xVNDI&noredirect=1
References
Sinno Jialin Pan and Qiang Yang. A Survey on Transfer Learning.
IEEE Transactions on Knowledge and Data Engineering,
22(10):1345-1359, Oct. 2010
Brazdil, P. et. al. Metalearning: Applications to Data Mining.
Springer, 2009.
Dai, W., Boosting for Transfer Learning, Proceedings of ICML 2007.
Video on transfer learning
http://www.youtube.com/watch?v=9ChVn3xVNDI&noredirect=1
Videos
Robot learns to flip pancakes
http://www.youtube.com/watch?v=W_gxLKSsSIE&noredirect=1
Robot learns to stack pancakes
http://www.youtube.com/watch?v=v9oeOYMRvuQ