Transcript Machine Learning Introduction

Machine Learning
Introduction
Study on the Coursera
All Right Reserved : Andrew Ng
Lecturer:Much
Database Lab of Xiamen University
Aug 12,2014
• Machine Learning
- Grew out of work in AI(Artificial Intelligence)
- New capability for computers
• Examples:
- Database mining
• Large datasets from growth of automation/web.
• Web click data, medical records, biology, engineering
- Applications can’t program by hand.
• Handwriting recognition, most of Natural Language
Processing (NLP), Computer Vision.
Machine Learning Definition
• Tom Mitchell (1998) Well-posed Learning
Problem:
A computer program is said to learn from
experience E with respect to some task T
and some performance measure P, if its
performance on T, as measured by P,
improves with experience E.
• Suppose your email program watches which
emails you do or do not mark as spam, and
based on that learns how to better filter spam.
What is the task T in this setting?
T : Classifying emails as spam or not spam
E : Watching you label emails as spam or not spam
P: The number of emails correctly classified as spam/not
spam
“A computer program is said to learn from experience E with respect
to some task T and some performance measure P, if its performance
on T, as measured by P, improves with experience E.”
Machine Learning Algorithms
- Supervised learning
- Unsupervised learning
- Others:
- Reinforcement learning
- Recommender systems.
Supervised Learning & Unsupervised Learning
x2
x1
Supervised Learning
Unsupervised Learning
Linear Regression with one
Variable
Housing Prices
(Portland, OR)
Price
(in 1000s
of dollars)
Size (feet2)
Supervised Learning
Regression Problem
Given the “right answer” for
each example in the data.
Predict real-valued output
Training set of
housing prices
Size in feet2 (x)
2104
1416
1534
852
…
Price ($) in 1000's (y)
460
232
315
178
…
Notation:
Training Set
m = Number of training examples
Learning Algorithm
x’s = “input” variable / features
y’s = “output” variable / “target” variable
Size of
house
Question : How to describe h?
h
Estimated
price
Size in feet2 (x)
2104
1416
1534
852
…
Training Set
Hypothesis:
‘s:
Parameters
How to choose
‘s ?
Price ($) in 1000's (y)
460
232
315
178
…
y
x
Idea: Choose
so that
is close to for our
training examples
Cost Function
Hypothesis:
Parameters:
Cost Function:
Goal:
Simplified:
Price
($)
in
1000’s
Size in feet2 (x)
Question:How to minimize J?
Gradient Descent
Have some function
Want
Outline:
• Start with some
• Keep changing
to reduce
until we hopefully end up at a minimum
Gradient descent algorithm
Correct: Simultaneous update
Incorrect:
Gradient descent algorithm
Notice : α is the learning rate.
If α is too small, gradient descent
can be slow.
If α is too large, gradient descent
can overshoot the minimum. It may
fail to converge, or even diverge.
at local optima
Current value of
Unchange
Gradient descent can converge to a local minimum, even with the
learning rate α fixed.
As we approach a local minimum, gradient descent will
automatically take smaller steps. So, no need to decrease α over
time.
Gradient Descent for
Linear Regression
Gradient descent algorithm
Linear Regression Model
Gradient descent algorithm
update
and
simultaneously
J(0,1)
1
0
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
(for fixed
, this is a function of x)
(function of the parameters
)
Linear Regression with multiple
variables
Hypothesis:
Parameters:
Cost function:
Gradient descent:
Repeat
(simultaneously update for every
)
New algorithm
Repeat
Gradient Descent
:
Previously (n=1):
Repeat
(simultaneously update
)
(simultaneously update
)
for
Examples:
1
1
1
1
Size (feet2)
Number of
bedrooms
Number of
floors
Age of home
(years)
Price ($1000)
2104
1416
1534
852
5
3
3
2
1
2
2
1
45
40
30
36
460
232
315
178
simultaneously update
Summarize
• This is a briefly Introduction about Supervised
Learning(Classification)in Machine Leaning.
• There is still a lot of things in this subject,such as
Clustering,
Support Vector Machine(SVM),
Dimensionality Reduction,
ETC.
The Core Idea of MS is very similar,hope you will
be fond of the Machine Learning！
Thanks for Listening !