Transcript Gaussian Process Tutorial

Daphne’s
Approximate
Group of
Students
Outline

Linear Regression






Unregularized
L2 Regularized
What is a GP?
Prediction with a GP
Relationship to SVM
Implications

What does this mean?
Linear Regression


Predicting Y given X
Y = wtx + n


w_ml = argmax
y[m+1] = w_mltx[m+1]
L2 Regularized Lin Reg

L2 Regularized (Gaussian Prior on w)



Y = wtx + n
w ~ N(0,S)
w_map = argmax blah + ||w||^2
What is a random process?

It’s a prior over functions
What is a Gaussian Process?


It’s a prior over functions that generalized a Gaussian
Random Vector
Prior over Y(x) ~ N(0,I)
Alternate Definition

The thing with Euler’s equation
This is weird


Not used to thinking of prior over Ys
Or are we?


We ARE used to thining about prior over w
What prior over y does this induce
Math P(w) -> P(Y)

Wow! This became a Gaussian Process!
Prediction with a GP

Predict y*[m+1] given y[1]…y[m]


We get a covariance = error bars
Wow! This prediction is the same as w_map but
we get error bars!
Generalize that shit - Covariance
Functions


Note that we have a thing here that is defined by
C(x1,x2) which can be kernelized
Has to be pos semidefinite

Is a kernel function
Relationship to SVM
Example
How do we reconcile these
views?

Does this change anything?