Transcript week4

How to find estimators?
• There are two main methods for finding estimators:
1) Method of moments.
2) The method of Maximum likelihood.
• Sometimes the two methods will give the same estimator.
week 4
1
Method of Moments
• The method of moments is a very simple procedure for finding an
estimator for one or more parameters of a statistical model.
• It is one of the oldest methods for deriving point estimators.
• Recall: the k moment of a random variable is
 k  E X k .
These will very often be functions of the unknown parameters.
• The corresponding k sample moment is the average .
1 n k
mk   xi
n i 1
• The estimator based on the method of moments will be the solutions
to the equation μk = mk.
week 4
2
Examples
week 4
3
Maximum Likelihood Estimators
• In the likelihood function, different values of θ will attach different
probabilities to a particular observed sample.
• The likelihood function, L(θ | x1, …, xn), can be maximized over θ,
to give the parameter value that attaches the highest possible
probability to a particular observed sample.
• We can maximize the likelihood function to find an estimator of θ.
• This estimator is a statistics – it is a function of the sample data. It is
denoted by ˆ.
week 4
4
The log likelihood function
• l(θ) = ln(L(θ)) is the log likelihood function.
• Both the likelihood function and the log likelihood function have
their maximums at the same value of ˆ.
• It is often easier to maximize l(θ).
week 4
5
Examples
week 4
6
Important Comment
• Some MLE’s cannot be determined using calculus. This occurs
whenever the support is a function of the parameter θ.
• These are best solved by graphing the likelihood function.
• Example:
week 4
7
Properties of MLE
• The MLE is invariant, i.e., the MLE of g(θ) equal to the function g
evaluated at the MLE.
• Proof:
• Examples:
week 4
8