Transcript Introduction to Engi..

Introduction to
Engineering
Fall 2006
Lecture 15: Probability 2
1
Review

Introduction to Probability

Probability in Engineering

Probability Puzzles
2
Review - Applications

Probability is used throughout engineering which may appear strange
because engineering should be an exact science.

Examples of Probabilities in Engineering





Thermal noise in electrical circuits
Detection of weak radio and radar signals
Information theory
Communication systems design
Reliability of systems




Networks and Systems Problems




Failure probabilities
Failure rates
Mean time to failure
Random arrivals of packets/jobs
Probability of buffer or queue overflow
Scheduling problems, priorities, QOS
Flow control and routing
3
Outline

Probability Part 2

Bayes Rule

Using MatLab
4
Probability (part 2)
5
Probability

DEFINITION: The probability of an event is
the ratio of the number of cases in which the
event occurs to the total number of possible
cases
Number of desired outcomes
P(outcome) = Number of possible outcomes
EXAMPLE
The probability of drawing a diamond out of
a deck of cards is:
P(diamond) =
13
52
6
Problems with Probability 1

Based on past experimental results, we can use the observed relative
frequency as an estimate of the probability of an event





Estimates are usually reasonably good if the number of trials was large
On N trials, the relative frequency that we observe might easily differ from
the true probability by as much as N–1/2 or more
For a sample space with countably infinite outcomes, at most N different
outcomes will be observed on N trials
The probabilities to be assigned to the infinitely many unobserved
outcomes is again a matter of guesswork!
Suppose that a fair coin is tossed a million times



Is there a logical reason why the coin will not turn up Heads each and
every time?
No, there is no logical reason why it couldn’t, but it is very unlikely to
do so
Yes, if the coin is fair, there is no way that it can turn up Heads a
million times in a row
7
Problems with Probability 2

A fair coin that is tossed a million times will not turn up Heads a million times
in a row?







Observed relative frequencies can serve only as estimates of probabilities


What is the largest number of consecutive Heads that you think can possibly
occur?
How does the coin know that you are watching, and therefore, it should turn up
Tails after a certain number of Heads?
A million Heads in a row might later be followed by a million Tails to give a relative
frequency of 1/2 for Heads
If only we had gone one for another million tosses, we would have been OK …
No guarantee that an observed relative frequency is close to the actual probability
We expect the relative frequency is close, but we cannot guarantee it
No guarantee that an observed relative frequency is close to the actual
probability, only a strong expectation
“What does probability mean?” It is a numerical expression of strength of
belief
8
Real Probabilities

Consider the following situation:
Clyde and George are betting on the value
of the top card of a deck of 52 cards
Clyde sneaks a quick peak without George
knowing and is able to tell that it is a king but
can not discern the suit
What is the probability of the king of spades to
Clyde?
1
4
What is the probability of the king of spades to
George?
1
52
What is the real
probability?
Probabilities exist only
relative to
information.
9
Probability Values

Probability values are often found from
experimental observation.

For example, given the following data:
Females
Liberal Arts Major
Non-LA Major
Both
3
1
4
Males
Both
0
6
6
3
7
10
What is the probability that a person selected
at random is a liberal arts major?
30%
10
Conditional Probabilities

Conditional probabilities are probability
values based on knowledge
EXAMPLE
What is the probability of
selecting a liberal arts major
from among the pool of males?
What is the probability of selecting a
liberal arts major from among the
pool of females?
P(L | M) = 0/6 = 0%
P(L | F) = 3/4 = 75%
Females
Liberal Arts Major
Non-LA Major
Both
3
1
4
Males
Both
0
6
6
3
7
10
11
Definition

Given two events A and B, the conditional
probability of A given that B has occurred is
written as P(A | B) and is defined as
(assuming P(B) > 0):
P(A B)
P(A | B) =
P(B)
12
Example One

Problem: Weather station records in a
certain region give the probability of a warm,
clear day in January as 0.15 and the
probability of a clear day in January as 0.6.

You look out your window on a January
morning and see that the day is clear. What
are the chances that the day will be warm?
P(warm | clear) =
.15
.6
= .25
13
Example Two

You throw a pair of dice - what is the probability
that the value of the first die, X, will be greater
than the value of the second, Y?
Define the probability of event A (X > Y)
for all possible values of X:
S P(Y<X | X=i) P(X=i)
6
P(A) =
i=1
S S P(Y=j) (.1667) = 5/12
6
P(A) =
i-1
i=1 j=1
14
Conditional Probability Rules

Mathematical basis of probability:
General multiplication rule of probability:
For any two events A and B with P(B) > 0
P(A B) = P(A|B)P(B)
Law of inflating probabilities:
P(A) = P(A B) + P(A B)
P(A) = P(A|B)P(B) + P(A|B)P(B)
Multiplication rule for independent events:
P(A B) = P(A) P(B)
15
Bayes Rule
16
Bayes Rule

Bayes rule gives a relationship between P(A |
B) and P(B | A):
Likelihood of A given B is true
New belief in B
P(A | B)P(B)
P(B | A) =
P(A)
Normalizing
constant
Previous belief about B
This rule is useful for those cases where P(A|B) can be
estimated but P(B|A) is hard to find experimentally
17
Example 1

A person at the next gambling table declares
the outcome “Twelve” and we wish to know
whether he was rolling a pair of dice or spinning
a roulette wheel.

We know the following:
P(12 | Dice) = 1/36
P(12 | Roulette) = 1/38
There are 10 tables, 6 dice and 4 roulette, so
P(Dice) = .6 and P(roulette) = .4
P(12 | Dice)P(Dice)
P(Dice | 12) =
P(12 | Dice)P(Dice)+P(12|Dice)P(Dice)
(.028)(.6)
=
= .618
(.028)(.6)+(.026)(.4)
18
Example 2

Let J be the event “defendant found guilty by
a jury” and G be the event “defendant is in
fact guilty”

If we know P(J | ~G) can we find P(~G|J)?
P(G|J) =
P(G)P(J|G)
P(G)P(J|G) + P(G)P(J|G)
(0.1)(0.01)
=
(0.1)(0.01)+(0.9)(0.95)
P(G) = 0.9
P(J|G) = 0.01
= .00117
P(J|G)=0.05
19
Odds Likelihood Ratio

Given a probability value P(E), the odds of E,
O(E) are defined by:
O(E) =
P(E)
1-P(E)
If A is the premise in a rule and H is the
conclusion, the likelihood ratios of A are
defined as:
P(A|H)
P(A|H)
l=
l=
P(A|H)
P(A|H)
20
Bayes Rule (Odds Form)

Take P(H|A) and P(~H|A) as given by Bayes
rule and divide them:
P(H|A)
P(A|H)P(H)
=
P(H|A)
P(A|H)P(H)
O(H|A)
l
O(H)
O(H|A) = lO(H)
21
Example

Situation: You are awakened one night by
the shrill sound of your burglar alarm

Facts:
There is a 95% chance that an attempted burglary
will trigger the alarm system
P(Alarm|Burglary) = .95
Previous crime patterns indicate that there is a one in
ten thousand chance that a given house will be
burglarized on a given night
Based on previous false alarms, there is a slight (1%)
chance that the alarm will be triggered by a mechanism
other than an attempted burglary
P(Burglary) = .0001
P(Alarm|No Burglary) = .01
22
Example (continued)

Question: What is your degree of belief that
a burglary attempt has taken place?

Using the odds format of Bayes Rule:
Since
O(Burglary|Alarm) = l(Alarm|Burglary)O(Burglary)
= 0.95 0.0001
0.01 1-0.001
O(A)
P(A) = 1+O(A)
0.0095
P(Burglary|Alarm) =
1+0.0095
What is the impact?
The probability of a burglary
is up by a factor of 100
23
Example
1 in every 10000 people in Ireland suffer from AIDS
There is a test for HIV/AIDS which is 95% accurate.
You are not feeling well and you go to hospital where
your Physician tests you.
He says you are positive for AIDS and tells you that
you have 18 months to live.
How should you react?
24
Answer


Let D be the event that you have AIDS
Let T be the event that you test positive for
AIDS
P(T|D)P(D)
P(D|T) =
P(T|D)P(D) + P(T|D)P(D)
=
0.95*0.0001
0.95*0.0001 + 0.05*0.9999
= .0189
25
Using MatLab
26
Bayes Rule Program
27
Program Run
28
Possible Quiz

Remember that even though each quiz is
worth only 5 to 10 points, the points do add
up to a significant contribution to your overall
grade

If there is a quiz it might cover these issues:



Define probability.
What is Conditional probability?
What is Bayes Rule?
29