Transcript S1P2

EMIS 7300
SYSTEMS ANALYSIS METHODS
Spring 2006
Dr. John Lipp
Copyright © 2002 - 2005 Dr. John Lipp
Session 1 Outline
• Part 1: Decision Theory I.
• Part 2: Probability Theory.
• Part 3: Decision Theory II.
• Part 4: Utility Theory.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-2
Today’s Session Topics
• Part 2: Probability Theory
–
–
–
–
–
–
–
–
–
–
EMIS 7300
Random Phenomena
Outcomes, Events, Sample spaces, and Venn Diagrams
Partitions
Probability Axioms
Joint Probability
Conditional Probability
Multiplication Rule
Total Probability Rule
Bayes’ Theorem
Statistical Independence
Copyright  2002 - 2006 Dr. John Lipp
S1P2-3
Random Phenomena
• When a phenomenon occurs that can have unpredictable
results under “controlled” conditions, it is denoted as a
random experiment or trial. The result is denoted as an event
or outcome of the experiment (or trial).
• The terms experiment and trail are used interchangeably.
However, event and outcome have specific meanings in the
context of probability theory.
– An outcome (elementary outcome) is the smallest unit or
division that can result from a random experiment.
– An event is a set of outcomes.
– The event containing all possible outcomes, often denoted
S, is called the sample space of the experiment.
– All events are subsets of S.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-4
Random Phenomena (cont.)
• Mathematically speaking, set theory describes outcomes and
events. Set illustrations are called Venn diagrams.
– Subset (“Events”):
A  S, B  S
– Set compliment (“Not A”):
Ac
– Set union (“A or B”):
AB
– Set intersection (“A and B”):
AB
– DeMorgan’s Law:
(A  B) c = Ac  Bc
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
A
B
S
A
Ac
S
A
B
S
A
B
S
A
B
S
S1P2-5
Sample
Space S
Random Phenomena (cont.)
• Example: Rolling a die
– Outcomes: 1,2,3,4,5, or 6 pips.
1 2 3 4 5 6
– Events:
A: Roll an even number
1
2
3
4
5
6
B: Roll greater than three
1
2
3
4
5
6
A and B (A  B)
1
2
3
4
5
6
not A, or B (Ac  B)
1
2
3
4
5
6
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-6
Random Phenomena (cont.)
•
The “null” or “empty” event is denoted . Note Sc = .
•
If A and B are events and A  B =  then they are referred to
as mutually exclusive (or disjoint) events.
A
B
S
•
By definition, all outcomes are mutually exclusive.
•
A set of events {Ai: i = 1,…,M} are mutually exclusive (or
pair-wise disjoint events) if Ai  Aj =  when i = j.
– Pair-wise disjoint implies triplet-wise disjoint, etc.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-7
Random Phenomena (cont.)
• If the set of events {Ai: i = 1,…,M} are mutually exclusive
and
M
A1  A2  A3    AM   Ai  S
i 1
then the {Ai: i = 1,…,M} is called a partition of S.
• The set containing all (elementary) outcomes is a partition.
• Question: Can a partition be infinite in size?
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-8
Probability
• The probability of an outcome or event is the relative
frequency of that outcome or event.
• That is, the ratio of the number of occurrences to the number
of trials as the number of trails grows larger and larger.
• The probability of an outcome or event is written as P(E).
• In the simplest case, where all outcomes are equally likely,
P(E) = 1/N, where N is the number of outcomes in S.
Example: Over time, an equal number of heads and tails are
expected to occur flipping a coin. That is, P(H) = P(T) = 1/2.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-9
Probability Axioms
1. Probability of S is 1, that is, P(S) = 1.
“The probability of some outcome is certain.”
2. Probability is bounded, 0  P(E)  1.
3. If A and B are disjoint events then
P( A  B)  P( A)  P( B)
– Note that  is disjoint from every other event and thus
P() = 0.
“The probability that nothing happens is impossible.”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-10
Probability Axioms (cont.)
• If the set of events {Ai: i = 1,…,} are mutually exclusive
then

P ( A1  A2  A3    A )   P ( Ai )
i 1
• The probability of an event A or its compliment Ac happening
is unity.
P( A  Ac )  P( A)  P( Ac )  1
A
EMIS 7300
Ac
Copyright  2002 - 2006 Dr. John Lipp
S1P2-11
Probability Axioms (cont.)
•
What happens if two events A and B are not disjoint?
S
B
A
•
Divide B with the partition {A, Ac}
S
A
B-A = B  Ac
BA
Then
and
EMIS 7300
P( B)  P( B  A)  P( B  A)
P( A  B)  P( A)  P( B  A)
 P( A)  P( B)  P( A  B)
Copyright  2002 - 2006 Dr. John Lipp
S1P2-12
Joint Probability
• In practice experiments do not happen in isolation. A random
phenomenon can encompass several co-occurring events.
• Example: Consider a drug test with two defined outcomes,
Pass or Fail. The drug test is uninteresting until applied to a
testee. The testee may be a drug user, or may be drug free.
• The events of each experiment in joint probability are
assigned a letter and subscript. The letter specifies the
experiment, and the subscript an event.
• Example: Denote the outcome of the drug test as A1 = “pass”
and A2 = “fail”, and the testee’s status as B1 = “drug free” and
B2 = “drug user.”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-13
Joint Probability (cont.)
SA
A1 = “Pass”
A2 =
“Fail”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-14
Joint Probability (cont.)
SB
B1 = “Drug Free”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
B2 = “Drug User”
S1P2-15
Joint Probability (cont.)
S = SA  SB
A1, B1 = “Pass, Drug Free”
A2, B1 =
“Fail,
Drug
Free”
EMIS 7300
A1, B2 = “Pass, Drug User”
A2, B2 = “Fail, Drug User”
S1P2-16
Copyright  2002 - 2006 Dr. John Lipp
Joint Probability (cont.)
• The joint probability of two events is the probability of the
first event happening while simultaneously (“and”) a second
event also happens. The joint probability for two events is
written P(A  B), or (more commonly) as P(A, B).
Example: The joint probabilities for the drug test and testee are
P(A1, B1)
“Drug test passed by drug free”
P(A2, B1)
“Drug test failed by drug free, aka, false alarm”
P(A1, B2)
“Drug test passed by drug user, aka, miss”
P(A2, B2)
“Drug test failed by drug user, aka, detection”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-17
Conditional Probability
• Consider two experiments and their joint probability. The
event from the first experiment is measured and therefore
known. It stands to reason that the event probabilities of the
remaining experiment have changed in light of the first
experiment’s results.
• Let A be the measured (given) event, and B the unknown
event. The probability of B given A is written as P(B|A) and is
known as conditional probability.
Example: If a legal battle over firing or non-hiring were to ensue
over a drug test, what would be the probability of interest? It
would be P(B2|A2), that is, the probability of being a drug user
if a drug test is failed.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-18
Joint Probability (cont.)
S = SA  SB
A1, B1 = “Pass, Drug Free”
A2, B1 =
“Fail,
Drug
Free”
EMIS 7300
A1, B2 = “Pass, Drug User”
A2, B2 = “Fail, Drug User”
S1P2-19
Copyright  2002 - 2006 Dr. John Lipp
Multiplication Rule
• An important relationship exists between joint and conditional
probability called the multiplication rule.
P( A, B)  P( B | A) P( A)
• The multiplication rule can be written both directions
P( A, B)  P( B | A) P( A)  P( A | B) P( B)
which, when P(A) > 0, is more usefully written as,
P( A | B) P( B)
P( B | A) 
P( A)
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-20
Conditional Probability (cont.)
Example: Using the above to rewrite P(B2|A2),
P( A2 | B2 ) P( B2 )
P( B2 | A2 ) 
P( A2 )
• P(A2 |B2) = probability of failing a drug test if a drug user (can
be determined by controlled experiments) = 99% effective.
• P(B2) = probability of being a drug user (can be hard to
determine, will cover next) = 2% “general bad apples,” and
• P(A2) = probability of drug test failure (can be obtained from
the history of doing drug testing) = 2.96% failure rate.
(0.99)(0.02)
P( B2 | A2 ) 
 0.67
(0.0296)
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-21
Total Probability Rule
• If {Ai: i = 1, …, M} is a partition then
M
P ( B)  P ( B, S A )   P( B | Ai ) P( Ai )
i 1
Example: From the drug test, P(A2) was given as 2.96%. That
number was computed since B1 and B2 form a partition, i.e.,
P( A2 )  P( A2 | B1 ) P( B1 )  P( A2 | B2 ) P( B2 )
 1  P( A1 | B1 ) 1  P( B2 )   P( A2 | B2 ) P( B2 )
 (1.00  0.99)(1  0.02)  (0.99)(0.02)  0.0296
More realistic would be to know P(A2) and from that compute
P(B2) and P(B1) = 1 – P(B2).
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-22
Total Probability Rule (cont.)
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-23
Bayes’ Theorem
• Combining a partition with the multiplication rule is known as
Bayes’ Theorem (provided P(B) > 0),
P ( Ai | B ) 
P ( B | Ai ) P ( Ai )
N
 P( B | Ai ) P( Ai )
i 1
Example: Consider the case of passing a drug test. What is the
probability of being drug free?
P( A1 | B1 )1  P( B2 ) 
P( A1 | B1 )1  P( B2 )   1  P ( A2 | B2 ) P ( B2 )
(0.99)(1  0.02)

 0.9998
(0.99) (1  0.02)  (1  0.99) (0.02)
P( B1 | A1 ) 
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-24
Statistical Independence
• Consider the outcome of flipping a coin vs. today’s price for
Lockheed Martin stock. Both are random events. But do they
have anything to do with each other? Not likely. These two
events are independent from each other.
• In terms of probability, events are independent if and only if
P( A, B)  P( A) P( B)
• Note the impact of independence on conditional probability
P( A, B) P( A) P( B)
P( A | B) 

 P( A)
P( B)
P( B)
P( A, B) P( A) P( B)
P( B | A) 

 P( B)
P( A)
P( A)
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-25
Statistical Independence (cont.)
• Which of the following players has an advantage:
– Player A: Tries to guess the suit of a card drawn randomly
from a deck of cards.
– Player B: Tries to guess the suit of a card drawn randomly
from a deck of cards after being told its face value.
• Which of the following players has an advantage:
– Player A: Tries to guess the suit of a card drawn randomly
from a deck of cards.
– Player B: Tries to guess the suit of the same card, but only
if player A guesses unsuccessfully.
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-26
In Class Assignment
In the Monte Haul Game show, a particular game was organized
as follows:
•
A prize was hidden behind one of three doors.
•
The contestant picked one of the three doors (trying to find
the prize, of course).
•
One of the two remaining doors (not picked by the
contestant) was opened (always revealing… nothing).
•
The contestant was then asked if he/she wanted to change
his/her mind on what door the prize is behind.
•
Should he/she?
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-27
In Class Assignment (cont.)
Tree
Diagram
Hide Prize
Door 1
Door 2
Door 3
Pick Door
Pick Door
Pick Door
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Y
N
EMIS 7300
Y
N
Y
N
Y
N
Y
N
Y
N
Copyright  2002 - 2006 Dr. John Lipp
Y
N
Y
N
Y
N
S1P2-28
In Class Assignment (cont.)
Hide Prize
Door 1
Door 2
Door 3
Pick Door
Pick Door
Pick Door
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
“Probability of winning if always switch”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-29
In Class Assignment (cont.)
Hide Prize
Door 1
Door 2
Door 3
Pick Door
Pick Door
Pick Door
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Door 1
Door 2
Door 3
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Switch?
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
Y
N
“Probability of winning if never switch”
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-30
Homework
• Mandatory (answers in the back of the book):
2-17
2-35
2-63 (part d correct answer = 24/39)
2-73
2-83
2-91
2-97
• Optional:
2-123 (answer: 0.344%)
2-125 (answer: 200 kits  $5,500 average earnings / week)
EMIS 7300
Copyright  2002 - 2006 Dr. John Lipp
S1P2-31