Transcript Stats Exam Prep. - New Jersey Institute of Technology

Stats Exam Prep.
Dr. Lin Lin
WARNING
• The goal of this workshop is to go over some
basic concepts in probability and statistic
theories required for IS 665
• It is NOT to help you pass the exam
• NO EXAM QUESTIONS will be covered here
Probability
• Probability is the measure of how likely
something will occur.
• It is the ratio of desired outcomes to total
outcomes.
– P(event) = (# desired events) / (# total events)
• Probabilities of all outcomes sums to 1.
BEFORE Probability
• We need to learn to count the number of possible
events
• Exercise I: How many different five-digit numbers exist?
• How did you get the answer?
BEFORE Probability
• Exercise II: How many different five-digit numbers
WITHOUT 0 exist?
• How did you get the answer?
BEFORE Probability
• Exercise III: US phone number is in the format of
(###) – ### - ####
– The first digit cannot be zero
– There cannot be a “000-0000” number
– How many numbers are possible?
• How did you get the answer?
BEFORE Probability
• Exercise IV: let’s make a three-digit number. There is only
one rule: no two digits could be identical. How many
numbers could we make?
• How did you get the answer?
BEFORE Probability
• Exercise V:
• You could rearrange these shapes anyway you want. However,
cannot be on either side. How many different ways could
we have?
• How did you get the answer?
Probability Example
• If I roll a number cube, there are six total
possibilities. (1,2,3,4,5,6)
• Each possibility only has one outcome, so each has a
PROBABILITY of 1/6.
• For instance, the probability I roll a 2 is 1/6, since
there is only a single 2 on the number cube.
Practice
• If I flip a coin, what is the probability I get
heads?
• What is the probability I get tails?
• Remember the equation? Number of desired
outcomes divided by number of possible outcomes
Answer
• P(heads) = 1/2
• P(tails) = 1/2
• If you add these two up, you will get 1, which
means the answers are probably right.
Answer
• Let’s make it harder – assuming that the coin is not
fair, and P(H) = 0.6
• What is the chance of getting a tail in one flip?
Bernoulli Trial
• In the theory of probability and statistics, a Bernoulli trial (or
binomial trial) is a random experiment with exactly two
possible outcomes, "success" and "failure", in which the
probability of success is the same every time the experiment
is conducted.
• So it is basically coin-flipping with a p of not necessarily 0.5
Two or more independent events
• If there are two or more independent events,
you need to consider if it is happening at the
same time (and) or one after the other (or).
And
• If the two events are happening at the same time,
you need to multiply the two probabilities together.
This probability is called joint probability
• Usually, the questions use the word “and” when
describing the outcomes.
• P(A & B) = P(A)*P(B)
Joint Probability

Just a fancy way of saying “AND”
◦ p(I will listen to Backstreet Boys Today) = 0.8
◦ p(I will eat at Subway today) = 0.7

What is the probability that I will listen to Backstreet
Boys AND eat at Subway?
◦ 0.7 * 0.8 = 0.56?

WHEN EVENT A AND B ARE INDEPENDENT:
◦ P(A&B) = P(A) * P(B)
Or
• If the two events are happening one after the other,
you need to add the two probabilities.
• Usually, the questions use the word “or” when
describing the outcomes.
• P(A or B) = P(A) + P(B)
Practice
• If I roll a number cube and flip a coin:
– What is the probability I will get a heads and a 6?
– What is the probability I will get a tails or a 3?
• How did you get them?
Answers
• P(heads and 6) = 1/2 x 1/6 =1/12
• P(tails or a 5) = 1/2 + 1/6 = 8/12 = 2/3
Summary: Independent Events
• One event has no influence on the outcome of
another event
• If events A & B are independent
then P(A&B) = P(A)*P(B)
P(A or B) = P(A) + P(B)
• Coin flipping
if P(H) = P(T) = .5 then
P(HTHTH) = P(HHHHH) = .5*.5*.5*.5*.5 = .55 = .03
Summary: Independent Events
• Coin flipping
if P(H) = P(T) = .5 then
P(HTHTH) = P(HHHHH) = .5*.5*.5*.5*.5 = .55 = .03
• What if P(H) = 0.6 (Bernoulli trial)?
– What is P(HTHH)?
– What is P(at least one head in five trials)?
• if you are flipping a coin and it has already
come up heads 6 times in a row, what are
the odds of an 7th head?
.5
• Is P(10H) = P(4H,6T)?
Next year the economy will experience one of three states: a
downturn, stable state, or growth. The following probability
matrix displays joint probabilities of a bond default and the
economic state:
For example, the joint probability that the economy is stable and
the bond defaults is 1.0%; the unconditional probability that the
economy will be stable is 50.0% = 49.0% + 1.0%.
Next year the economy will experience one of three states: a
downturn, stable state, or growth. The following probability
matrix displays joint probabilities of a bond default and the
economic state:
For example, the joint probability that the economy is stable and
the bond defaults is 1.0%; the unconditional probability that the
economy will be stable is 50.0% = 49.0% + 1.0%.
Conditional Probability
• Concern the odds of one event occurring,
given that another event has occurred
• P(A|B)=Prob. of A, given B
Examples
• P(Professor Lin walks in without a Pepsi in his hand) = 0.1
• HOWEVER…
• P(Professor Lin walks in without a Pepsi in his hand | you
promise to give me $1,000,000 if I do so) = 1 !!
• What changed my behavior?
Conditional Probability (cont.)
• P(B|A) = P(A&B)/P(A)
• if A and B are independent, then
P(B|A) = P(A)*P(B)/P(A) = P(B)
The Chain Rule

What if A and B ARE dependent of each other?
◦ p (I am teaching IS 665 today) = 1/7
◦ p (I am eating at Subway today) = 0.7

What is the chance that I am teaching 665 today and eating at
Subway?
◦ p (I am teaching IS 665 today & I am eating at Subway today) = 0!

WHY?
◦ Because to teach 665, I have NO TIME to eat at Subway!
◦ In other words, these two events are dependent
The Chain Rule

What is the chance that I am teaching 665 today and eating
at Subway?
◦ p (I am teaching IS 665 today & I am eating at Subway today) =
p (I am eating at Subway today | I am teaching 665) *
p (I am teaching 665)
= 0 * 1/7 = 0

To put it (semi) formally:
◦ P(A & B) = P (A | B) * P (B) = P(B | A) * P(A)
The Bayes Rule

The Chain Rule Shows us:
◦ P(A & B) = P (A | B) * P (B) = P(B | A) * P(A)
P (A | B) = P(B | A) * P(A) / P(B) !!!
This is the Bayes Rule
The Bayes Rule
P(B | A) = P(B) * P (A | B) / P (A)
PB P A | B 
P  B | A 
PB P A | B   P~ B P A |~ B 
Exercise
If we observe that the bond has defaulted, what is the (posterior)
probability that the economy experienced a downturn?
a. 0.60%
b. 19.40%
c. 26.33%
d. 31.58%
Exercise
If it is snowing, there is a 80% chance that class will be
canceled. If it is not snowing, there is a 95% chance that
class will go on. Generally, there is a 5% chance that it
snows in NJ in the winter.
If we are having class today, what is the chance that it is
snowing?
PB P A | B 
P  B | A 
PB P A | B   P~ B P A |~ B 
Distribution
• How to Read a Histogram
Normal Distribution
• Watch the demo
Regression
• Predict the target value 𝑦 with the attributes 𝑋 by a function: 𝑦 =
𝛽0 + 𝛽1 ∗ 𝑥1 + 𝛽2 ∗ 𝑥2 + ⋯ + 𝛽𝑚 ∗ 𝑥𝑚 + 𝜀
• Handle numeric attributes and predict numeric value.
Regression
• Goal: minimize the error 𝜀
• An example
– 𝑆𝑎𝑙𝑎𝑟𝑦 = 𝛽0 + 𝛽1 ∗ 𝑒𝑑𝑢𝑙𝑒𝑣𝑒𝑙 + 𝛽2 ∗ 𝐼𝑄 + 𝛽3 ∗ 𝑒𝑥𝑝𝑒𝑟𝑖𝑒𝑛𝑐𝑒 + 𝛽4 ∗ 𝑔𝑒𝑛𝑑𝑒𝑟
Variable
Estimate
Std. Error
t value
Pr(>|t|)
(Intercept)
5000
XXX
XXX
0.020
edu_level
1000
XXX
XXX
0.001
IQ
50
XXX
XXX
0.814
experience
300
XXX
XXX
0.004
gender
-2000
XXX
XXX
0.300
– 𝑆𝑎𝑙𝑎𝑟𝑦 = 5000 + 1000 ∗ 𝑒𝑑𝑢_𝑙𝑒𝑣𝑒𝑙 + 50 ∗ 𝐼𝑄 + 300 ∗ 𝑒𝑥𝑝𝑒𝑟𝑖𝑒𝑛𝑐𝑒 −
2000 ∗ 𝑔𝑒𝑛𝑑𝑒𝑟
What is Regression, anyway?
Number of nights I illegally parked
Chance that I will get a ticket
0
3
1
21
2
36
3
44
4
66
5
81
y = 15.229x + 3.7619
Coefficient Intercept
If I parked illegally 6 nights in a row, how likely am I to get a ticket?
P value
What is Regression, anyway?
• You now know how to interpret a regression
model
• But how do we build one?
– That will be covered in IS 665