Transcript slides

Introduction to Policy
Processes
Dan Laitsch
1
Overview (Class meeting 4)


Sign in
Agenda
–
–
–
–
–
–
–
–
–
–
Cohort break outs
Review last class
Mid term assessment
PBL Groups
Significance [dismiss]
Policy and unifying content
T-tests
PBL groups
Action research
PBL and dismiss
2
Class : Review

Stats
– Hypothesis testing
– Z scores

PBL
– Topic determined

Policy
– Role Play
3
Cohort Break Out

Courses and dates (summer session)
– EDUC 813: organizational Theory (Drescher)


April24/25, May 8/9, May 22/23, June 6/7, June 19/20, and
June 26/27
Summer Institute
– EDUC 822: Evaluation of Educational Programs



July 2, 3, 6, 7, 8, 9, 10, 13, 14, 15, 16. (Mornings 8:30 to
1:30 or Evenings 4:30 to 9:30). SI public lecture times
included as part of class hours (July 6, Evening; 7,9,14 and
16, 1:00 pm to 3:00 pm).
Action Research Time Frame
Comprehensive exams
4
Midterm Assessment

Data drive decision making
– What do the following data “tell” you?
– What questions do they leave unanswered?

Analysis and response
5
6
7
Response

Heavy workload
– Addressing past student concerns
– Creating balance
– Unifying vision

Possible solutions
– Goals: meet course description (policy processes)
– Prepare students for Action Research
– Continued tomorrow
8
PBL groups


Touch base
Status check
– Group functioning?

Forming, storming, norming, performing?
– Topic identified?
– Action plan?
– Turn in report (handout)

Plan for tomorrow
– 2-3 hours of group time (2 break out 1 hour to 1.5
hours each)
9
Part IV: Significantly Different
Using Inferential Statistics
Chapter 9  
Significantly Significant
What it Means for You and Me
10
What you learned in Chapter 9

What significance is and why it is
important
– Significance vs. Meaningfulness
Type I Error
 Type II Error
 How inferential statistics works
 How to determine the right statistical test
for your purposes

11
The Concept of Significance

Any difference between groups that is due
to a systematic influence rather than
chance
– Must assume that all other factors that might
contribute to differences are controlled
12
If Only We Were Perfect…

Significance level
– The risk associated with not being 100% positive that what
occurred in the experiment is a result of what you did or
what is being tested

The goal is to eliminate competing reasons for
differences as much as possible.

Statistical Significance
– The degree of risk you are willing to take that you will
reject a null hypothesis when it is actually true.
13
The World’s Most Important
Table
14
Type I Errors (Level of Significance)
The probability of rejecting a null
hypothesis when it is true
 Conventional levels are set between .01
and .05
 Usually represented in a report as
p < .05

15
Type II Errors
The probability of rejecting a null hypothesis
when it is false
 As your sample characteristics become closer
to the population, the probability that you
will accept a false null hypothesis decreases

16
Significance Versus
Meaningfulness



A study can be statistically significant but not
very meaningful
Statistical significance can only be interpreted for
the context in which it occurred
Statistical significance should not be the only
goal of scientific research
– Significance is influenced by sample size…we’ll talk
more about this later.
17
How Inference Works
A representative sample of the population
is chosen.
 A test is given, means are computed and
compared
 A conclusion is reached as to whether the
scores are statistically significant
 Based on the results of the sample, an
inference is made about the population.

18
Deciding What Test to Use
19
Test of Significance
1. A statement of null hypothesis.
2. Set the level of risk associated with the null hypothesis.
3. Select the appropriate test statistic.
4. Compute the test statistic (obtained) value
5. Determine the value needed to reject the null hypothesis
using appropriate table of critical values
6. Compare the obtained value to the critical value
7. If obtained value is more extreme, reject null hypothesis
8. If obtained value is not more extreme, accept null
hypothesis
20
The Picture Worth a Thousand
Words
21
Glossary Terms to Know
Significance level
 Statistical significance
 Type I error
 Type II error
 Obtained value

– Test statistic value

Critical value
22
End of Class
PBL Work if time allows
 Clarifying grades

Journal, portfolio, stats notebook

Homework:
– Thinking about research
What areas are you thinking about?
 What questions do you have?
 Prepare to chat with colleagues tomorrow

23
Agenda
–
–
–
–
–
Policy and unifying content
T-tests
PBL groups
Action research
PBL and dismiss
24
Unifying themes

Diffusion models
– Communication networks


Diffusion of innovation
Adoption
– Internal (policy window)
• Severity (crisis)
• Opportunity
– External (policy borrowing)
• National
• Regional
• Leader-Laggard
• Isomorphism (similar states)
• Vertical
25
Unifying themes

Internal (policy window)
– Severity (crisis)
– Opportunity

Evidence/Data (my insert)
– Research
– Statistics

Problem
Solution
External (policy borrowing)
– Governments (CMEC)
– Organizations (CTF, JCSH, CERC-CA)
Policy
Study
Unifying themes

Problems
PBL
– Identification (what is the problem)
– Analysis (what is the cause)

Solutions
– Research (what has been done)
– Context (how does it fit here)

Research Reviews
Action Research
Policies
– Action (what are the rules and procedures)

Evaluation
– Analysis (what happened)
– Refinement (what might we change)
Policy Analysis
Unifying themes

Leadership
– Identifying context (observation and data gathering)


Data gathering and synthesis (problem identification)
Identifying parameters (policy analysis)
– Setting direction (goals and outcomes)



Research (identify interventions)
Policy (identify rules and procedures for action)
Analysis (identify consequences)
– Achieving Goals (problem solving)


Implementation of actions and activities
Application of rules and procedures (policy)
– Evaluation (refining context)
Part IV: Significantly Different
Using Inferential Statistics
Chapter 10   
t(ea) for Two
Tests Between the Means of Different Groups
What you learned in Chapter 10
When to use a t test
 How to compute the observed t value
 Interpreting the t value and what it means

t Tests for Independent
Samples

Determining the correct statistic
Computing the Test Statistic

Numerator is the difference between the
means

Denominator is the amount of variation
within and between each of the two groups
Degrees of Freedom
Degrees of freedom approximate the
sample size
 Degrees of freedom can vary based on the
test statistic selected
 For this procedure…


n1 – 1 + n2 – 1
So How Do I Interpret…
 t (58) = -.14, p > .05
– t represents the test statistic used
– 58 is the number of degrees of freedom
– -.14 is the obtained value (from the formula)
– p > .05 indicates the probability (n.s.)
 p = n.s.
– p < .05 indicates the probability (sig.)
Special Effects…
Effect size is a measure of how different
two groups are from one another
 Standardized difference between to group
means
 Jacob Cohen

Computing Effect Size
X1  X 2
ES 
,
SD



Small = 0.0 - .20
Medium = .20 - .50
Large = .50 and above
Effect Size Calculator

http://web.uccs.edu/lbecker/Psy590/escalc3.htm
Glossary Terms to Know
Degrees of freedom
 t Test

– Independent t Test
– Obtained value
– Critical value

Effect size
Part IV: Significantly Different
Using Inferential Statistics
Chapter 11   
t(ea) for Two (Again)
Tests Between the Means of Related Groups
What you learned in Chapter 11
When to use a t test for dependent means
 How to compute the observed t value
 Interpreting the t value and what it means

t Tests for Dependent Samples

Determining the correct statistic
Computing the Test Statistic

Numerator reflects the sum of the
differences between two groups
Degrees of Freedom
Degrees of freedom approximate the
sample size
 Degrees of freedom can vary based on the
test statistic selected
 For this procedure…

– n – 1 (where n is the number of observations)
So How Do I Interpret…
 t (24) = 2.45, p > .05
– t represents the test statistic used
– 24 is the number of degrees of freedom
– 2.45 is the obtained value (from the formula)
– p > .05 indicates the probability (n.s.)
p
= n.s.
– p < .05 indicates the probability (sig.)
PBL Groups
Break into groups
 Lunch

45
Action Research

Pair share

Model and paper process
–
–
–
–
–

Observations
Questions
Data
Methods
Analysis
Discuss
46
PBL Groups

PBL Work if time allows
47