Transcript Slide 1

2012 OSEP Leadership Conference
Leading Together to Achieve
Success from Cradle to Career
Child Outcomes for Early Intervention and
Preschool Special Education: The National
Picture
Kathy Hebbeler
ECO at SRI
Presentation # EB7
Lynne Kahn
ECO at FPG
Overview
• National numbers and the
quality of the state data
• The findings for 2010-11
• Findings for the last 3 years
• State variation and data quality
2
Early Childhood Outcomes Center
OSEP Reporting Requirements:
Child Outcomes
3
• Positive social emotional skills
(including positive social
relationships)
• Acquisition and use of
knowledge and skills (including
early language/
communication [and early
literacy])
• Use of appropriate behaviors
to meet their needs
Early Childhood Outcomes Center
If you have not seen it….
4
Early Childhood Outcomes Center
State Approaches to Outcomes Data, 2010-11
Approach
Part C
Preschool
(N=56)
(N=59)
COS* 7 pt. scale
43/56 (77%)
36/59 (61%)
One tool
statewide
7/56 (13%)
9/59 (15%)
Publishers’ online
analysis
3/56 (5%)
6/59 (10%)
Other
5/56 (9%)
8/59 (14%)
*Child Outcomes Summary Rating
Child Outcomes: What we know
• States are in various stages with regard
to building their child outcomes
measurement systems
• Some phrased in
• Some changed approaches
6
Early Childhood Outcomes Center
3 Methods Methods for
Calculating National Estimates
1. States with the highest quality data
weighted by child count to represent all
states*
2. All states weighted by child count
3. All states averaged (each state weighted
as 1)
*The data we will be presenting for the national picture
7
Early Childhood Outcomes Center
Identifying the States with Highly
Questionable Data
Criteria for eliminating states:
• Not reporting data on enough children
• Odd patterns in the data
• Review of method reported suggested the
data were of questionable quality
8
Early Childhood Outcomes Center
Continued increase in the
number of states included
Number of States that Met Criteria
for Inclusion in the National Analysis
08-09 09-10 10-11
9
Pt. C
19
29
39
Part B
Preschool
15
33
36
Early Childhood Outcomes Center
OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer to
functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged peers
but did not reach it
d. Improved functioning to reach a level comparable to sameaged peers
e. Maintained functioning at a level comparable to same-aged
peers
3 outcomes x 5 “measures” = 15 numbers
10
Early Childhood Outcomes Center
Illustration of 5 Possible Paths
70
60
Score
50
40
30
20
10
0
1
6
11
16
21
26
31
36
41
46
51
56
Age in Months
Maintained functioning comparable to age peers
Achieved functioning comparable to age peers
Moved nearer functioning comparable to age peers
Made progress; no change in trajectory
Did not make progress
• And now the data….
And now the
data….
12
Early Childhood Outcomes Center
*Based on 39 states with highest quality data
13
*Based on 36 States with the highest quality data.
The Summary Statements
1. Of those children who entered the program
below age expectations in each outcome, the
percent who substantially increased their rate of
growth by the time they turned 3 [6] years of
age or exited the program.
2. The percent of children who were functioning
within age expectations in each outcome by the
time they turned 3 [6] years of age or exited the
program.
15
Early Childhood Outcomes Center
*Based on 39 states with highest quality data
*Based on 36 States with the highest quality data.
Part C: Trends over Time
18
Early Childhood Outcomes Center
Part C: Trends over Time
19
Early Childhood Outcomes Center
Part B Preschool: Trends over Time
20
Early Childhood Outcomes Center
Part B Preschool: Trends over Time
21
Early Childhood Outcomes Center
Looking for Patterns
22
Early Childhood Outcomes Center
%
24
Early Childhood Outcomes Center
Early Childhood Outcomes Center
Part C: States continue to report
on more children
Total Number of Children States Included in the Child
Outcomes Data
26
Number of States and Jurisdictions
Number of
children
reported
2007-08
(N=56)
2008-09
(N=56)
2009-10
(N=54)
2010-11
(N=54)
99 or less
13
6
5
4
100-499
25
16
9
10
500-999
6
13
10
7
1,000-1,999
9
11
10
13
2,000+
3
10
20
22
Early Childhood Outcomes Center
27
Early Childhood Outcomes Center
Part B Preschool: States continue to report on more children
Total Number of Children Included in Progress Data
Number of States and Jurisdictions
Number of
children reported
FFY 2007
(N=58)
FFY 2008
(N=58)
FFY 2009
(N=58)
FFY 2010 (N=59)
10 or fewer
1
1
1
1
10-99
11
7
6
7
100–499
14
6
4
2
500-999
10
12
10
8
1000–1999
8
7
9
5
2000–2999
5
10
11
12
3000–4999
5
11
9
13
5000-8999
3
3
7
9
9000+
1
1
1
2
%
30
Early Childhood Outcomes Center
Maximum
75th Percentile
50th Percentile
25th Percentile
Minimum
Outlier
Variation across states percent “a” :
Part B 619 2010-11
Variation across states percent “b” :
Part B 619 2010-11
Variation across states percent “c” :
Part B 619 2010-11
Variation across states percent “d” :
Part B 619 2010-11
Variation across states percent “e” :
Part B 619 2010-11
Variation across states in greater than
expected growth (Summary Statement
1) : Part B 619 2010-11
Variation across states in exits at age
expectations (Summary Statement
2) : Part B 619 2010-11
Variation across states percent “a” :
Part C 2010-11
Variation across states percent “b” :
Part C 2010-11
Variation across states percent “c” :
Part C 2010-11
Variation across states percent “d” :
Part C 2010-11
Variation across states percent “e” :
Part C 2010-11
Variation across states in greater
than expected growth (Summary
Statement 1) : Part C 2010-11
Variation across states in greater than
expected growth (Summary Statement
2) : Part C 2010-11
Part C Improvement Activities
# states
11
5
3
3
2
4
2
2
2
7
46
Topic/theme
Training/TA on evidence based practices, effective
practices, family centered services, DAP
TACSEI/CEFEL
Training/TA on social emotional development (not
TACSEI/CEFEL specific)
Training/TA on more functional IFSP outcomes
Revised IFSP format to enhance practices
Training/TA on Literacy
Training/TA on RBI
Training/TA on ASD
Training/TA on PSP
Other topics: coaching, instruction, self-help, transition
practices, Special Quest, assistive technology,
relationship
based services in NE
Early Childhood Outcomes Center
Conclusions
• The data show good outcomes for many
children who attend Part C and Part B,
Preschool.
• The data quality continues to get better.
• The data will be a powerful tool for
program improvement when
– The data are consistently good at the local level
– States and local programs consistently use it.
47
Early Childhood Outcomes Center
You might be interested in…..
Sponsored jointly by ECO and NECTAC
48
Early Childhood Outcomes Center
For more information….
www.the-eco-center.org
49
Early Childhood Outcomes Center
50
Early Childhood Outcomes Center
Identifying States with Quality Data
51
Early Childhood Outcomes Center
Calculating Missing Data for Part
C
Proxy for missing data =
Number with data for C3/
Exiting Data (618)
•
•
52
Do not expect this number to be 100%
..but we don’t expect it to be 10% either
Early Childhood Outcomes Center
Percentage of Exiters Reported by
Part C Continues to Increase
53
Early Childhood Outcomes Center
Calculating Missing Data for 619
Proxy for missing data =
Number with data for B7/
Child count
•
•
54
Do not expect this number to be 100%
..but we don’t expect it to be 10% either
Early Childhood Outcomes Center
Percentage of Child Count Reported by Part B
Preschool Continues to Increase
Pt. B Preschool: % of Child Count Reported in Child Outcomes Data
18
Number of States
16
<10%
14
10-20%
12
10
21-30%
8
31-40%
6
41-50%
4
>50%
2
0
08-09
55
Early Childhood Outcomes Center
09-10
10-11
Many states reporting a low percentage for “a”
but still have a few outliers.
Number of States by % Reported in “a,” 2010-11
7 to <13%
Part C
32
11
5
0
28 to 49%
3
<2%
2 to <5%
5 to <7%
56
Early Childhood Outcomes Center
Part B
Preschool
30
11
3
5
0
Number
excluded
Percent
(n=51)
State is sampling
No outcomes data reported
Reported outcomes data on less than 28% of
reported exiters
Had at least one outcome with category a
greater than 10% or category e greater than
65%
3
0
3
6
0
6
4
8
Reported outcomes data on less than 28% of
reported exiters
1
2
1
2
39
76
Reason Part C state was excluded
AND
Had at least one outcome with category a
greater than 10% or category e greater than
65%
Questionable data quality based on review of
SPP/APR and knowledge gained through TA
States included in the analysis
Number
excluded
Percent
(n=51)
4
1
8
2
2
4
4
8
Reported outcomes data on less than 12% of
child count
AND
Had at least one outcome with category a
greater than 10% or category e greater than 65%
0
0
Questionable data quality based on review of
SPP/APR and knowledge gained through TA
2
4
1
2
1
2
36
71
Reason Part B state was excluded
State is sampling
No outcomes data reported
Reported outcomes data on less than 12% of
child count
Had at least one outcome with category a
greater than 10% or category e greater than 65%
No child count data available
Reported Summary statements but no progress
categories
States included in the analysis
Producing National Estimates Using the States
with the Highest Quality Data
1. Divide the states into those with large,
medium, and small child counts.
2. Identify the states that appear to have the
best data.
3. Weight by child count and calculate the
overall OSEP percentages using the “best
data” states in each category.
4. Use the OSEP percents and total child count
for each category to get the national
estimates.
59
Early Childhood Outcomes Center
Variation across states in Outcome 1: Part B 619 2010-11
Variation across states in Outcome 2: Part B 619 2010-11
Variation across states in Outcome 3: Part B 619 2010-11
Variation across states in Outcome
1: Part C 2010-11
Variation across states in Outcome
2: Part C 2010-11
Variation across states in Outcome
3: Part C 2010-11