Transcript File - RtI Innovations Conference

District Evaluation of MTSS
RtI Innovations 2016
Anna Harms & Jose Castillo
Who We Are
Anna Harms
•Evaluation & Research
Coordinator for
Michigan’s Integrated
Behavior & Learning
Support Initiative
Jose Castillo
• Currently - Evaluation
Coordinator for the FL
PS/RtI Project
• Previously - RtI Coach
in a FL school district
Introductions
Share with others at your table:
•Name
•Why you selected this session
•Where you work
•How you’re involved with the evaluation of MTSS
Agenda & Objectives
Agenda
Objectives
Discussions from Establish a foundation for today’s session
the online
modules
Evaluation
planning
Cameo from Florida
District Capacity
Assessment
(DCA)
Explore how the evaluation of MTSS fits into
the work of a district implementation team
-Communication
-Evaluation questions
-Access to data
-Process for data-based decision making
-Reporting
DISCUSSIONS FROM THE
ONLINE MODULES
Establish a foundation for today’s session
District MTSS Evaluation Strand
Format
Scope and sequence
• Pre-recorded online • Module 1: Overview,
modules
Assumptions, and Stages of
• In-person discussion Change/Implementation
and activities
• Module 2:
Exploration/Adoption and
Installation
• Module 3: Initial
Implementation: Illustration
of District MTSS Evaluation,
Elaboration, Continuous
Online Module Check-In
• Did you view Module 1?
• Module 2?
• Module 3?
• Detailed feedback can be provided to us
individually and through the session evaluation
survey (email).
Partner Activity: Assumptions
Consider the assumptions presented regarding
evaluating MTSS implementation at the district level
•Which assumptions do you agree with?
•For which assumptions do you disagree?
• How might differences in people’s assumptions regarding
MTSS and evaluation create variations in the
implementation and evaluation of MTSS?
Stages of Change/Implementation
Should we
do it?
Exploration/Ad
option
Consensus
Work to do Installation
Infrastructure
it right!
Initial
Implementation
Implementation
Work to do Elaboration
it better!
Continuous
Regeneration
“It” is the district evaluation of MTSS.
Discuss as a table:
Exploration/Adoption
• What are some of the successful
strategies you have used to ensure
readiness for evaluating MTSS?
• What could you do differently to support
readiness for evaluating MTSS?
• Select 1 idea to share with the whole
group
EVALUATION PLANNING
Cameo from Florida
DISTRICT CAPACITY
ASSESSMENT
DCA
District Capacity Assessment
The primary purpose of the District Capacity Assessment
(DCA) is to assist school districts to implement effective
innovations that benefit students. The capacity of a district
to facilitate building-level implementation refers to the
systems, activities, and resources that are necessary for
schools to successfully adopt and sustain Effective
Innovations.
Focus in this session:
•Access to data & using data for decision making
•Communication & reporting
DCA Administrator Short Course:
http://implementation.fpg.unc.edu/resources/districtcapacity-assessment-dca
Activity: Preview DCA
• In the DCA document, read the list of 26 items on
page 10
• Highlight the following 4 items:
• Item 10: District Implementation Team uses a
communication plan
• Item 14: DIT has access to data for the effective
innovation
• Item 15: DIT has a process for using data for
decision making
• Item 16: District provides a status report on the
effective innovation to the school board
DCA Item 10
District Implementation Team uses a communication
plan
1. Read the full item scoring guide on page 17
2. Highlight or underline the criteria your district has met
3. What score would you give your district (use
responder)?
Communication Plan
Formal process or system for gathering
and disseminating important
information specific to a common focus
that has been mutually agreed upon by
all groups / teams who will be impacted
by its use
Plan Components
1. Groups / Teams or staff
2. Protocols for what information needs to be
gathered and disseminated to each group / team,
in what format, and in a pre-determined timeframe
1. Designees from each team that are assigned the
task of gathering and disseminating information
1. Survey and dates to assess the effectiveness of
communication
Groups / Teams
• Considerations for including groups / teams or
staff to the communication plan:
• Team’s / group’s overlap in the components of an
integrated behavior and reading MTSS model
• Leadership groups / teams that have the ability to
remove barriers impeding people’s use of the MTSS
components (e.g., central office, administrative team)
• People in critical roles who will be in a supporting role
for MTSS components
• Other groups / teams who can influence PD, curriculum
and assessment decisions since the MTSS work
touches on curriculum, instruction, intervention and
assessment
Plan Components
1. Groups / Teams or staff
2. Protocols for what information needs to be
gathered and disseminated to each group /
team, in what format, and in a pre-determined
timeframe
1. Designees from each team that are assigned the
task of gathering and disseminating information
1. Survey and dates to assess the effectiveness of
communication
Plan Components
1. Groups / Teams or staff
2. Protocols for what information needs to be
gathered and disseminated to each group / team,
in what format, and in a pre-determined timeframe
1. Designees from each team that are assigned
the task of gathering and disseminating
information
1. Survey and dates to assess the effectiveness of
communication
Communication Designees
Consider the following:
•The information that needs to be gathered “from” and
disseminated “to” might suggest one DIT member over
others; especially if there are follow-up questions that
would require deeper knowledge about the information
•Easy access to the team or members of the team
•Ability to follow through with communication shortly
following team meetings (1-2 days)
Timelines and Format Considerations
• Timelines:
• General rule of thumb: 3-5 business days for
responding to challenges; 20 business days for
addressing challenges (e.g., either eliminating the
challenge or starting to address the challenge with other
groups / teams, or entities)
• Format:
• Gathering information: email, face-to-face, phone call
• Responding to challenges: email
Plan Components
1. Groups / Teams or staff
2. Protocols for what information needs to be
gathered and disseminated to each group / team,
in what format, and in a pre-determined timeframe
1. Designees from each team that are assigned the
task of gathering and disseminating information
1. Survey and dates to assess the effectiveness
of communication
Benefits of a Communication Survey
• Regular opportunity to assess the effectiveness
of communication
• Refinements are done based on data
• Conveys the message that communication is
important
• Reduces comments about poor communication
since the survey data and transparency about
refinements based on data are provided on a
regular basis
Example Communication Survey
Activity
Use this sentence frame to describe the benefits
of a Communication Plan:
• One benefit of a Communication Plan is
_____________.
At your table:
• Take turns sharing benefits of a communication
plan, starting with the sentence frame.
Evaluation Questions
1. What is the reach of our MTSS supports across
2.
1.
2.
3.
the district?
To what extent are students in our district
meeting expectations for performance and
growth?
To what extent are district schools
implementing MTSS with fidelity?
What is our capacity to support implementation
of MTSS?
What level of buy-in exists for implementing
MTSS?
Individual Activity: Evaluation Questions
• Review the sample evaluation questions, data sources,
and decision rules
• Start generating questions, data sources, and decision
rules that fit your context in the following areas:
• Student outcomes (academic and behavior)
• Implementation Fidelity
• If relevant: district capacity, reach
• How close are you to being able to answer these questions
and evaluate MTSS implementation and impact?
• What work needs to be done related to:
• Exploration/adoption and consensus?
• Installation of necessary data, systems and practices?
DCA Item 14
DIT has access to data for the effective innovation
Recall critical features of the MIBLSI Database (module 3):
• Data views designed to align with the questions being asked and the
problem solving process
• Single point of data entry to feed up to district, ISD, and state views of
the same data
• Pulls together multiple data sources, not replicate functions of other
systems
• Based on multiple rounds of usability testing
1. Read the full item scoring guide on page 19
2. Highlight or underline the criteria your district has met
3. What score would you give your district (use responder)?
DCA Item 15
DIT has a process for using data for decision making
• What specific model is being used for data-based
decision making in your district?
• How often are data analyzed and when?
• What decisions are made?
1. Read the full item scoring guide on page 20
2. Highlight or underline the criteria your district has met
3. What score would you give your district (use
responder)?
Gather – Study – Plan – Do
Timeline of Data Reviews
Fall ---------------Winter-------------Spring-------
Coaching
Support
Session
District
Data
Review
School
Data
Review
Coaching
Support
Session
School
Data
Review
Coaching
Support
Session
District
Data
Review
School
Data
Review
DCA Item 16
District provides a status report on the effective
innovation to the school board
• Consider alignment with the communication plan and the
evaluation questions
• Consider format and audience
1. Read the full item scoring guide on page 21
2. Highlight or underline the criteria your district has met
3. What score would you give your district (use
responder)?
Tying the Pieces Together
• Each table will be assigned 1 of the 4 DCA items
• Independently complete these 2 sentences: One criteria
our district has met for this DCA item is ______. One thing
our district did to meet this criteria was ______.
• When done with the independent task, move to the area
of the room for your assigned DCA item. Share and
record your ideas on the chart paper.
• Rotate as a group and read what others wrote on their
poster paper for the remaining 3 items.
Takeaways
Share one takeaway with the group.
Sentence starters:
• One thing I would like to learn more about next is _____
• One tip I will try out in my district is _____
• One connection I made is _____
Contact Us
Anna Harms
Jose Castillo
• Michigan’s Integrated
• University of South
Behavior and Learning
Support Initiative
• [email protected]
Florida, Florida Problem
Solving/Response to
Intervention Project
• [email protected]