Transcript SPRAT Feedback May 06 – August 06

RCPCH SPRAT Feedback
May 06 – August 06
A Guide to Interpretation
Interpreting the Feedback
• As you click through the slides you will be
guided through the feedback
• This is designed to help both trainees and
trainers in interpreting feedback
• You may want to go through this first and
then look at your own/your trainee’s
feedback matching it to the sample slides
in this presentation.
Copies of the feedback
• Trainees receive a copy of the feedback
excluding the group mean scores for their peer
group (i.e. core [year 2] or post-core [year 4]).
• Educational supervisors receive a copy of the
feedback including the group mean scores for
their trainee’s peer group (i.e. core [year 2] or
post-core [year 4]).
• Trainees need to retain a copy including the
comparator data for their portfolio after
discussion with their educational supervisor.
Presentation of graph
In response to feedback on the graph format
we have produced a black and white version
with different symbols for the 3 data points.
This is an interim measure until we make
further improvements where we hope to
have colour and black and white versions
available.
The Feedback Graph
These numbers refer to the scale of the
SPRAT assessment tool where 1 = below
expectations for a trainee at the same
stage of training and 6 = above
expectations for a trainee at the same
stage of training.
These data points () are your/your trainee’s
self rating score for each question.
These data points (×) are the mean score
per question from assessors who
responded to that question.
These data points (Φ) are the mean score
+/- 2 SD for the relevant peer group (i.e.
core [year 2] or post-core [year 4]) for each
question.
Number of assessors who completed and
submitted a SPRAT form.
The questions of SPRAT, shown here
divided into the domains of GMP.
Good clinical care
The questions of SPRAT, shown here
divided into the domains of GMP.
Maintaining good medical practice
The questions of SPRAT, shown here
divided into the domains of GMP.
Teaching and training, appraising and assessing
The questions of SPRAT, shown here
divided into the domains of GMP.
Relationship with patients
The questions of SPRAT, shown here
divided into the domains of GMP.
Working with colleagues
Statistical Interpretation
Standard Deviations (SDs) are a measure of spread of data from
the mean and not a range (range meaning highest and lowest
scores)
• Use of SDs helps provide a feel for where a score is in relation to the
whole comparative group by giving some idea of the distribution of
ratings around the cohort mean.
• By focussing less on the actual mean score (remembering that 50%
of people will of course be below the mean) you/your trainee should be
able to place performance better in context.
• For individual questions where the score falls close to or outside
+2SDs, this may be an area for discussion of strengths.
• For individual questions where the score falls close to or outside
-2SDs, this may be an area of weakness which requires developmental
planning.
Statistical Interpretation Contd.
• If you/your trainee scores less than 4.0 overall then further evaluation
is advisable.
Why does the standard deviation plot appear to show a score
greater than 6 when 6 (above expectations) was the highest score
obtainable?
As standard deviation is calculated using a mathematical formula, this
formula does not know that the maximum score a doctor can achieve is
6 and may provide values in excess.
The Comments Sheet
This is the overall self rating mean score for
the 25 questions of SPRAT.
This is the overall assessor mean score for the
25 questions of SPRAT.
This is the overall mean score for all
trainees in the relevant peer group
(i.e. core [year 2] or post-core [year
4]) for the 25 questions of SPRAT.
These are strengths identified by the
assessors who completed the form. They are
typed verbatim where each paragraph
represents a comment from a different
assessor.
These are suggestions for development
identified by assessors completing the
form. They are typed verbatim where
each paragraph represents a comment
from a different assessor.
These are probity or health concerns raised
by assessors. They are typed verbatim where
each paragraph represents a comment from
a different assessor.
Thank you for your time - we hope
this guidance has been helpful