Transcript Evaluation of training courses

Methods of evaluation of
the effects of care
Course for Young Psychiatrists
Addis Ababa, 28th April 2006
David Goldberg
What are the real aims of
your care?
• Acute illnesses – include measures of
symptom severity
• Chronic illnesses – spend less time on
symptom severity, use a brief
measure, remember Quality of Life
• ALWAYS include disability measures
• Usually include satisfaction measures
What are the real aims of
your care?
• Acute illnesses – include measures of
symptom severity
• Chronic illnesses – spend less time on
symptom severity, use a brief
measure, remember Quality of Life
• ALWAYS include disability measures
• Usually include satisfaction measures
Other indicators of care
• Unmet Need – measures include the
“CAN”, Camberwell Assessment of
Need
• Economic measures – if you can
reduce the cost of care, politicians in
your country will be greatly
interested!
Evaluation design
• Before/after evaluations
• Controlled studies
• Short/long term outcomes
Design of the Evaluation
There are two sorts of control groups:
1) Random assignment of individuals to the
new treatment or the “usual treatment”
(called “TAU”)
2) Choosing two different places with
comparable patients, and introducing the
new treatment in only one of them
Evaluations without a control group are a
waste of everyone’s time!
Methods of evaluation of
success of training courses
Course for Young Psychiatrists
Addis Ababa, 28th April 2006
David Goldberg
Evaluation: impact on
clinicians
• Is it possible to bring about a change
in clinician knowledge, attitudes or
skills?
• Does the intervention improve morale
or confidence?
• How satisfied are the clinicians with
the intervention?
Assessing knowledge
• Self-evaluation
• Instructor’s opinions
• objective measures
Assessing Attitudes
• Questionnaires (e.g. Depression
Attitude Questionnaire; Attitudes to
Psychiatry)
• Interviews
– structured
– semistructured
Skill acquisition
• Observed Structured Clinical Examinations
– rate against predefined scorecard.
Both of these are “before and after”:
• Videotaped interviews with real or roleplayed patients
– rated ‘blind’ using structured rating scales
• Changes in ability to identify or assess
accurately emotional disorder using
comparison of patient GHQ rating and PCP
rating; numbers of patients identified;
prescriptions issued
Improvement in
morale/confidence
• Self-rated linear analogue scales
• Interviews
• Post-training assessment rated
against pre-training self-assessment
of needs/objectives
Evaluation: Impact on
process and outcome of care
Process of care:
–
–
–
–
frequency and length of visits
prescription of medication
referrals
use of investigations
Outcome of care:
– hospitalisations
– numbers improved, symptom free
Evaluation: Impact on
process and outcome of care
• Patient satisfaction
• Compliance
• Clinical outcome
– symptoms
– disability
• Economic outcome
- how many return to work?