Transcript Session 4 - Teaching Slides

Measuring Quality
Understanding methods of
performance measurement
1
Objectives
After this session, participants will understand:
• Where we can measure quality
• How to choose and calculate an indicator
• How to collect the data
2
Measuring Quality
• Once we know how care should be delivered, how
do we know whether we are doing it?
 We measure it
3
Importance of measurement
• Measuring is important to:
– Identify where a problem areas/gaps
– prioritize where QI is needed
– Guide discussion on possible causes of gaps and
their solutions
– Measure change over time
4
How can we measure quality?
In other words, how do we know
there is a problem?
5
How do we know there is a
problem?
• You see it or hear about it
• You use routine data reporting or
measurement
• You do targeted measurements
6
6
How do we know there is a
problem? (1)
• You see it or hear about it
– You hear of a “never” event – something you
NEVER want to happen.
• Can someone think of any example?
– It is “obvious”
– A patient makes a complaint
– Morbidity and mortality reports
7
7
How do we know there is a
problem? (2)
• Routine measurement
– MOH reports (monthly/quarterly reports)
– Formal chart reviews
• Targeted measurements
– How well is TB referral going
8
8
Important steps in measuring
•
•
•
•
•
Identify the area you are interested in
Define what you will measureindicators
Collect data
Analyze data*
Discuss and use the results*
*discussed in later talks
9
Important steps in measuring
• Identify the area you are interested in
• Define what you will measure:
indicators
• Collect data
• Analyze data
• Discuss and use the results
10
Three areas to measure the quality
of care
Structure
Process
Outcome
• Material
resources
• Human
resources
• Organizational
characteristics
• Systems
•Health care
services provided
•Health status
•Knowledge acquired
by patients and family
members
•Behavior of patients or
family members
•Satisfaction of patients
and their family
members
11
Structure
Structure
• Material
resources
• Human
resources
• Organizational
characteristics
• Systems
• Clinic infrastructure and
inventories
• Laboratory infrastructure and
lab test availability
• Pharmacy – availability or stock
outs of key medications
• Success of referrals
• Staff training needs
• Systems: How a process is
completed in the clinic
12
Process – Were the recommended
services provided?
Process
•Health care
services provided
• TB screening (active case finding)
• CTX prophylaxis
• Monitoring: CD4 or Viral load
testing
• Antiretroviral treatment
(appropriate, timing)
• Adherence assessment
• HBV/HCV testing
13
What were the outcomes?
• Retention in care &
treatment services
• Adherence
• Undetectable viral load
• Fewer missed visits
• Lab and pharmacy
availability
• Patient satisfaction
Outcome
•Health status
•Knowledge acquired
by patients and family
members
•Behavior of patients or
family members
•Satisfaction of patients
and their family
members
14
Identify what we are interested in
• Example: Let’s say we are interested in
knowing whether patients are getting PCP
prophylaxis as they should be.
15
Important steps in measuring
•
•
•
•
•
Identify the area you are interested in
Define what you will measureindicators
Collect data
Analyze data
Discuss and use the results
16
What is an indicator?
• Definition: A variable that measures one aspect of a
program or project.
• In health care a quality indicator is used to measure
how well we are delivering care
• Usually based on standards (MOH or WHO
guidelines) which define how a service should be
delivered.
17
Example: Standard vs. Indicator
• Standard (MOH guideline): Cotrimoxazole
should be prescribed to all patients with CD4 ≤
350 or WHO stage III or IV disease.
• Indicator: Cotrimoxazole prescription for
eligible patients at the most recent visit.
18
What is a “good” indicator
• Relevant (you care about the area)
– Linked to a condition or service that occurs frequently or
has large impact.
• Measurable
– Can be realistically measured given resources
– Measurement can show change
• Accurate
– Based on accepted guidelines or developed through groupdecision making methods/consensus
• Improvable
– Can be improved (by the site or program)
19
Adapted from HIVQUAL guide for QI in HIV care, 2006
Raise your hand if the indicator is
good
• The percent of patients with CD4 < 350 who
received CTX at their most recent visit
• The percent of rooms with pink walls
• The percent of patients receiving ART within
one month of eligibility
• The number of patients on ART who were
born in June
20
A few examples of good indicators in
HIV
• % of patients screened for active TB at
most recent visit
• % of eligible patients receiving CTX
• % of eligible patients on ART
• % of patients with CD4 testing in last 6
months.
• % of children up to date with vaccinations
21
Important steps in measuring
• Identify the area you are interested in
• Define what you will measure:
indicators
• Collect data
• Analyze data
• Discuss and use the results
22
Measuring performance
• Measure whether there is a gap in an area by
measuring the performance.
• Example: You’ve decided you want to know
whether your clinic is providing good quality
of care in the area of PCP prophylaxis and you
will turn the indicator into a measure:
– “What percentage of patients with CD4 < 350
received CTX or Dapsone at the most recent visit.”
23
Steps to measure performance in a
selected indicator
1. Who is your starting population?
– All active patients at OPC on the date of review
2. Who is in the denominator?
– Number of patients who should have received the service
3. Who is in the numerator?
– Number of patients who received services
⅞
24
Steps to measure indicator
Population targeted for
performance measurement
All active patients at date of review
Denominator
Number of patients who should
have received the service
Numerator
Number of patients
who received the
service
25
How would you measure the
following indicator?
Percent of patients eligible for
cotrimoxazole who received it at the
most recent visit at one OPC with 300
active patients
26
Who is eligible to have their quality
measured?
• All active patients on the date of review
• At least one visit in the previous year
• 300 patients are targeted for
performance measurement
• But…who is eligible for cotrimoxazole
measurement?
27
Denominator
• What part of this population should have
received the component of care being
measured?
• All patients with CD4 <=350 or WHO stage
III/IV as of the last visit
• After reviewing the 300 charts you find that
100 patients are eligible for CTX prophylaxis at
most recent visit
28
Numerator
• What number of those who should have
received the service (CTX) did receive the
service?
• All patients with CD4 <=350 and WHO stage
III/IV who were prescribed CTX at most recent
visit
• Out of 100 eligible patients, 80 patients were
prescribed CTX
29
Steps to measure indicator
Population targeted for
performance measurement
All active patients at date of review
N=300
Denominator
Number of patients who should
have received CTX
N=100
Numerator
Number of patients
who received CTX
N=80
80/100 = 80% of patients who should have been receiving
prophylaxis were actually receiving it
30
Important steps in measuring
• Identify the area you are interested in
• Define what you will measure:
indicators
• Collect data
• Analyze data
• Discuss and use the results
31
How much data should you
collect?
• Do you have to review every chart every
6 months? No.
• It is not a good use of resources to collect
data from every active patient in your
clinic
• The goal of measurement in QI is to
routinely collect “just enough” data
32
“Just enough”
• You sample enough so you are confident that
your data gives you a good estimate of the
clinic’s performance in that area
• Keep it simple enough to be able to do
routinely
• Sampling: Collecting data from a smaller
number of charts that are representative of
your population (active patients)
33
So how many charts do we need to
review?
• Choose the minimum number of charts to be
sure enough of the true performance.
• Allow a margin of error (we use +/- 8%)
• The size of your sample is determined how big
is your population.
34
Sample size when
measuring indicators
• This table was constructed to give
us 95% confidence that the
performance rate we get is
correct +/- 8%.
•
Eg. If we find the rate is 80%
then we are very confident that
the real rate lies between 72 and
88%. Good enough for QI
Number
1-20
21-30
31-40
41-50
51-60
61-70
71-80
81-90
91-100
101-119
120-139
140-159
160-179
180-199
200-249
250-299
300-349
350-399
400-449
450-499
500-749
750-999
1000-4999
Sample size
All
26
32
38
43
48
53
57
61
67
73
78
82
86
94
101
106
110
113
116
127
131
35
146
Choosing the charts – registry
available
• If clinic registry is available. Choose from clinic
registry
• Random sample – random number generator
• Or systematic sample
– Example: choose 94 (based on table) in 200 ART:
choose every 2nd patient (2=200/94).
– Pull patient chart out corresponding with the
selected ID patient above.
36
Choosing the charts – no registry
available
• Choose patient charts from the shelves.
• Example: OPC with 200 patients.  Sample
size: 94 charts.
• Select every 2nd chart (2= 200/94) and check
to see if patient is active
– Yes – Abstract data
– No – choose the next chart on the shelf.
Note: who is on the shelf will affect your sample. Be
careful if charts are grouped (ART/Care/death – ask
TA provider for for assistance)
37
How do you ensure data quality?
• Standardized tool or method of collection
• Staff trained in how to collect the data
• Review during data collection to ensure being
done right.
• More to come in data collection and software
training.
38
Summary (1)
• Performance measurement is built from
indicators based on approved standards
• Indicators may be qualitative or quantitative
and are generally a mix of process, structural
and outcomes
39
Summary (2)
• “Just enough”: You don’t need to review all
charts for QI. Amount of data should be
feasible for routine collection. If you are not
going to use it, don’t collect it.
• Indicators should be chosen based on
relevance; ability to measure, intervene and
monitor for improvement over time.
40