Transcript Part - end+disparities Learning Exchange

end+disparities Learning
Exchange
Part IV: Calculator Assumptions
Quick Review
 This presentation is the fourth in a series of presentations
intended to familiarize you with disparities calculation
 Part I: Disparity, a National Priority
 Part II: Subpopulations
 Part III: Calculating Disparity
 Part IV: YOU ARE HERE!
 Part V: Selecting QI Projects
2
"All human beings are born free and equal in
dignity and rights."
- United Nations, Article 1 of the Universal
Declaration of Human Rights
3
Learning Objectives
 Describe how was the calculator methodology
developed
 Understand the assumptions made within the
calculator tool
4
Supreme Court of the United States
 Statisticians and scientists disagree on how to calculate
disparities – there are many methods
 The most uniform method for calculating and interpreting
disparities comes from SCOTUS
 SCOTUS has decades of history
dealing with disparities in case law
called Disparate Impact
 Housing
 Employment
 Jury Selection, etc.
5
NQC Guide:
Qualifying Disparities in HIV Care
 Overviews
 What is a disparity?
 How do we qualify disparities statistically?
 How do we intersect probability and impact for disparity work?
 Mathematical backgrounds and calculation walkthroughs
 Absolute Disparity
 Rate Ratio
 Comparative Disparity
 Odds Ratio
 Absolute Impact
6
NQC Guide: Qualifying Disparities in HIV Care
Instructional Slides
 Co-dissemination of Materials
 Disparities analysis is a challenging task that can cause confusion or
concern among users who are not adequately trained in how to use
the tools for QI
 NQC Guide + Instructional Slides + Calculator Tool = happy users
7
Examples in this Presentation
8
Youth (ages 13-24)
MSM of Color
African-American
& Latina Women
Transgender People
 Throughout this presentation, we’ll use examples
that involve hypothetical groups of people
But First Things First…
Terminology used in disparities statistics:
1. Significance
2. Power
3. Confidence
9
Statistics for Disparities
 Power – ability of a statistical test to identify an effect
 We avoid discussing power in this tool, because it is not
completely scientific
 We simplify terminology and assumptions around power
to focus on whether or not results are useful
 The important thing to remember
is that UNDEFINED RESULT is
a product of low power among
other statistical assumptions
10
Statistics for Disparities
 Confidence Intervals: the lower and upper bounds of
the range containing the true result with 95%
confidence
 If a confidence interval contains the value "1",
the calculated result is not significant and should
be ignored
 Narrower confidence intervals signify lower
standard error, wider intervals signify higher
standard error
 For our purposes, we are only interested in the
confidence intervals that fall within the range of
detectable disparities based on our assumptions
11
Probability Methods
 Absolute Disparity Method
 The absolute difference in scores between two groups
 Relative Risk Method
 The ratio of the scores of two different groups divided into each other
 Comparative Disparity Method
 The relative risk minus 1 (method is used to highlight the work that has
yet to be done and supportively frames the values in terms of need
 Odds Ratio Method
 A measure of association between a status
(e.g., race) and an outcome (e.g., the number
of patients achieving or not achieving VLS)
based on odds.
12
Probability Method Limitations
 Absolute Disparity Method
 Limitation: measured scores all > 50%*
 Relative Risk Method
 Limitation: measured scores all < 80%*
 Comparative Disparity Method
 Limitation: measured scores all < 80%*
 Odds Ratio Method
 Limitation: additional calculations required
and challenging to interpret (use as last resort)
* If these thresholds are not met, the method
does not have adequate power to detect disparity
13
Why So Many Probability Methods?
 All statistical methods have limitations
 Completing all four probability methods increases the
likelihood we will detect true disparities
 At least one of the methods will always work (odds ratio),
regardless of the limitations of the other methods
 Applying all four methods in all cases allows for consistent
analysis
 The most “probable” disparity is the
group with the greatest number of
SIGNIFICANT results
14
Example
 Let’s go back to our populations example, but let’s
just look at two groups for simplicity’s sake!
15
Example
 Assume you have 100 people total and that 47 of
them are blue and 53 of them are yellow
16
Example
 Now, let’s assume that some of the people are
retained in HIV care and some are not
17
Example
 39 of the 47 blue people are retained in care (83%)
 48 of the 53 yellow people are retained in care (91%)
18
Example
 Let’s walk through each of the probability
calculations to see if there is a disparity in retention
19
“For to be free is not merely to cast off one’s chains, but to
live in a way that respects and enhances the freedom of
others.”
- Nelson Mandela
20
Some Visual Cues to Help You
 Power Slider
 Reminds us of the power limitations of each method
50%
UNDEFINED RESULT
ABLE TO DETECT DISPARITIES
 Significance Slider
 Reminds us of the significance thresholds of each method
NO DISPARITY DETECTED
21
∞
0.10
0.05
0
MAYBE
DISPARITY DETECTED
Absolute Disparity
 Does the Absolute Disparity Method qualify a disparity?
 Are all scores > 50%
50%
UNDEFINED RESULT
ABLE TO DETECT DISPARITIES
 Absolute Disparity is the difference between measured scores
 In this method:
― A difference 0% to 5% is a NO DISPARITY
― A difference 5% to 10% is a MAYBE DISPARITY
― A difference more than 10% is a YES DISPARITY
 91% - 83% = 8%
 8% falls between 5% and 10% so a MAYBE DISPARITY
0%
5%
NO DISPARITY DETECTED
22
∞
10%
MAYBE
DISPARITY DETECTED
Relative Risk
 Does Relative Risk Method Qualify Disparity?
 Are all measured scores <80%
ABLE TO DETECT DISPARITIES
80%
UNDEFINED RESULT
 Relative Risk is the ratio between measured scores
 In this method:
― A result greater than 0.875 is a NO DISPARITY
― A ratio between 0.8 and 0.875 is a MAYBE DISPARITY
― A ratio less than 0.8 is a YES DISPARITY
 83% ÷ 91% = 0.91
 Measured scores >80% so method has UNDEFINED RESULT
0.8
0
DISPARITY DETECTED
23
∞
0.875
MAYBE
NO DISPARITY DETECTED
Comparative Disparity
 Does Comparative Disparity Method quality a disparity?
 Are measured scores <80%
80%
ABLE TO DETECT DISPARITIES
UNDEFINED RESULT
 Comparative Disparity is the Relative Risk minus 1
 In this method:
― Greater than -0.125 is a NO DISPARITY
― Between -0.125 and -.02 is a MAYBE DISPARITY
― Less than -0.2 is a YES DISPARITY
 (83% ÷ 91%) - 1 = - 0.09
 Measured scores >80% so method has UNDEFINED RESULT
-∞
DISPARITY DETECTED
24
-0.125
-0.2
MAYBE
∞
NO DISPARITY DETECTED
Odds Ratio
 Odds ratios should be used as a last resort, because they can be
challenging to interpret based on the math behind them
 Odds ratios are produced through cross-multiplication
 (39 * 5) ÷ (48 * 8) = 0.53
 In this method:
YES
NO
Blue People
39
8
Yellow People
48
5
― Greater than 0.67 is a NO DISPARITY
― Less than 0.67 is a YES DISPARITY
 0.53 is less than 0.67so is a YES DISPARITY
0.67
0
DISPARITY DETECTED
25
∞
NO DISPARITY DETECTED
Let’s Review Probability Method Findings
 Absolute Disparity – MAYBE DISPARITY
 Relative Risk – UNDEFINED RESULT
 Comparative Disparity – UNDEFINED RESULT
 Odds Ratio – YES DISPARITY
 Based on the findings – there is very likely a real
disparity between red and blue ball durability
26
The Calculator Does All Four Every Time!
 All statistical methods have limitations
 Completing all four probability methods increases the
likelihood we will detect true disparities
 At least one of the methods will always work (odds ratio),
regardless of the limitations of the other methods
 Applying all four methods in all cases allows for consistent
analysis
 The most “probable” disparity is the
group with the greatest number of
YES DISPARITY results
27
Determining Priorities for Disparities
Medium
Low
IMPACT
High
FOCUS
AVOID
Low
Medium
PROBABILITY
28
High
Impact Method
 Absolute Impact
 Measures the number of lives that are affected if the
performance is raised to match average performance
 Multiply the absolute difference by the size of the
population being assessed
 Limitation: be aware whether the population
being assessed is doing better or worse than
the average
29
Absolute Impact Example
 Absolute impact is the absolute disparity multiplied by the
population size of focus (blue people in this case since their
measured performance is lower).
 Absolute Disparity is the difference between measured scores
― 91% - 83% = 8%
― 47 * 0.08 = 4 red balls
* If blue people were retained in HIV care as well as
yellow people are retained in HIV care, we would expect
an additional 4 blue people of the original 47 to be
retained in HIV care
30
Don’t Forget! The Tool Does This All For You!
31
Question & Answer
Additional disparities calculation and QI resources are available www.NationalQualityCenter.org
or email [email protected]
32
Ending disparities will end the HIV epidemic.
33
Next Steps
 This presentation is the fourth in a series of presentations
intended to familiarize you with disparities calculation
 Part I: Disparity, a National Priority
 Part II: Subpopulations
 Part III: Calculating Disparity
 Part IV: YOU ARE HERE!
 Part V: Selecting QI Projects
34
Next Steps
 Want to check out the NQC Disparities Calculator?
Click this link and you’ll be taken there!
THIS WILL BECOME AN ICON LINKED TO
CALCULATOR
 Want to learn more about the NQC Guide on this
topic? Click this link and you’ll be taken there!
THIS WILL BECOME AN ICON LINKED TO
GUIDE
35
212-417-4730
NationalQualityCenter.org
[email protected]