Transcript 20 OR for statin prescription in high risk patients

Evaluation Overview
Wanda Hemsworth
Dr. Finlay McAlister
Dr. Merril Knudtson
April 14, 2008
1
Agenda

Expected Outcomes from the Evaluation

Definitions & Principles

Evaluation Framework / Plan

Key Performance Indicators

Data Collection Process

Data Sources

Lessons Learned

Importance of Local Data Collection

Regional Cardiac Data from APPROACH
2
Expected Outcomes from Evaluation
WTMSC
Accountability
/Requirement
s Met
Viability &
sustainabilit
y of
provincewide rollout
Improved
access to
cardiac
health
services
demonstrate
d
Increased
patient &
practitioner
satisfaction
demonstrate
d
Alignment
with Alberta
Quality
Matrix
Shown
Improved
compliance
with care
guidelines
demonstrate
d
Improved
health
outcomes
demonstrate
d
Baseline for
future
initiatives
Regional
differences /
similarities
highlighted
Data source
for research
papers
Scorecard for
ongoing
performance
monitoring
Shared
learnings for
policy &
program
development
3
Definitions and Principles
• Describes what happens subsequent to the delivery of the project
Evaluation
• Assesses whether the expected outcomes have been achieved
• Determines the overall project impact and estimates the relative costs
Indicator
• Key Performance Indicators are quantifiable performance measurements used
to define success factors and measure progress toward the achievement of
goals
• Conducted to maximize credibility and objectivity of results
• Open and transparent
Principles
• Breadth and depth is reasonable and possible given the realities of the
environment
• Opportunities to share results and lessons learned is optimized
4
Strong, well-developed Evaluation
Framework / Plan
1. Evaluation Objectives
2. Evaluation Questions
3. Indicators
4. Data Elements
5. Data Sources and Primary Research Projects
6. Data Collection Methods
7. Analytical Methods
Evaluation Framework Building Blocks
Program
Goals
Benefit / Expected
Outcome
Planned Intervention
/ Activity
Evaluation
Questions
Key Performance Measures
Potential Data Source / Data
Collection Plan
Analysis Plan
AH&W Quality Matrix
Support
Initiative Profile
5
Selection of Appropriate Key
Performance Indicators
Regions/Sites
Programs/Indicators
Early
Reperfusion
Site 2
(urban)
(rural)
Site 1
Site 2
Site 1
Site 2
Site 1
Site 2
m
In
Site 1
Region 9
Ag
r
di eed
ca P
to ro
rs gr
a
Region 1
(total sites
xx )

How Sites were selected:
Level 1
Heart
Failure
Level 2
Level 3
• Patient Relevance (availability of data)

Level 4
• Capacity (cycles to undertake)
Patient
Navigation

Evaluation ER
Plan
HF


PN

• Readiness to Deploy (resources,
physicians)
• Balance of Coverage (Programs - Regions)

• ACAC Budget (available funding/budget)
rs
How Indicators were selected (by site):
ato
Indic
ram
Prog ites
S
from
• Representative Population Base
• Optimal Program Coverage & Site Contribution
• Noticeable improvement expected within evaluation timeframe
• Indicators should be Outcome based
• Demonstrate achievement of stated goals
• Definitions consistent across implementation variations
• Baseline available or easily determined
• Data easily collected & measured
6
Examples of Key Performance Indicators
Access to Cardiac Services
 Time to lytics < 30 minutes
 Time to primary PCI < 90 minutes
 Time from referral to initial HFC visit < 4 wks
 % of HF patients enrolled in a Heart Failure
Clinic
 Time from onset of symptoms to GP visit
 # of services booked by nurse navigator
Satisfaction
 Patient Satisfaction
 Clinician Satisfaction
 Provider Satisfaction
 Interviews with Project Teams
Compliance with Care Guidelines
 # of STEMI patients provided with discharge
instructions in compliance with Safer
Healthcare Now guidelines.
 # and % patients receiving education and
discharge instructions from HFC before
hospital discharge
 # of appropriate referrals
Project Success
 Number and type of multidisciplinary teams
formed; # of staff recruited & trained
 # and type of resources (e.g. tools) and
interventions developed and made available
through patient navigation system
 Number of multi-regional meeting/working
sessions held
 Number of sites participating
 Financial impact
7
Flexible Data Collection Process
External Evaluators
Conduct Data
Aggregation &
Analysis
Compile Evaluation
Report
Submit Evaluation
Report to Alberta
Wait Times Steering
Committee
Alberta Health and
Wellness
Provide data for
comparison and
supporting purposes
Surveys
Databases
Chart Reviews
Spreadsheets
Evaluation
Repository
Individual Sites
in Each Region
Collect Data
Regional Data Collection
Coordinators
Collect raw data from each site
in their region
Strip Identifiable patient
information from raw data
Conduct Data Quality Checks
Data transferred to the
Evaluation
Repository in
Capital Health
through secure
channels (i.e. SFTP)
Evaluation
Report
8
Use of Existing Data Sources:
e.g. Alberta Health and Wellness Data
Alberta Inpatient
Discharge Abstract
Database (DAD)
Alberta Health
Insurance Plan
Registry
Alberta Physician
Claims Database
Ambulatory Care
Database (ACCS)
Blue Cross
Medication
Database
Cohort 1: Congestive Heart Failure
(ICD-9-CM: 428;
ICD-10: I50)
Cohort 2: Acute Coronary
Syndrome
(ICD-9-CM: 410, 411, 413;
ICD-10: I20, I21, I22, I24)
Cohort 3: Non-Acute Ischemic
Heart Disease
(ICD-9-CM: 414;
ICD-10: I25)
Cohort 4: Ventricular Arrhythmia
(ICD-9-CM: 426, 427;
ICD-10: I44-I49)
Baseline: fiscal years 1999/00 to 2006/07;
Post-implementation: fiscal year 2007/08
Evaluation
Repository
9
Lessons Learned


Collaborative and iterative approach to defining indicators

Involvement of project teams, program co-chairs in identification / definition

Workshops with regions to clarify and identify issues / variations
Desire to measure everything of interest

Focus on measures that will showcase improvement / achievement of goals

Balance local collection of data with provincial data sources

Factor in regional variations (implementation plans, processes,
definitions) where possible when determining indicators

If variations are too great, consider selecting other indicators

Leverage project teams’ and regions’ past experiences with research and
evaluation

Look for solutions that take into consideration:

Limited capability and resources in field to collect data

Aggressive timelines
10
Why do we need primary data collection too?

The Treatment-Risk Paradox in coronary disease – a cautionary
tale

Some pts are less likely to be prescribed proven efficacious Rx:
 Older pts
 Women
 Minorities
 Socially Disadvantaged
 Multiple co-morbidities
11
Lipid lowering therapy with statins in highrisk elderly patients
OR for statin prescription in high risk pts vs. others:
Ko, D. T. et al. JAMA 2004;291:1864-1870.
Copyright restrictions may apply.
0.75
12
Explanations for the Treatment-Risk
Paradox
1)
“…physicians may have misconceptions about the benefitharm tradeoff”
2)
“…physicians may prejudge the compliance of their
patients”
3)
“…may be explained by physician inattentiveness to
cardiovascular prevention”
“…the survival benefits of statin therapy may never be
fully realized until physicians appropriately attune their
prescribing behaviors to the risk profiles of their pts”
13
Editorial in Am Heart J

“…it is the premise of matching risk to level of care that
physicians fail to accept, heed, or understand”

Blazing M. Am Heart J 2005;149:381-383.
14
Is the treatment-risk paradox really
due to clinician bias?
 Prospective cohort study
 3871 pts with CAD in AB cath labs between Feb 2004 and
Nov 2005
 Excluded deaths/CABG during index hospitalization
 Detailed CLINICAL data at baseline
 Mean age 64 (52% younger than 65), 78% men, Duke
Jeopardy score mean 38%
15
Co-morbidities
70
60
50
40
30
20
10
0
HTN
DM
CKD
COPD
Smoker
16
Use of EBM meds
70
60
50
40
Low risk
Medium risk
High risk
30
20
10
0
Statins
ACEi
ASA
All 3
17
OR for statin prescription in high
risk patients
28% less likely to get statin if “high risk”
18
OR for statin prescription in high
risk patients
22% less likely to get statin if “high risk” after adjustment
19
OR for statin prescription in high risk
patients
20
Bottom line
There is no treatment-risk mismatch if you include
all clinical variables, including coronary anatomy,
patient functional status, and quality of life (none
of which are in administrative data)
21
So…
Administrative databases don’t capture the
richness and subtleties of clinical care
22
APPROACH 2003-2007
Procedure Rates by Fiscal Year
Age & Sex adjusted
2003
2004
2005
2006
2007*
CATH rate
480
515
496
481
462
PCI rate
186
204
192
182
174
84
73
72
65
61
CABG rate
*2006 and 2007 use the same population numbers - new population numbers not available
23
Age and Sex-Adjusted Catheterization Rates
by RHA
700
Catheterizations 2003
Catheterizations 2004
Catheterizations 2005
Catheterization 2006
Catheterization 2007
600
500
400
300
200
100
0
Chinook
Palliser
Calgary
David
East Central
Thompson
Capital
Aspen
Peace
Country
Northern
Lights
Total
24
BMS and DES Utilization by Quarter
(Fiscal 2003 – 2007)
100
80
DES
BMS
STENT
DES_FMC
DES_UAH
DES_RAH
60
%
40
20
0
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
2003
2004
2005
2006
2007
25
Risk-adjusted 30-Day AMI Mortality Rates
2000-02
2003-05
14
12
10
8
%
6
4
2
0
Chinook Palliser Calgary
DTHR
E
Central
Capital
Aspen
Peace
Country
www.CIHI.ca/ Indicators
26
Questions?
27