Transcript Sources of Error in Estimating Community Health Center Physicians

Sources of Errors in Estimating
Community Health Center Physicians
Catharine W. Burt , Ed.D.
Esther Hing, M.P.H.
Division of Health Care Statistics
June 3, 2008
Centers for Disease Control and Prevention
National Center for Health Statistics
Objectives
 To
assess coverage error among CHC
physicians in the National Ambulatory
Medical Care Survey (NAMCS)
 To estimate bias and reliability in
percentage of CHCs using electronic
medical records
NAMCS
Survey
Type of Data
Years
fielded
Current sample
size
(approximate)
National
Ambulatory
Medical Care
Survey (NAMCS)
Visits to
office-based
physicians
19731981,
1985,
1989present
3150
physicians
25,665
encounters
NAMCS Methodology
• National probability sample survey of officebased physicians
• Complex sample design
• 112 geographic PSUs
• Physicians stratified by specialty
• Sample of visits within physicians
• Sample frame: AMA and AOA masterfiles
• Data collected by Census Bureau
Scope of the NAMCS
 Physicians



must be:
Primarily engaged in office-based, patient
care
Nonfederally employed
Not in anesthesiology, radiology, or pathology
In-Scope NAMCS Locations








Freestanding private solo or group
practice
Freestanding clinic/urgicenter
Neighborhood medical and mental health
centers
Privately operated clinics
Non-Federal government clinic
Health maintenance organization
Community health center
Faculty practice plan
Methods

In 2006, dual sample of physicians: traditional +
separate stratum of 104 CHCs
 Sampling frame: NACHC and IHS
 Random selection of 3 providers within each
CHC


Physicians and midlevel providers
After selection of providers, normal NAMCS
procedures are followed
• In-person induction interview
• Sample of ~30 visits during sample week
Coverage error
 Compare
the traditional sample of CHC
physicians with the separate stratum of
CHC physicians on physician
characteristics
Traditional
Separate Stratum
Content bias and consistency
 Use
separate stratum to estimate EMR
use
 We have multiple providers in the sampled
CHCs to answer the EMR questions
 Measure the reliability of the item
response with Cohen’s kappa statistic
Results
Traditional
sample
(n=1,311)
Separate Stratum
(n=156)
6,775
Traditional
sample
MDs who
work in
CHCs
(n=31)
8,596
39.4%
(nonofficebased)
60.6%
Comparison of percent distributions
Physician
characteristic
Traditional
sample - CHC
Separate CHC
stratum
Age
<35
35-44
45-54
55-64
65+
8.0
47.8
13.2
19.2
11.9
17.3
30.7
29.6
18.9
3.5
Gender Male
Female
44.9
55.1
41.6
58.4
MSA status
MSA
Non-MSA
87.8
12.2
93.1
6.9
EMR use among CHC physicians
33.6
Percent of physicians
35
30
25.2
25
20
15
10
5
0
Tradional sample
No significant difference @ p=.05
Separate stratum
Response consistency of physicians
reporting use of electronic health record
systems in their CHC
Provider 2
Provider 1
EMR
No EMR
EMR
8
4
12
No EMR
5
45
50
13
49
62
n=62 CHCs with multiple providers who answered the EMR items
The test-retest counts yield an I =“Index of inconsistency”
=0.214
It can be shown that Cohen’s kappa statistic=1-I
=0.786
indicating moderate reliability.
Summary
 Separate



stratum of CHCs
increases sheer volume of CHC providers
(reduces RSE by 23.8%)
Indicates little bias in key characteristics in the
traditional NAMCS
Allows for a consistency measure of item
response for EMR use