Transcript Sample size and sample tracking tool

The Random Sampling/Tracking
Tool: A Response to OverSurveying
Steve Graves
Intel Corporation
Overview
 Problem Statement
 Recommended Solution
 Features
 User Interface
 Demo
 Benefits
 Concerns
 Next Steps
 Possible tool enhancements
 Beyond Intel
 Summary
Problem Statement
 Perception that too many surveys are deployed to
end users from IT – Source: Annual Partnership
Excellence Survey with top Intel Execs
 Likely causes identified (based on experience as a
survey consultant)
– Groups tended to survey entire population of interest
instead of a sample (e. g. survey of habits of frequent
travelers -- 26,000 people)
– When criteria for a population is only partially available a
screening survey is necessary – results in more people
surveyed
– Simple random sampling will tend to hit a small
percentage of people multiple times
General
Population
Study A
Study B
Same
Participant
Problem Statement
 Effective and efficient process to stop oversurveying does not exist
– Previous process relied on incomplete information and
“Guardians” – go between groups
– No mechanism to track who has received surveys
– Guardians may be overly cautious
– Meetings used to determine what to do – time consuming
and inefficient
– Process is not followed universally
Recommended Solution
 Sampling tool containing a repository of
Intel employee data
– Utilizes data from employee database (nonconfidential)
– Generates random samples of target audience
– Tracks who has been sampled over a given time
period – currently 5 months
– Sampled individuals will not be selected again
for during this period
Recommended Solution
Database perspective
Employee Database
Retrieval Database
Sampling Database
Sampling Database
Randomly Selected Employees
Tracking Table
Storage Database
Recommended Solution
Database perspective – previously sampled individuals
removed before Sample is generated
Tracking Table
Generated table based on Criteria
Table that meets criteria and contains no excluded individuals
Recommended Solution
 Incorporate into an overall process where
researchers are required to utilize the tool and
consult with sampling advisors
– Advisors can advise researchers on appropriate sampling
techniques – so researchers to survey an entire
population (e.g. 3,000 instead of 26,000)
 Currently implemented using RAD database
programming language
–
–
–
–
Tool has been in use for almost 2 years
Tool is performing reliably and customers are satisfied
Usage is only through word of mouth
Approximately 60 samples generated last year
Sampling Tool Features
 Current features
– Tracking and recording mechanism
– Recording of samples generated by others
– Random Sampling of entire Intel population
– Random sample from group that uses criteria not part of
employee database (for example: Employees who use
PDAs)
– Random sampling of Sub-Groups (For example:
Employees in Folsom California)
– Stratified sampling
User Interface
Demo (Double Click to Start)
Benefits
Saves researchers time
–Reduces the need for screening surveys
–No need to re-invent the wheel
–Saves time compared to previous
process
Benefits
Improvement in sampling process
–Reduce or eliminate “over-sampling”
– Delay surveys when target population is not
available
– Researchers receive consultation on using
appropriate sample sizes
–Distribute sampling more evenly across
groups
Concerns
May cause sampling bias in some
situations
Next Steps
 Need to develop more sophisticated business rules
around sampling process
 Currently working to make this an IT wide process
– Early results are promising
 Looking at making it an Intel wide process
 Port to SQL server and have a web based solution
– Current access is through Remote Desktop Connection
 Develop strategies to mitigate sampling bias
Possible Future Tool
Enhancements
 Complex samples through UI – currently
requires custom query
 Linking to Other database that contains
product usage information
Beyond Intel
Similar tools could be used in similar
situations in other companies or
institutions
Summary
Problem: Over-surveying by our IT
organization
Response: Random Sampling/Tracking
Tool in conjunction with business rules
to manage survey sampling process
Benefits: Saves researchers time,
reduces over-surveying, improves
sampling process
Working to make this an IT wide official
process and eventually Intel wide
Summary
Next steps:
–Link to other databases
–Improve business rules
–More robust software platform
–Strategies to mitigate sampling bias
Similar tools/business rules could be
used in other institutions