Transcript Research Methodology - Wright State University

Human Factors Research Methodologies
Copyright  2003 by Dr. Gallimore, Wright State University
Department of Biomedical, Industrial Engineering & Human Factors Engineering
Overview
• Descriptive Studies
– Characterize population according to attributes
• Experimental Research
– Test effect of variables on performance
• Evaluation Research
– Test system or produce
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Research Settings
•
•
•
•
Laboratory
Field
Simulation
Work Sampling
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Selecting Variables
• Independent Variables
–
–
–
–
Treatment conditions that are manipulated
Task related
Environmental
Subject related
• Most studies include only a few IVs
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Selecting Variables
• Dependent Variables
– Systems descriptive criteria
• equipment reliability, cost of operation
– Task performance criteria
• Quantity of output, quality of output, performance time, errors
– Human criteria
• Frequency measures, intensity measures, latency measures,
duration measures, physiological indices, subjective responses
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Choosing Subjects
• Representative
• Random
• Sample size depends upon
– Degree of accuracy required
– Amount of variance in population
– Statistic being used
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Collecting Data
• Descriptive Studies
– Mostly survey and interviews, questionnaires
– Important to avoid bias
• Experimental Research
– Most controlled situation for data collection
– Must be careful to select relevant experimental design
and sampling technique.
• Evaluation Research
– Researcher observation most often suffice as form of
collection
– Must work with system designers and builders to collect
data
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Statistics in a Nutshell
• Interested in Entire Population
• Large Population = Expensive Surveys
• Sample a Small Group
– Infer Population Characteristics
– from Sample Characteristics
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Examples
•
•
•
•
•
Exit polls on election day
Marketing surveys for new products
Medical trials for new vaccines
Neilson ratings of television shows
Air measurements for pollutants
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Definitions
Population --- Universe (Entire Group)
Parameter
Numeric Characteristic of a Population
Average, Variance, Standard Deviation, etc
Sample --- Subgroup of the Population
Statistic
Numeric Characteristic of a Sample
Average, Variance, Standard Deviation, etc
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Sampling
Sampling is faster, cheaper, easier.
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Sampling
Sampling is faster, cheaper, easier.
Hope, that the sample is representative of the
population; and therefore, the sample statistic
is an accurate estimate of the population
parameter.
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Sampling
Sampling is faster, cheaper, easier.
Hope, that the sample is representative of the
population; and therefore, the sample statistic
is an accurate estimate of the population
parameter.
If the sample is not representative of the
population, then all bets are off !
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Statistics
• Descriptive Statistics
– Collecting and Analyzing Data
– “Tells a story about the numbers”
• Inferential Statistics
– Drawing conclusions about a population
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Descriptive Statistics
• Measures of Central tendency
– Mean (Arithmetic Average)
– Median (Middle Value)
• Measures of Variation
– Variance
– Standard Deviation
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Measures of Central
Tendency
• Mean
–
–
–
–
Arithmetic Average
Balance Point
Basis for other statistics
Influenced by extremes
• Median
– Middle Value
– Different Balance Point
– Not influenced by extremes
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Important Statistics
• Mean
– Tells where the data are centered
• Standard Deviation
– Tells how far the data are spread out
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Analyzing Data
• Descriptive
– Central tendency: mode, medium, mean
– Dispersion: range, interquartile range, (25th-75th
percentile), standard deviation
– Effiecient, unbiased
• Correlational
• Inferential (ANOVA, MANOVA, regression)
– Results only as good as the quality of selected
variables, sampling of subjects, and experimental
design.
– Statistical significance doesn’t mean results are
“meaningful”
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Analyzing Data
• Inferential (Cont)
– Pitfalls in statistical analysis
• Type I error: rejecting a hypothesis when it is true
• Type II error: accepting a hypothesis when it is false
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Requirements for Research
Criteria
• Practical Requirements
– Objective, quantitative, unobtrusive, easy to collect,
minimal cost (money and effort)
• Reliablity
– Consistency across time and samples
• Validity (of dependent variables)
– Face validity
– Content validity (domain sampling)
– Construct validity (basic behavior)
• Freedom from contamination
• Sensitivity
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering
Measure of Human Reliability
• Probability of success due to error-free
performance
• Data bases
• Technique for human error rate prediction
• Stochastic simulation models
• Criticisms
–
–
–
–
All errors result in failures?
Sources of reliability data too subjective
Reliability data lack breadth of coverage
Now have very complex systems
Copyright  2001 by Dr. Gallimore, Wright State University
Department of Biomedical, Human Factors, & Industrial Engineering