Transcript Testing

Tools for Successful
Selection
Employment Testing: General
Types of Employment Tests

Ability Tests
• Mental (Cognitive) Ability
• Psychomotor Ability
• Physical Ability
• Sensory Ability

Achievement and Skill Tests
• Job Knowledge
• Performance/Work Sample
Types of Employment Tests


Personality/Interest Inventories
Other Types of Tests
• Integrity (honesty)
• Drug
Selecting Employment Tests


Which KSAs are relevant to
testing?
Search sources of tests
• General


Tests: A Comprehensive Reference...
Web-based information sources
• Reviews

Mental Measurement Yearbook
• Useful article on using the MMY


British Psychological Society
Test Critiques
Evaluating Test Manuals


Evidence of reliability
Relevant evidence of validity
• Use of appropriate criteria
• Norms, based on appropriate samples


Evidence of test fairness
Practicality
Tools for Successful
Selection
Cognitive Ability (Intelligence)
Tests
What is “Intelligence”?
And is it
relevant to job
performance??
“Successful” Intelligence
Ability to achieve success in life and work;
requires a balance among:

Analytical intelligence
• “Academic” intelligence

Practical Intelligence
• “Common sense” as well as social or
emotional intelligence; “street smarts”

Creative Intelligence
Analytical Intelligence

Traditional aspects of intelligence
measured by standard tests
• Verbal and numerical ability, memory,
processing speed

Predicts job performance in nearly all jobs
• More important as jobs become more complex
• Probably due to:



Broader knowledge base
Ability to learn on the job more quickly
Ability to problem-solve
Practical Intelligence

Ability to deal with everyday life challenges
• E.g., computing lowest price/unit when grocery
shopping, or interpreting a bus schedule

Social/emotional intelligence
• Ability to understand people and relationships
• Impression management skills
• Ability to persuade others



Adaptability to different social situations
Shown to add to predictive power of
analytical intelligence for sales and execs
May be culture-specific
Creative Intelligence

Relies on a base of knowledge +
ability to apply it to new situations or
problems
• Ability
ideas
• Ability
define
• Ability
to generate novel high-quality
to recognize opportunities or
challenges in new ways
to adjust to new situations
Measuring Intelligence

Wide variety of measures of analytical
intelligence/general mental ability
• Wonderlic Personnel Test is most popular


Sternberg Multidimensional Abilities Test
measures all 3 intelligences
Sternberg and Wagner’s test of Practical
Managerial Intelligence:
•
•
•
•
ability to write effective memos
ability to motivate people
knowledge of when to delegate
ability to “read” people
Lingering Controversies about
IQ



Is intelligence testing fair?
Is intelligence—even when measured
more broadly—sufficient?
Do you want “bright” or “well
trained” employees?
Fairness and Intelligence
Testing: A Dilemma

CA predicts performance in a wide
variety of jobs
• Measures knowledge
• Measures ability to learn, reason, and
solve new problems
• Few alternative predictors with
comparable utility for entry-level jobs

But are they “fair” to all groups?
True Validities of Predictors
For Entry-Level Jobs
Ability Test Battery
Interview (structured)
Job Tryout
Biographical Inventory
Reference Checks
Personality Inventories
Experience Ratings
Interview (unstructured)
Academic Achievement
Interest Inventories
For Promotions
Work Sample
Ability Test Battery
Peer Ratings
Experience Ratings
Job Knowledge Test
Assessment Center
.53
.44
.44
.37
.26
~.23
.18
.14
.11
.10
.54
.53
.49
.49
.48
.43
Definitions of Test “Fairness”
1. Differences between groups’ average test
scores
• ~1 SD difference for African-Americans
• ~.75 SD for Latinos
But what do these differences mean?
• Groups differ in ability?
• Test discriminates against one group?
Definitions of Test “Fairness”
2. Differential Validity: Group
differences in validity of test
Definitions of Test “Fairness”
2. Differential Validity: Group
differences in validity of test
White
Job
Perf
Black
Test Scores
Definitions of Test “Fairness”
3. Test Bias: Group differences in
predicted job performance with same
cutoff score
Definitions of Test “Fairness”
3. Test Bias: Group differences in
predicted job performance with same
cutoff score
Job
Perf
Black
White
Test Scores
Definitions of Test “Fairness”
3. Test Bias: Group differences in
predicted job performance with same
cutoff score
Job
Perf
Black
White
Test Scores
Typical CA Test Situation



Adverse Impact
No differential validity
No test bias
White
Job
Perf
Black
Test Scores
Typical CA Test Situation



Adverse Impact
No differential validity
No test bias
White
Job
Perf
Black
Test Scores
Minimizing Adverse Impact





Recruit high quality applicants
Use tests only when appropriate
Use tests as one part of process
Use to eliminate unqualified (rather
than to differentiate among most
qualified)
Be sure test is valid (job-related)!!
Lingering Controversies about
IQ



Is intelligence testing fair?
Is intelligence—even when measured
more broadly—sufficient?
Do you want “bright” or “well
trained” employees?
Why Intelligent People Fail







Lack of motivation
Lack of impulse control
Lack of perseverance
Fear of failure
Procrastination
Inability to delay gratification
Too little/too much self-confidence
(1986)
Source: Sternberg
Lingering Controversies about
IQ



Is intelligence testing fair?
Is intelligence—even when measured
more broadly—sufficient?
Do you want “bright” or “well
trained” employees?
Intelligence or Training?


Is being smart a pre-requisite for
learning?
Is being smart sufficient for learning?
Tools for Successful
Selection
Personality Testing
Invasiveness



Employer right to know vs. right to
privacy
Over-emphasis on individual items
Reducing invasiveness:
• Use only for job-related traits
• Use employment (not clinical) measures
• Use as part of overall system
• Explain purpose as matching not
screening
Extreme Items from MMPI
I have never indulged in any unusual sex practices.
I have never been in trouble because of my sex behavior.
My soul sometimes leaves my body.
Evil spirits possess me at times.
I sometimes feel like I’m going to pieces.
Often I feel as if there were a tight band around my head.
I have had no difficulty starting or holding my bowel movements.
Sample Items Modeled After Those on NEO-PI
I am generally not a worrier.
I would rather cooperate than compete.
I don’t mind bragging about what I’ve accomplished.
I’m pretty set in my ways of doing things.
I try to keep all my thoughts realistic and to avoid flights of fancy.
I have no sympathy for “street people.”
I prefer jobs that allow me to work alone without being interrupted by others.
I am a person who always gets the job done.
Is Personality Job-Relevant?

The “Big Five” Personality Traits
• Conscientiousness
• Emotional Stability
• Extraversion
• Agreeableness
• <Openness to Experience>

Others may apply in particular jobs
• e.g., customer service, safety, attention
to detail
Do Personality Tests Predict?



Is the situation weak or powerful?
Is the measure valid?
Validities typically in low .20s (when
based on job analysis)
Faking & Personality Tests

Applicants often distort their answers
to look good
• Some applicants more than others
• This can affect who is hired

Reducing faking:
• Use faking (lie) scales
• Reduce motivation to fake
• Use less transparent questions
Social Desirability Scale Items
I sometimes drive faster than the speed limit.
My table manners are not quite as good at home as when I’m out in company.
I gossip a little at times.
Sometimes I vote for people I know very little about.
I do not like everyone I know.
I sometimes tell lies if I have to.
Fairness of Personality Tests


Little evidence of racial
discrimination
Men and women differ on many
scales
• but no evidence of “discrimination”

Watch out for adverse impact due to:
• religious beliefs
• disability
Ways to Assess Personality



Paper & Pencil Inventories
Individual (clinical) assessment
During interviews
Ways to Assess Personality
Correlation Between Self- and Interviewerr
Extraversion
0.27
Conscientiousness
0.22
Openness to Experience
0.11
Agreeableness
-0.29
Emotional Stability
-0.30
Ways to Assess Personality





Paper & Pencil Inventories
Individual (clinical) assessment
During interviews
Reports from others
Behavioral observation
Tools for Successful
Selection
Assessing Deviance:
Drug Testing
Integrity (Honesty) Testing
What is “Job Performance”?
In-Role Behavior
ExtraRole
Behavior
CounterProductive
Behavior
Risk
Factors
General Rationale

Employee deviance is a costly
problem
• $6 to 200 billion annual cost
• 33-75% of employees


It’s said that 30-50% of business
failures are due to theft
Drug/alcohol abuse results in $197B
in productivity costs
How Do You Screen out
Deviant Employees?

Some are based on past behavior
• analysis of drug metabolites in urine;
past theft; reference and background
checks

Others focus on likely future
behavior
• attitudes toward theft, drug use;
personality

Only a few focus on current behavior
• impairment testing
Physiological Drug Tests

Urinalysis
• Quick screening
• Confirmatory: Gas
Chromatography/Mass Spectrometry
(GC/MS)



Hair analysis
Blood analysis
Breath analysis
What’s Measured (& for how
long?)

The “NIDA 5”:
• Cannabis
• Cocaine
• Amphetamines
• PCP
• Opiates
What’s Measured (& for how
long?)

The “NIDA 5”:
• Cannabis
• Cocaine
• Amphetamines
• PCP
• Opiates
2 – 90 days
1 – 2 days
.5 – 2 days
30 – 90 days
2 – 4 days
Other Forms of Drug Tests


Paper-and-pencil integrity tests
Impairment testing
• based on person’s own “baseline”
performance on a computer task
Types of Drug Testing
Programs


Pre-employment
“For cause” testing
• mandatory post-accident
• based on supervisor’s observations

Fitness for duty
• daily (impairment testing)
• post-rehabilitation

Random
Types of Honesty/Integrity Tests


“Clear Purpose” tests
“Disguised Purpose” tests
• Attitudes about deviance
• Personality

Polygraph/Voice Stress Analyzer
Clear Purpose Test
- Honesty/Morality Items




If you received an extra $1.00 in
change when you purchased
groceries, would you keep it?
If you saw a co-worker stealing,
would you turn her in?
How many people have you known
who have stolen valuables at work?
Most people cheat on their income
tax.
Clear Purpose Test:
- Drug-Related Items




I have never been a heavy drinker.
On occasion I have used
alcohol/drugs excessively
If I thought I could get away with it,
I'd probably try cocaine
Have you ever missed work because
you were too high/drunk?
Types of Honesty/Integrity Tests


“Clear Purpose” tests
“Disguised Purpose” tests
• Attitudes about deviance
• Personality

Polygraph/Voice Stress Analyzer
Disguised Purpose Test
- Drug Attitude Items



A drug addict who steals to support
his habit should get the same
penalty as a person who steals for
profit.
Employers expect a certain amount
of drug and alcohol use among their
workers.
How many people have you known
who drink or do drugs while working?
Disguised Purpose Test:
- Personality Items





I enjoy making people feel better.
I don't care if others like the things I
do.
I shouldn't do many of the things I
do.
I have never been in trouble with the
law.
Sometimes I enjoy going against the
rules.
Types of Honesty/Integrity Tests


“Clear Purpose” tests
“Disguised Purpose” tests
• Attitudes about deviance
• Personality

Polygraph/Voice Stress Analyzer
Drug & Integrity Testing Issues





Legality
Accuracy/Job relevance
Faking (integrity tests)
Consequences of failing
Negative reactions (invasion of
privacy/fairness)
Integrity Testing: A Caution


Integrity tests assume cause of
deviance is bad employees
But the situation may also be a
factor
• role models, group norms may “push”
people towards deviance
• dissatisfaction may “pull” towards
deviance

Integrity tests don’t replace good
management
Tools for Successful
Selection
Employment Testing:
Performance Tests & Work
Samples
Signs vs Samples


Signs: indirect indicators of
performance potential
Samples: direct indicators of
performance potential
Sign-Sample Continuum

Probationary employment
Apprenticeships
Internships
Mock-ups
Assessment centers
Role-playing interviews

Paper/pencil tests





Sample
Sign
Pros & Cons
Advantages



High validity
Low adverse impact
High face validity
Disadvantages
• More time consuming
• Direct costs
Step 1: Define Performance

Which performance dimensions?
• Refer to Performance Attributes Matrix
• Don’t focus only on easily simulated
dimensions
Performance Attributes Matrix
App.
Attribute
Blank Interview
Reference
Work
Checks
Sample
Knowledge Personality
Test
Inventory
Ability to work
accurately
X
X
X
X
Ability to work
well with customers
X
X
X
X
Work-specific
knowledge
X
Computeroperating skill
X
Conscientiousness
X
X
X
X
X
X
Step 1: Define Performance

Which performance dimensions?
• Refer to Performance Attributes Matrix
• Don’t focus only on easily simulated
dimensions


Determine reasonable standards for
the test/sample
What type of validation would you
use?
Step 2: Creating Sample

Key principles
• Specificity
• Comprehensiveness (vs Narrowness)
• Fidelity
• Practicality

Use Performance Attributes Matrix
to:
• Ensure all KSAs are included in plan
• Weight all KSAs appropriately
Example:
Marketing Specialist

Performing market analysis
• Have candidates perform a market
analysis for a particular product/market

Developing appropriate marketing
materials
• Critique current materials

Developing a marketing plan
• Develop part of a plan