Transcript ITAB IT Aptitude Battery

ITAB
NAMIS Product Fair
Fairfax, VA.
January 11th, 2006
© CogniMetrics, Inc., 2006
IT Aptitude Battery
Content
navigator
ITAB: IT Aptitude Battery
•
•
What is the ITAB?
ITAB and the U.S. Navy
ITAB: procedural skills
•
•
What is “procedural skill”?
Examples of “procedural skills”
ITAB: tests
•
•
•
•
•
•
•
•
Measurement purpose
ITAB 1: Hidden Target Test
ITAB 2: Battery Test
Computer game-like tests
How do ITAB tests measure PKL aptitude?
Innovative features
ITAB: Test Data
ITAB: Test Report
ITAB: Information
•
•
© CogniMetrics, Inc., 2006
ITAB: technical information
ITAB: access information
The compass icon appears on many slides in
the slide show. Click on one of its arrows to
return to the navigator page.
ITAB
IT Aptitude Battery
What is the ITAB?
The ITAB is the world’s first battery
interactive tests of human intelligence.
A product of:
CogniMetrics, Inc.
San Antonio, TX
© CogniMetrics, Inc., 2006
of
fully
ITAB
IT Aptitude Battery
ITAB and the U.S. Navy
The Navy Mission requires its personnel to be able to
operate a great variety of technical systems. The
cognitive skill required for using such systems is
called procedural knowledge, or procedural skill.
Traditional tests of intelligence, such as the ASVAB,
were not designed to measure aptitude for
procedural skill learning.
© CogniMetrics, Inc., 2006
ITAB
procedural skills
What is “procedural skill”?
Procedural skill is knowledge that a person uses to
interact with a physical, or conceptual, system to
achieve a particular goal.
© CogniMetrics, Inc., 2006
ITAB
procedural skills
Examples of “procedural skill”
Procedural skills vary in complexity from performing
operator maintenance on a M16A1 rifle, to preparing
board weapon systems for operation, to operating
board weapon systems, and to remotely controlling
UAVs.
Cognition research in the nineties has deepened our
insight into how procedural skills are learned.
© CogniMetrics, Inc., 2006
ITAB
tests
Measurement purpose
The ITAB tests measure the aptitude to learn
procedural skills (procedural knowledge).
We will refer to this aptitude as PKL aptitude
(procedural knowledge learning aptitude).
© CogniMetrics, Inc., 2006
ITAB
tests
ITAB 1: Hidden Target Test
HELP
menu
<MOVE> <TEST> keys
Task: The Interface of the Hidden Target
test generates a two-dimensional search
space consisting of a rectangular grid of
equally spaced horizontal and vertical
lines.
The subject is required to determine where
a target is located in as few steps as possible.
The subject can move a cursor across the
two-dimensional space from junction to
junction using a set of arrow keys. To indicate
his or her first guess the subject clicks at the
<TEST> key.
The feedback of the task-environment indicates
the distance between the tested location and
the location of the hidden target in City Block
metric..
© CogniMetrics, Inc., 2006
<START> bar
ITAB
tests
ITAB 2: Battery Test
HELP MENU
Task: the test requires the individual
to test whether the presented batteries
work. Each new trial starts with a set
of nine batteries to be tested. The
battery tester is designed such that
batteries have to be tested in pairs.
The goal of the task is to identify the
defective batteries in as few steps as
possible. Each set of nine batteries
contains two defective batteries.
<START> button
Feedback window
© CogniMetrics, Inc., 2006
<TEST> button
<REJECT> button
ITAB
tests
The feel of a computer game
Combining a game-like character and an intuitive
interface gives ITAB tests an advantage in testing
populations that have developed an aversion against
school math and the static logic of traditional
intelligence tests.
© CogniMetrics, Inc., 2006
ITAB
tests
How do ITAB tests measure
PKL aptitude?
The tests provide an interactive task environment in
which the individual has to achieve a goal. The tests
measure:
• how feedback is incorporated into follow-up actions
• how quickly this leads to a more or less efficient
procedure
© CogniMetrics, Inc., 2006
ITAB
tests
Innovative features
1.
© CogniMetrics, Inc., 2006
ITAB tests provide an (interactive) task
environment.
2. Actions of individuals are not scored as singleton
answers to distinct problems, but analyzed as
sequence patterns.
======
3. The test report is organized according to an
empirically tested structural equations model.
CONTINUE
ITAB
tests
ITAB:
TEST DATA
Each ITAB test provides an interactive taskenvironment.
An examinee action (user action)
changes the environment (system action) and the
examinee receives feedback about the new state of
the task-environment (system output).
The diagnostic components of the ITAB tests
measure these changes per user step and analyses:
© CogniMetrics, Inc., 2006
•
sequential aspects of the user actions
•
The informational
system state
content
of
each
new
ITAB
tests
ITAB:
TEST REPORT
The diagnostic components of the ITAB tests
organize the many measurements into three
scores that are intuitively accessible.
The individual test reports provide these
scores for prediction purposes as well as
detailed analysis of test performance that may
help understanding and improving it.
© CogniMetrics, Inc., 2006
ITAB
tests
ITAB:
TEST REPORT
e1
e2
e3
e8
systematic
approach
ENT_B
e4
PKL(B)
e5
intelligence
e6
e7
© CogniMetrics, Inc., 2006
e9
MEAN_BRT
e10
VAR_BRT
e11
ITAB
tests
ITAB:
TEST REPORT
e1
e2
e3
e8
systematic
approach
ENT_B
e4
PKL(B)
e5
intelligence
e6
e7
© CogniMetrics, Inc., 2006
e9
MEAN_BRT
e10
VAR_BRT
e11
ITAB
tests
ITAB:
diagnostic information
TEST REPORT
predictor score
e1
e2
e3
e8
systematic
approach
ENT_B
procedural
e4
learning
aptitude score
PKL(B)
e5
intelligence
e6
e7
© CogniMetrics, Inc., 2006
e9
MEAN_BRT
e10
VAR_BRT
e11
ITAB
tests
ITAB:
diagnostic information
TEST REPORT
predictor score
e1
e2
e3
e8
systematic
approach
e4
PKL(B)
e5
intelligence
e6
e7
© CogniMetrics, Inc., 2006
consistent and careful
work style
ENT_B
e9
MEAN_BRT
e10
VAR_BRT
e11
efficiency information
processing
ITAB
tests
ITAB:
TEST REPORT
e1
e2
e3
e8
systematic
approach
ENT_B
e4
PKL(B)
e5
e9
MEAN_BRT
e10
VAR_BRT
e11
intelligence
e6
e7
© CogniMetrics, Inc., 2006
Battery Test
ITAB
tests
ITAB:
TEST REPORT
e1
e2
e3
e8
systematic
approach
ENT_H
e4
PKL(H)
e5
e9
MEAN_HRT
e10
VAR_HRT
e11
intelligence
e6
e7
© CogniMetrics, Inc., 2006
RETURN
Hidden Target Test
ITAB
information
ITAB:
Measurement claim:
Number of tests:
Administration time:
Languages:
CogniMetrics, Inc.
March 2004
see: www.cognimetrics.com
bulk discounts negotiable
Reliabilities:
HTT
BT
FOR MORE
•
•
construct validity studies:
INFO...
•
predictive validity studies:
Download ITAB manuals:
www.cognimetrics.com
Report on construct validity: www.internationalmta.org/2004
(paper 066; powerpoint presentation 066).
© CogniMetrics, Inc., 2006
procedural knowledge handling
(aspect of general intelligence)
2 (HTT and BT)
10 minutes per test (average)
English, Spanish, German,
Dutch
Test Publisher:
Published:
Price:
Validities:
•
Technical Information
(0.88 – 0.97)
(0.80 – 0.87)
see: FOR MORE INFORMATION
currently under way with:
- Air Force Research Lab, Brooks
City Base;
- Navy Directorate Selection and
Classification, Washington, D.C.
ITAB
information
ITAB:
© CogniMetrics, Inc., 2006
Access Information
Web site:
www.cognimetrics.com
Select
<login tests>
Select language:
English, Spanish,
German, Dutch
Username:
Password:
Test:
ASMR
ASMRS
Hidden Target
ASMR
ASMRU
Battery Test
Thanks for reviewing this
CogniMetrics proposal
For further information,
Please contact:
Dr. Martin J. Ippel
(210) 481 – 0261
[email protected]
© CogniMetrics, Inc., 2006
ITAB
tests
Interactive Task Environment:
Complete interactivity is achieved by creating an
internal representation of the task-environment.
Artificial Intelligence technology is used to compute
the “intelligence” of each step.
© CogniMetrics, Inc., 2006
RETURN
ITAB
tests
Analysis of behavior sequences
© CogniMetrics, Inc., 2006
RETURN
Unlike traditional tests, ITAB tests do not consists of
items with a standard set of responses.
The
individual is free to act within the task-environment.
The individual must produce a sequence of actions to
achieve a goal, while using the feedback provided by
the task-environment. Performance assessment is
based on analysis of these action sequences.