Transcript Presentation

Beyond Task/Technology Fit:
How Information Technology
Affects Performance
By Transforming the Task
Dale L. Goodhue
Stefano Grazioli
Barbara D. Klein
Technology
Characteristics
?
Individual
Performance
This the Question We Are Interested In.
Outline
• A comparison of the previous way of
conceptualizing TTF with a new way -- Different
technologies present the task doer with different
options for task completion processes – some of
which are more “attractive” than others.
• An experiment applying these ideas to the task of
accessing information from integrated or nonintegrated databases
Expanding the
Task/Technology Fit Perspective
• Original insight from TTF: technology
improves performance when the technology
“fits” the task
• Use alone is not enough!
• What is “fit”, and how does it improve
performance?
• How does a technology improve
performance at a task?
Task
Characteristics
TaskTechnology
Fit
Technology
Characteristics
Individual
Characteristics
Individual
Perceptions/
Beliefs
Individual
Performance
Use
The Technology-to-Performance Chain
(Goodhue and Thompson, 1995)
A Different Perspective:
Two Different Tasks?
• Organizational researchers don’t distinguish
between technology and task; see task as presented
to the task doer (after the application of
technology)
• TTF researchers see task as existing before the
application of technology
• There are two tasks! The underlying task and the
task as presented to the task doer.
• Technology changes the task as presented to the
task doer
Task as Problem
Task as Solution
Task
As Underlying
Problem or
Motivation
Jarvenpaa (89)
Choose a restaurant using different choice
rules.
Vessey & Galleta (91)
Determine point values vs. relationships
Goodhue (95)
T
e
c
h
n
o
l
o
g
y
Meet different mgmt info requirements
McGrath (1984)
Task
As Sequence
of Actions
Used To
Meet the
Task Need
Perrow, Fry and Slocum
Actions used to transform inputs into
outputs
Wood
Required acts and info cues, etc.
Task circumplex
There are two tasks!
Task as Problem
Task
As Underlying
Problem or
Motivation
Changing the technology,
Changes
the strategy options
for task completion
(the possible action
sequences)
T
e
c
h
n
o
l
o
g
y
1
T
e
c
h
n
o
l
o
g
y
2
Task as Solution
Sequence A
for Actions To
Meet the
Task Need
Sequence B
for Actions To
Meet the
Task Need
Sequence C
for Actions To
Meet the
Task Need
Sequence D
for Actions To
Meet the
Task Need
Sequence E
for Actions To
Meet the
Task Need
Different
strategy options
have different
“attractiveness”
A technology
that makes
possible an
“attractive”
strategy option
has high TTF
A Simple Example: Which
Technology Will be Chosen, Which
Gives Better Performance and Why?
• Task -- Decide if either of two divisions is making
excessive use of high cost shipping alternatives.
• Three Different Technologies
– Paper based systems with all original documents
– Division specific accounting database systems
– Integrated accounting database system
How conceptualize and measure TTF of 3 Systems?
The Old Way:
• Decide what the task requirements are. For information
access, they might be:
–
–
–
–
–
–
–
–
–
–
–
–
right data,
right level of detail,
easy to locate,
understandable meaning,
accessiblity,
reliable systems,
training,
assistance,
accuracy,
currency,
compatibility,
Etc.
TTF the Old Way
• Now, rate the three technologies on meeting task needs for
these dimensions. The best technology has highest TTF
Div Spec Integr.
Paper
DB
DB
–
–
–
–
–
–
–
–
–
–
–
–
right data,
right level of detail,
easy to locate,
understandable meaning,
accessiblity,
reliable systems,
training,
assistance,
accuracy,
currency,
compatibility,
Etc.
high
high
low
high
low
high
high
high
high
high
low
high
low
high
high
high
high
med
med
high
high
low
med
low
high
high
high
high
low
med
high
high
high
How conceptualize and measure TTF of 2 Systems?
A New Way:
• Examine the task process (the actions needed)
when using each of the three systems to carry out
the task.
• Characterize the “attractiveness” of the three ways
of accomplishing the task.
• The “attractiveness” is the TTF of each
technology for that task.
Technologies Change the Processing Options
Presented to Task Doer
Underlying Task
Decide if
either of two
divisions is
making
excessive use
of high cost
shipping
alternatives.
(Recover
aggregate
info for each
division from
records of
shipping
transactions)
3 Different Technologies
Task Presented to Task Doer
(or Technology/Processing Options)
Situation 1: Paper
Documents for Each
Transaction
For both divisions, manually select
all shipping transactions, translate
to problem categories, and
consolidate.
Situation 2: Separate
Accounting DB
Systems for Each
Division
Above, or: For each division
separately: translate acctg DB
system categories to problem
categories, use queries to gather
totals for relevant acctg categories,
consolidate.
Situation 3: Integrated
Accounting DB
Systems Across Both
Divisions
Above, or: For both divisions
combined: translate acctg DB
system categories to problem
categories, use queries to gather
totals for relevant categories,
consolidate.
The Big Problem
• We need to have a way of characterizing the
“attractiveness” of these options.
How Characterize the “Attractiveness” of
Different Processing Options?
• Narrow the focus to “intellective tasks” (McGrath 1984):
solving a problem that has a correct answer (not psycho-motor,
creativity, planning, etc.)
• What is it about a processing option that is changed by
technology and task, and affects performance?
– Task complexity (Wood 1986):
• Component complexity: How many distinct actions, information cues?
• Coordinative complexity: How many precedence relationships?
• Dynamic complexity: How fast is the underlying reality changing
– Difficulty (Campbell 1988): reliance upon skills, abilities,
experience of individual task doer
– Task complexity is independent of the task doer, difficulty is
dependent on task doer.
How Characterize the “Attractiveness” of
Different Processing Options?
• Question: Can we really capture the essential
differences between strategy options using task
complexity and difficulty?
How Characterize the “Attractiveness” of
Different Processing Options?
• Question: Can we really capture the essential
differences between strategy options using task
complexity and difficulty?
• Answer: Perhaps, if the strategy options are not
too different.
Why Go to All this Trouble?
• Humans choose technologies on the basis of
the most attractive task processing option,
not the best technology characteristics
• The more they know about how the
technology works, the more this is true
• Individual performance as well is a function
of the technology/task process option
chosen and its attractiveness
Task as problem
Task
Characteristics
Technology
Characteristics
Task as Solution
A Set of
Processing
Options, Each
With It’s
Attractiveness
(Task
Complexity,
Difficulty?)
Individual
Characteristics
Individual
Perceptions/
Beliefs
Individual
Performance
Choice of
One
Processing
Option
(and the
associated
Technology)
The Task Transformation Model
Part 2
Applying These Ideas to
Integrated vs. Non Integrated
Databases
Data Integration
• Definition: standardization of data
definitions and structures across a collection
of data sources
• Assumption: when questions require data
from multiple sources, DI should reduce
manual and intellectual retrieval effort
• Important part of value of ERP and DW is
provision of integrated data
• No scientific assessment of how, or if
assumption is true!
Examples of Non-Integrated Data
Non-Integrated Environment
1. Codes for Part
Numbers:
Codes for 3/4"
BOLT
2. Codes for
Customer_ID:
Codes for ABC, Inc.
3. Codes and
Definitions for
Accounts Showing
Sales and Sales
Expenses
Division A
Division B
115899
337189
42765
42675,
49345,
47293
Comments
Potentially
different codes
for same part
Potentially
different
structure of
codes
301 GROSS SALES (net of 301 GROSS SALES (net of sales
Potentially
returns and allowances)
discounts)
different
302 SALES DISCOUNTS
401 RETURNS AND ALLOWANCES
definition
726 ADVERTISING
713 SML ACNT SALES EXPENSES:
schemes
727 PROMOS, MAILINGS
advert./promo.
723 LRG ACNT SALES EXPENSES:
advert./promo.
How Does Data Integration
Change the Task Complexity of
the Information Retrieval Task?
• To understand this we need a model of the
information retrieval task.
• Then focus on the impact of DI on
component complexity and coordinative
complexity of different sub-processes in
that overall task
Example Task
• Management is concerned about ratio of
advertising and promotions expenses to
sales revenue in two divisions
• task doer is asked to find (for each division)
– year-to-date advertising and promotion
expenses
– year-to-date sales (net of discounts, returns and
allowances)
Division A (65 Account Codes)
.
.
301 GROSS SALES (net of
returns and allowances)
302 SALES DISCOUNTS
.
.
726 ADVERTISING
727 PROMOS, MAILINGS
.
.
Division B (65 Account Codes)
.
.
301 GROSS SALES (net of sales
discounts)
.
.
401 RETURNS AND ALLOWANCES
.
.
713 SML ACNT SALES EXPENSES:
advert./promo.
723 LRG ACNT SALES EXPENSES:
advert./promo.
Account Codes in
A Non Integrated Data Environment
Data Environment
Problem Statement
1. Split Off One Subtask
2. Semantic Specification
3. Syntactic Specification
Yes,
Repeat
Steps 1-6
5. Error Repair
Error Detected
No
4. Error Detection Errors
Detected
Query
Query Processor
6. Any
More
Subtasks?
No
7. Consolidate
Subtask Results
Displayed Results
Overall Result: -accuracy
-time required
Process Model of Information Retrieval
Hypotheses For Non Integrated Data
• Subprocess 1: greater component complexity (more data
items to consider) will encourage task doers to sub-divide
the task into more subtasks.
• Subprocess 2: greater component complexity (more data
items to consider) makes it more likely task doer will
misclassify at least one data item, in total task.
• Subprocess 3: less component complexity (fewer elements
in a less complex query) makes it less likely task doer will
make syntax or logic errors in any given query.
• Subprocess 3, more precedence requirements (keeping
straight which database) make it more likely task doer will
confuse or mis-specify the database.
Impact of Number of Sub-tasks and
Error Profiles on Performance
• Time to complete will increase with the number of subtasks
• Time to complete will increase with the number of high
feedback errors (syntax and logic) in the total set of
queries used
• Likelihood of totally correct answers will decrease with
the existence of one or more low feedback errors
(neglecting a needed account code or included a nonneeded account code)
Number
and Type
of Errors
Underlying
Task
Integrated vs.
non
Integrated
DB
Different
Mix of
Processing
Options
Choice of
Option
With
Given Task
Complexity
Size of
Sub-Queries
Impact of Data Integration on Performance
Performance
- accuracy
- time
Method
• 107 student pairs
• Given: managerial questions, SQL query
processor, and either integrated or non
integrated database
• 4 sessions: 2 training, 2 with treatments
• Captured time to complete, accuracy, and
every query submitted (1164 queries)
• LISP program determined subtasks used and
error profiles of each query
• (Kappa coefficient of agreement between LISP
and human coders: .93 or excellent)
Type of
Error
Description
High Feedback
Errors
Syntax
Violation of syntax
rules, misspellings,
etc.
Logic
Misusing the logic
of the WHERE
clause
Database Looking for Div A
acct code in Div B
database
Low Feedback
Errors
Select
Specifying
Low
plausible but
Feedback wrong field
Under
Leaving out a
Spec.
needed category
Over
Adding an
Spec.
incorrect category
Examples
Frequency
>= 1 error in
214 Sessions
WEHRE acctcode = ‘301’ or
acctcode = ‘302’) (Should be
WHERE)
Where acctcode < ‘300’ and acctcode
> ‘303’ (No account codes in range)
63%
Where acctcode = ‘401’ (when
seeking Div A info – acctcode 401
exists only on the Div B database)
9%
Select acctcode, Month_to_Date
(instead of “Select Year_to_Date”)
43%
34%
Where acctcode = ‘301’
42%
(when 302 or 401 is also needed)
Where acctcode = ‘301’ or acctcode = 38%
‘302’ or acctcode = ‘920’
(when 920 should not be included)
Interesting Aspects of Analysis
• Analyzed only the first query attempt at any
subtask. (Remainder are error correction
queries and much harder to predict.)
• Distribution of errors was highly skewed.
Many made no errors. Inappropriate as
dependent variable in regression. Used
Logistic Regression for those analyses
• Used pair characteristics as additional
explanatory variables: tendency to speed,
tendency to accuracy
Impact of Integrated Data
At the Individual
Query Level
At the Total Task
Level
On Ultimate
Performance
Fewer Subtasks
Fewer Subtasks Leads to
Shorter Time to Complete
Mixed Impact on
Erroneously Including
or Excluding Account
Codes
(Low Feedback Errors)
Fewer Erroneously
Included or Excluded
Account Codes
(Low Feedback Errors)
Fewer Low Feedback
Errors Leads to
Greater Accuracy
Many More Logic Errors
But Only a Hint of
More Syntax Errors
(High Feedback Errors)
More Logic Errors
But No More Syntax
Errors
(High Feedback Errors)
More Logic Errors
Has No Impact On
Time To Complete
(More Syntax Errors
Does Impact Time)
Research Implications
• We can understand task doers’ choice of technologies and
the impact on individual performance by considering the
“attractiveness” of the processing options provided by
different technologies
• We should focus on the process of carrying out the task,
not on the characteristics of the technology
• We can use “task complexity” to understand the better TTF
of DI for multi-division tasks
• When we do, we see that DI does not improve
performance at the query level, but allows task doers to
“take larger bites” with each query, and use fewer queries,
hence making fewer hard to catch errors and taking less
time overall.
• In this way DI reduces time and increases accuracy for
data retrieval