Transcript Evaluating the competence of inexperienced IS auditors

UWCISA Symposium 2007
Auditing system development:
Constructing the meaning of “systematic
and rational” in the context of legacy
code migration for vendor incentives
A. Faye Borthick
Georgia State University, [email protected]
Paul L. Bowen
Florida State University, [email protected]
J. Mack Robinson College of Business / Georgia State University
Simulation in response to learning needs
1. Growing emphasis on controlling system
development to comply with SOX 404
2. Scarcity of cases with data for practicing system
development audits
J. Mack Robinson College of Business / Georgia State University
The challenge
Design and implement learning experiences for
novices to develop expertise in auditing system
development
The approach
Create authentic tasks (tasks that IS auditors would
perform on the job) to simulate learning on-the-job
J. Mack Robinson College of Business / Georgia State University
The task: Organofood system development audit
• Organofood: Grocery chain
• Objective: Audit migration of vender
incentive code to an enterprise system
(ES) with database querying
• Approach: Audit vendor incentive accounts
through analysis of the provided data in
the context of the situation
J. Mack Robinson College of Business / Georgia State University
Organofood: An authentic task in a simulation
1. Situation narrative: Context in conversation
2. Transaction data: For analysis with database or
audit software querying
3. Links to background information
4. SDLC approval for migration project
5. Reporting template
6. Readiness questions for Saving Sergeant Pabletti
epiphany
J. Mack Robinson College of Business / Georgia State University
Situation narrative: Context in conversation
The scene: Auditors planning a system
development in two parts:
1. Audit of migration of legacy code for vendor
incentives to an enterprise system (ES) for a
grocer
2. Audit of system development generally
1. Compliance
2. Productivity
J. Mack Robinson College of Business / Georgia State University
Transaction data: Testdata database tables
1. Accounts: Accounts in general ledger
2. GeneralLedger: GL transactions
3. Incentive: Terms of vendor incentives
4. IncentiveCode: Incentive codes by type
5. Invoice: Vendor invoice
6. Purchase: Details of purchases
7. SKU: Information about stock keeping units
(SKUs)
8. StoreSales: Sales by period, 4 weeks by week
J. Mack Robinson College of Business / Georgia State University
Transaction data: ProgLibrary database tables
1. LibraryTransaction: Program library
transactions
2. StageCode: Definition of stageCodes
J. Mack Robinson College of Business / Georgia State University
Four-column reporting template
1. Audit objective
2. Audit procedure
3. Results from executing queries
4. Reporting
• Findings from querying
• Recommendations, if any
• Data limitations, if any
J. Mack Robinson College of Business / Georgia State University
Organofood audit report
Audit
procedure
Results from
execution of
queries
Statement of Explanation of
the audit
the audit
objective
procedure for
implementing
the audit
objective in
terms of the
data attributes
1. Names of
queries
2. Statement
of query
results
3. Explanation of
results
Audit
objective
Exceptions
1. Findings
from
querying
2. Recommen
dations, if
any
3. Data
limitations, if
any
J. Mack Robinson College of Business / Georgia State University
Prerequisite skills and knowledge
1. Querying proficiency
2. Familiarity with system development
3. Some experience developing audit objectives
and procedures
J. Mack Robinson College of Business / Georgia State University
Analysis tool choices
• Microsoft Access QBE
• Audit software, e.g., ACL, IDEA
• SQL
J. Mack Robinson College of Business / Georgia State University
Readiness questions: Sergeant Pabletti epiphany
Saving Sergeant Pabletti: Video game played by
80,000 Army recruits each year
1. Recruits inept in first play of game
2. With learning, recruits save the sergeant
3. Vicarious experience for the twitch generation
on the value of learning and thinking
4. Objective: Afford learners in IS audit a Saving
Sergeant Pabletti epiphany about the value of
thinking deeply about the situation
J. Mack Robinson College of Business / Georgia State University
Organofood readiness question
The data in OrganofoodProgLibrary.mdb permit
verifying:
1. Completeness of requirements
2. Effectiveness of user participation
3. Completeness of quality assurance
4. The adequacy of the SDLC method
5. Separation of duties in development
J. Mack Robinson College of Business / Georgia State University
Learning objectives
1. Understand up the business situation
2. Perform audit steps
1. Develop audit objectives*
2. Design audit procedures
3. Execute audit procedures with querying
4. Communicate objectives, procedures, and
results in a report
* Could be provided for AIS classes
J. Mack Robinson College of Business / Georgia State University
References for situation model building
• Barsalou. 1999. Language
comprehension: Archival memory or
preparation for situated action? Discourse
Processes 28 (1): 61-80
• Zwaan and Radvansky. 1998. Situation
models in language comprehension and
memory. Psychological Bulletin 123 (2):
162-185
J. Mack Robinson College of Business / Georgia State University
The task: Organofood audit
Dysfunctional conditions to find:
1. Sales data not extrapolating to achieving
volume discounts
2. Volume discounts booked at wrong percent
(too high)
3. Developers performing incompatible duties
for some components
4. eXtreme Programming (XP) more productive
than a more traditional SDLC approach
J. Mack Robinson College of Business / Georgia State University
Why are these conditions hard to find?
1. Developing audit objectives requires
integrating the concepts of system
development objectives:
• Satisfying user requirements
• Being on time and on budget
2. The querying is complicated, requiring
sequences of queries, cf. Hendrawirawan
et al. 2007
J. Mack Robinson College of Business / Georgia State University
Learner reaction to Organofood
1. Liked readiness questions for calibrating
understanding of the situation
2. Frustrated initially but eventually pleased with
developing concept of “systematic and rational”
3. Appreciated opportunity to test their sensemaking ability
4. Interested in how to develop better audit
objectives in future audits
J. Mack Robinson College of Business / Georgia State University
What is Organofood’s contribution?
Give students practice in:
• Making sense of an authentic audit situation
from conversation
• Developing system development audit
objectives
• Querying and analyzing transaction data in
an authentic audit situation
• Interpreting query results
J. Mack Robinson College of Business / Georgia State University
Access
Web staging of the simulation including a link to
the database:
http://www2.gsu.edu/~wwwsys/pro/project/Orga
nofood/site/Organofood.htm
Use name = ac863 and password = Qd0319
J. Mack Robinson College of Business / Georgia State University