Transcript Managing the business risk of fraud

Managing the Business
Risk of Fraud using
Sampling and Data Mining
Mike Blakley
Presented to:
Fall 2009
Managing the business risk of fraud using sampling and
data mining
EZ-R Stats, LLC
PWC Global Survey – Nov, 2009
“Economic crime in a downturn”



Sharp rise in accounting fraud
over the past 12 months
Accounting fraud had grown to 38
percent of the economic crimes in
2009
Employees face increased
pressures to :
–
–
–
meet performance targets
keep their jobs
keep access to funding
Managing the business risk of fraud
EZ-R Stats, LLC
Survey findings




Greater risk of fraud due to increased
incentives or pressures
More opportunities to commit fraud, partially
due to reductions in internal finance staff
While companies are expecting more fraud,
they have not done much
People who look for fraud are more likely to
find it
Managing the business risk of fraud
EZ-R Stats, LLC
Session objectives






Understand the framework for managing the business
risk of fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential
sampling and other sampling techniques
Apply SAS 56, the new SAS suite and the revised
(2007) Yellow Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the
audit process, without losing effectiveness.
Managing the business risk of fraud
EZ-R Stats, LLC
Session agenda - 1






Introduction and the Process for Managing the Business Risk of
Fraud
Introductions All Around
Course Objectives
Framework of risk management for fraud
Fundamentals of data mining
Data mining: The Engine That Drives analysis
–
Analytics and Regression






Sources of Analytics Data
Basic and Intermediate ARTs
SAS 56
IIA Practice Advisory 2320
The Yellow Book (2007 revision)
The Guide – “Managing the Business Risk of Fraud”
Managing the business risk of fraud
EZ-R Stats, LLC
Session Agenda (cont’d) –
Sampling refresher




Sampling
The sampling process
Sampling methods
RAT-STATS
–
–
–
–
–
–
–
–






Random Numbers
Determining Sample Size
Case Study
Attribute sampling
Variable Sampling
Case study
Stratified Sampling
Obtaining and Interpreting the results
Other Sampling Approaches
DCAA Audit Package
Sequential Sampling
Overview of the process
Attribute Sampling
Variable Sampling
Managing the business risk of fraud
EZ-R Stats, LLC
Session Agenda (cont’d) –
Linear regression as an
audit tool






Regression Analysis
Overview
Terms
Statistical basis
Charting Regression … Seeing Is Believing
Plotting Data
–

Statistical Intervals
–
–
–

Inserting a “Trend line”
Confidence Intervals
Prediction Intervals
Calculation of Statistical “Confidence Bounds”
Case Study - Wake County Schools Bus Maintenance
Managing the business risk of fraud
EZ-R Stats, LLC
Session Agenda (cont’d) –
Data mining, or
How to test 100%




Overview
Statistical Basis
Data Conversion and Extraction
Data mining objectives
–
–
–
–
Classification
Trends
Identification of extremes
Major types of data analysis



Numeric
Date
Text
Managing the business risk of fraud
EZ-R Stats, LLC
Session Agenda (cont’d) –
Excel as an Analytics tool



Macros
Tools – Data Analysis
The Macro facility
–
–
Adding a little “class” to your audit
VBA – “friend” or “foe”
Managing the business risk of fraud
EZ-R Stats, LLC
Handout (CD)



CD with articles and software
PowerPoint presentation
More info at www.ezrstats.com
Managing the business risk of fraud
EZ-R Stats, LLC
“Cockroach” theory of auditing

If you spot one roach….
Managing the business risk of fraud
EZ-R Stats, LLC
“Cockroach” theory of auditing

There are probably 30
more that you don’t
see…
Managing the business risk of fraud
EZ-R Stats, LLC
Statistics based “roach” hunting
Many frauds coulda/woulda/shoulda been detected with analytics
Managing the business risk of fraud
EZ-R Stats, LLC
Overview
Fraud patterns detectable with
digital analysis
 Basis for digital analysis
approach
 Usage examples
 Continuous monitoring
 Business analytics

Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
The Why and How






Three brief examples
ACFE/IIA/AICPA Guidance Paper
Practice Advisory 2320-1
Auditors “Top 10”
Process Overview
Who, What, Why, When & Where
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1a
Example 1
Wake County Transportation Fraud




Supplier Kickback – School Bus
parts
$5 million
Jail sentences
Period of years
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1a
Too little too late




Understaffed internal audit
Software not used
Data on multiple platforms
Transaction volumes large
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1a
Preventable



Need structured, objective
approach
Let the data “talk to you”
Need efficient and effective
approach
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Regression Analysis

Stepwise to find
relationships
–
–

Forwards
Backwards
Intervals
–
–
Confidence
Prediction
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Data outliers


Sometimes an “out
and out Liar”
But how do you
detect it?
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Data Outliers


Plot transportation costs vs.
number of buses
“Drill down” on costs
–
–
–
Preventive maintenance
Fuel
Inspection
Managing the business risk of fraud
EZ-R Stats, LLC
Scatter plot with prediction and
confidence intervals
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1a
Example 2
Cost of six types of AIDS drugs
Total Cost of AIDS Drugs
Dollar Amount
200
150
NDC1
NDC2
100
NDC3
50
NDC4
0
NDC5
NDC1
NDC2
NDC3
NDC4
NDC5
NDC6
NDC6
Drug Type
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Medicare HIV Infusion Costs



CMS Report for 2005
South Florida - $2.2 Billion
Rest of the country combined $.1 Billion
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Pareto Chart
Medicare HIV Infusion Costs - 2005 ($Billions)
data source: HHS CMS
120.0%
Annual Medicare Costs
100.0%
80.0%
Pct
60.0%
Cum Pct
40.0%
20.0%
15
13
11
9
7
5
3
1
0.0%
County
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1a
Example 2
Typical Prescription Patterns
AIDS Drugs Prescription Patterns
Dollar Value
60.0
NDC1
50.0
NDC2
40.0
NDC3
30.0
NDC4
20.0
NDC5
10.0
NDC6
0.0
Prov 1
Prov 2
Prov 3
Prov 4
Prov 5
Prov 6
Prescriber
Managing the business risk of fraud
EZ-R Stats, LLC
Example 2
Objective 1a
Prescriptions by Dr. X
Dollar Amount
Dr. X compared with Total Population
350
300
250
200
150
100
50
0
Population
Dr. X
NDC1
NDC2
NDC3
NDC4
NDC5
NDC6
Drug Type
Managing the business risk of fraud
EZ-R Stats, LLC
Example 2
Objective 1a
Off-label use

Serostim
–
–
–
Treat wasting syndrome, side effect of
AIDS, OR
Used by body builders for recreational
purposes
One physician prescribed $11.5 million
worth (12% of the entire state)
Managing the business risk of fraud
EZ-R Stats, LLC
Example 3
Objective 1a
Revenue trends
Overall Revenue Trend
Annual Billings
1.2
1.15
1.1
Overall
1.05
Linear (Overall)
1
0.95
0.9
2001
2002
2003
Calendar Year
Managing the business risk of fraud
EZ-R Stats, LLC
Example 3
Objective 1a
Dental Billings
Rapid Increase in Revenues
Annual Billings
($millions)
5
4
Billings A
3
Billings B
2
Linear (Billings A)
1
0
2001
2002
2003
Calendar Year
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Guidance Paper





A proposed implementation approach
“Managing the Business Risk of Fraud: A
Practical Guide” http://tinyurl.com/3ldfza
Five Principles
Fraud Detection
Coordinated Investigation Approach
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Managing the Business Risk of
Fraud: A Practical Guide


ACFE, IIA and AICPA
Exposure draft issued
11/2007, final 5/2008
Section 4 – Fraud
Detection
Managing the business risk of fraud
EZ-R Stats, LLC
Guidance Paper

Five Sections
–
–
–
–
–
Fraud Risk Governance
Fraud Risk Assessment
Fraud Prevention
Fraud Detection
Fraud Investigation and
corrective action
Managing the business risk of fraud
EZ-R Stats, LLC
Risk Governance


Fraud risk management program
Written policy – management’s expectations
regarding managing fraud risk
Managing the business risk of fraud
EZ-R Stats, LLC
Risk Assessment


Periodic review and assessment of potential
schemes and events
Need to mitigate risk
Managing the business risk of fraud
EZ-R Stats, LLC
Fraud Prevention


Establish prevention techniques
Mitigate possible impact on the organization
Managing the business risk of fraud
EZ-R Stats, LLC
Fraud Detection



Establish detection techniques for fraud
“Back stop” where preventive measures fail,
or
Unmitigated risks are realized
Managing the business risk of fraud
EZ-R Stats, LLC
Fraud Investigation and Corrective
Action



Reporting process to solicit input on fraud
Coordinated approach to investigation
Use of corrective action
Managing the business risk of fraud
EZ-R Stats, LLC
“60 Minutes” – “World of Trouble”

2/15/09 – Scott Pelley
–
–
–
–
–
Fraud Risk Governance – “one grand wink-wink,
nod-nod “
Fraud Risk Assessment - categorically false
Fraud Prevention – “my husband passed away”
Fraud Detection - We didn't know? Never saw one.
Fraud Investigation and corrective action - Pick-APayment losses $36 billion
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Section 4 – Fraud Detection





Detective Controls
Process Controls
Anonymous Reporting
Internal Auditing
Proactive Fraud Detection
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Proactive Fraud Detection

Data Analysis to identify:
– Anomalies
– Trends
– Risk indicators
Managing the business risk of fraud
EZ-R Stats, LLC
Fraud Detective Controls



Operate in the background
Not evident in everyday business
environment
These techniques usually –
–
–
–
–
Occur in ordinary course of business
Corroboration using external information
Automatically communicate deficiencies
Use results to enhance other controls
Managing the business risk of fraud
EZ-R Stats, LLC
Examples of detective controls



Whistleblower hot-lines (DHHS and OSA
have them)
Process controls (Medicaid audits and edits)
Proactive fraud detection procedures
–
–
–
Data analysis
Continuous monitoring
Benford’s Law
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Specific Examples Cited




Journal entries – suspicious
transactions
Identification of relationships
Benford’s Law
Continuous monitoring
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1b
Data Analysis enhances ability to
detect fraud





Identify hidden relationships
Identify suspicious transactions
Assess effectiveness of internal
controls
Monitor fraud threats
Analyze millions of transactions
Managing the business risk of fraud
EZ-R Stats, LLC
Continuous Monitoring of Fraud
Detection



Organization should develop ongoing
monitoring and measurements
Establish measurement criteria (and
communicate to Board)
Measurable criteria include:
Managing the business risk of fraud
EZ-R Stats, LLC
Measurable Criteria – number of






fraud allegations
fraud investigations resolved
Employees attending annual ethics course
Whistle blower allegations
Messages supporting ethical behavior
delivered by executives
Vendors signing ethical behavior standards
Managing the business risk of fraud
EZ-R Stats, LLC
Management ownership of each
technique implemented

Each process owner should:
–
–
–
–
Evaluate effectiveness of technique regularly
Adjust technique as required
Document adjustments
Report modifications needed for techniques which
become less effective
Managing the business risk of fraud
EZ-R Stats, LLC
Practice Advisory 2320-1
Analysis and Evaluation


International standards for the professional
practice of Internal Auditing
Analytical audit procedures
–
–
Efficient and effective
Useful in detecting



Differences that are not expected
Potential errors
Potential irregularities
Managing the business risk of fraud
EZ-R Stats, LLC
Analytical Audit Procedures

May include
– Study of relationships
– Comparison of amounts with
similar information in the
organization
– Comparison of amounts with
similar information in the
industry
Managing the business risk of fraud
EZ-R Stats, LLC
Analytical audit procedures




Performed using monetary amounts, physical
quantities, ratios or percentages
Ratio, trend and regression analysis
Period to period comparisons
Auditors should use analytical audit
procedures in planning the engagement
Managing the business risk of fraud
EZ-R Stats, LLC
Factors to consider





Significance of the area being audited
Assessment of risk
Adequacy of system of internal control
Availability and reliability of information
Extent to which procedures provide support
for engagement results
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1c
Peeling the Onion
Fraud Items
Possible Error Conditions
Population as Whole
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1d
Fraud Pattern Detection
Round Numbers
Market Basket
Benford’s Law
Stratification
Gaps
Target Group
Trend Line
Univariate
Duplicates
Holiday
Day of Week
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
Digital Analysis (5W)
A little about the basics of digital analysis….
Who
 What
 Why
 Where
 When

Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
Who Uses Digital Analysis




Traditionally, IT specialists
With appropriate tools, audit
generalists (CAATs)
Growing trend of business
analytics
Essential component of
continuous monitoring
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
What - Digital Analysis

Using software to:
–
–
–

Classify
Quantify
Compare
Both numeric and non-numeric
data
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
How - Assessing fraud risk




Basis is quantification
Software can do the “leg work”
Statistical measures of difference
– Chi square
– Kolmogorov-Smirnov
– D-statistic
Specific approaches
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
Why - Advantages








Automated process
Handle large data populations
Objective, quantifiable metrics
Can be part of continuous monitoring
Can produce useful business analytics
100% testing is possible
Quantify risk
Repeatable process
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
Why - Disadvantages



Costly (time and software costs)
Learning curve
Requires specialized knowledge
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
When to Use Digital Analysis




Traditional – intermittent (one off)
Trend is to use it as often as possible
Continuous monitoring
Scheduled processing
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1e
Where Is It Applicable?

Any organization with data in digital
format, and especially if:
–
–
–
Volumes are large
Data structures are complex
Potential for fraud exists
Managing the business risk of fraud
EZ-R Stats, LLC
Disadvantages of digital analysis

Cost
–
–
–

Software
Training
Skills not widely available
Time consuming
–
–
Development costs
Testing resources
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1
Objective 1 Summarized





Three brief examples
CFE Guidance Paper
“Top 10” Metrics
Process Overview
Who, What, Why, When & Where
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 1 - Summarized






Understand the framework for managing the business
risk of fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential
sampling and other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007)
Yellow Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the
audit process, without losing effectiveness.
Next is plan, perform …
Managing the business risk of fraud
EZ-R Stats, LLC
Statistical Sampling




Brief History / Timeline
Overview
Attribute Sampling – Compliance
Variable Sampling – Numeric Estimates
Managing the business risk of fraud
EZ-R Stats, LLC
History of Sampling



Basis is two laws/theorems of probability
Law of Large Numbers
Central Limit Theorem
Managing the business risk of fraud
EZ-R Stats, LLC
Law of large numbers
Simulated rolling of dice
7
6
Value
5
4
Result
Average
3
Linear (Result)
2
1
0
1
7 13 19 25 31 37 43 49 55 61 67 73 79 85
Observation
Managing the business risk of fraud
EZ-R Stats, LLC
Time Line - LLN



Indian mathematician Bramagupta 600 AD
Italian mathematician Cardon 1500’s
Statement without proof that empirical
statistics improve with more trials
Managing the business risk of fraud
EZ-R Stats, LLC
Time line LLN (continued)


Jacob Bernoulli first to prove in 1713
Foundation for central limit theorem
Managing the business risk of fraud
EZ-R Stats, LLC
Central limit theorem
Classic measure
Mean of a sufficiently large
number of random samples
will be approximately
normally distributed.
Managing the business risk of fraud
EZ-R Stats, LLC
The traditional explanation
Managing the business risk of fraud
EZ-R Stats, LLC
Central Limit Theorem




See it in action today
Any population
Large number of samples
Average is “normally” distributed
Managing the business risk of fraud
EZ-R Stats, LLC
History of Central Limit Theorem





French mathematician
Abraham de Moivre
1733 – approximate
distribution from tossing
coin (heads/tails)
Ho hum reaction
French Mathematician
LaPlace – expanded it
Ho hum reaction
Managing the business risk of fraud
EZ-R Stats, LLC
History of CLT (cont’d)



Russian mathematician
Lyapunov
Proof in 1901
Same reaction
Managing the business risk of fraud
EZ-R Stats, LLC
Industrial revolution
Manufacturing
Engineering
Excitement!
Managing the business risk of fraud
EZ-R Stats, LLC
Student’s T
William Gosset - 1908
Guinness Brewery
Managing the business risk of fraud
EZ-R Stats, LLC
SAS 39


Effective June, 1983
Exposure draft for
revision in 2009
Managing the business risk of fraud
EZ-R Stats, LLC
Attribute sampling



Buonaccorsi (1987)
Refined calculations
Few software packages use it
Managing the business risk of fraud
EZ-R Stats, LLC
Overview




Sample size calculations
Attribute sampling
Variable sampling
Random number generators
Managing the business risk of fraud
EZ-R Stats, LLC
Sample size calculation




It’s a guess…
Every package – different
answer
Need to know the
population
But that’s why you’re
taking a sample!
Managing the business risk of fraud
EZ-R Stats, LLC
Attribute Sampling
Using RAT-STATS
Unrestricted populations
Managing the business risk of fraud using sampling and
data mining
EZ-R Stats, LLC
Session Objectives
1.
2.
3.
4.
Understand what is attribute sampling and
when to use it
Understand unrestricted populations
Overview of the process using RAT-STATS
Understand the formula behind the
computations
Managing the business risk of fraud
EZ-R Stats, LLC
Attribute sampling
 “Attribute”
 Compliance
testing
 Signatures on approval
documents, attachment of
supporting documentation, etc.
Managing the business risk of fraud
EZ-R Stats, LLC
Statistical approach
 Recommended
 Economical
 Efficient
 Requires
determination of a
sample size
Managing the business risk of fraud
EZ-R Stats, LLC
Overview of process

Determine the sampling objective
–
–




Confidence
Precision
Determine required sample size
Identify samples to be selected based upon random
numbers
Pull the sample and test
Compute the sampling results (i.e. estimate of
range)
Managing the business risk of fraud
EZ-R Stats, LLC
How this is done in RAT-STATS





The sampling parameters are first developed
by the auditor
RAT-STATS is used to compute sample size
RAT-STATS used to generate random
numbers
Pull the sample and test
Enter results in RAT-STATS to compute
estimates
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
4.
Size of population
Expected error rate
Required confidence
Required precision
Managing the business risk of fraud
EZ-R Stats, LLC
Step 2 – Obtain the random numbers


Done by entering info into RAT-STATS
Output can be a variety of sources:
–
–
–
–
Text File
Excel
Microsoft Access
Print File
Managing the business risk of fraud
EZ-R Stats, LLC
Step 3 – Pull the sample


Each random number selected corresponds
with an item
Put the selected item on a separate schedule
Managing the business risk of fraud
EZ-R Stats, LLC
Step 4 - Test each selected item

Generally requires reviewing documents
Managing the business risk of fraud
EZ-R Stats, LLC
Step 5 – Compute the results


Enter summary information into RAT-STATS
Output can be in a variety of formats:
–
–
–
–
–
Excel
Microsoft Access
Text File
Print File
Printer
Managing the business risk of fraud
EZ-R Stats, LLC
That’s It!



Now we’ll see an actual demo using the RATSTATS software
Excel population of 5,000 invoices
Results of test of attributes stored in the
worksheet
Managing the business risk of fraud
EZ-R Stats, LLC
Variable Sampling
Using RAT-STATS
Unrestricted populations
Managing the business risk of fraud using sampling and
data mining
EZ-R Stats, LLC
Session Objectives
1.
2.
3.
4.
Understand what variable sampling is
and when to use it
Understand unrestricted populations
Overview of the process using RATSTATS
Understand the formula behind the
computations
Managing the business risk of fraud
EZ-R Stats, LLC
Variable sampling
 “Variable”
 Estimating
account balances
 Estimating transaction totals
Managing the business risk of fraud
EZ-R Stats, LLC
Statistical approach
 Recommended
 Economical
 Efficient
 Requires
determination of a
sample size
Managing the business risk of fraud
EZ-R Stats, LLC
Overview of process

Determine the sampling objective
–
–




Confidence
Precision
Determine required sample size
Identify samples to be selected based upon random
numbers
Pull the sample and test
Compute the sampling results (i.e. estimate of
range)
Managing the business risk of fraud
EZ-R Stats, LLC
How this is done in RAT-STATS





The sampling parameters are first developed
by the auditor
RAT-STATS is used to compute sample size
RAT-STATS used to generate random
numbers
Pull the sample and test
Enter results in RAT-STATS to compute
estimates
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
Probe sample
Statistical measure
Excel formula
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
Size of population
Average value
Standard deviation
Managing the business risk of fraud
EZ-R Stats, LLC
Step 2 – Obtain the random numbers
Done by entering info into RAT-STATS
 Output can be a variety of sources:
– Text File
– Excel
– Microsoft Access
– Print File

Managing the business risk of fraud
EZ-R Stats, LLC
Step 3 – Pull the sample
 Each
random number selected
corresponds with an item
 Put the selected item on a
separate schedule
Managing the business risk of fraud
EZ-R Stats, LLC
Step 4 - Test each selected item
 Generally
requires reviewing
documents
 Example data contains both
“examined” and “audited” value.
Managing the business risk of fraud
EZ-R Stats, LLC
Step 5 – Compute the results


Enter summary information into RAT-STATS
Output can be in a variety of formats:
–
–
–
–
–
Excel
Microsoft Access
Text File
Print File
Printer
Managing the business risk of fraud
EZ-R Stats, LLC
That’s It!
 Now
we’ll see an actual demo
using the RAT-STATS software
 Excel population of 5,000 invoices
 Audited values stored in the
worksheet
Managing the business risk of fraud
EZ-R Stats, LLC
Attribute Sampling
Using RAT-STATS
Stratified populations
Managing the business risk of fraud using sampling and
data mining
EZ-R Stats, LLC
Session Objectives
1.
2.
Understand what is stratification and when
to use it
Overview of the process using RAT-STATS
Managing the business risk of fraud
EZ-R Stats, LLC
Stratified sampling
 “Strata”
 Homogenous
 More
efficient in some instances
Managing the business risk of fraud
EZ-R Stats, LLC
Overview of process


Separation into strata
Determine the sampling objective
–
–




Confidence
Precision
Determine required sample size
Identify samples to be selected based upon random
numbers
Pull the sample and test
Compute the sampling results (i.e. estimate of
range)
Managing the business risk of fraud
EZ-R Stats, LLC
How this is done in RAT-STATS





The sampling parameters are first developed
by the auditor
RAT-STATS is used to compute sample size
RAT-STATS used to generate random
numbers
Pull the sample and test
Enter results in RAT-STATS to compute
estimates
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
4.
Size of population
Expected error rate
Required confidence
Required precision
Managing the business risk of fraud
EZ-R Stats, LLC
Step 2 – Obtain the random numbers


Done by entering info into RAT-STATS
Output can be a variety of sources:
–
–
–
–
Text File
Excel
Microsoft Access
Print File
Managing the business risk of fraud
EZ-R Stats, LLC
Step 3 – Pull the sample


Each random number selected corresponds
with an item
Put the selected item on a separate schedule
Managing the business risk of fraud
EZ-R Stats, LLC
Step 4 - Test each selected item

Generally requires reviewing documents
Managing the business risk of fraud
EZ-R Stats, LLC
Step 5 – Compute the results


Enter summary information into RAT-STATS
Output can be in a variety of formats:
–
–
–
–
–
Excel
Microsoft Access
Text File
Print File
Printer
Managing the business risk of fraud
EZ-R Stats, LLC
That’s It!



Now we’ll see an actual demo using the RATSTATS software
Excel population of 5,000 invoices
Results of test of attributes stored in the
worksheet
Managing the business risk of fraud
EZ-R Stats, LLC
Variable Sampling
Using RAT-STATS
Stratified populations
Managing the business risk of fraud using sampling and
data mining
EZ-R Stats, LLC
Session Objectives
1.
2.
3.
4.
Understand what stratified sampling is
and when to use it
Populations benefiting from stratified
sampling
Overview of the process using RATSTATS
Understand the formula behind the
computations
Managing the business risk of fraud
EZ-R Stats, LLC
Stratified variable sampling
 “Stratified”
 “Variable”
 Estimating
amounts
 Narrower standard deviation
Managing the business risk of fraud
EZ-R Stats, LLC
Overview of process

Determine the sampling objective
–
–




Confidence
Precision
Determine required sample size
Identify samples to be selected based upon random
numbers
Pull the sample and test
Compute the sampling results (i.e. estimate of
range)
Managing the business risk of fraud
EZ-R Stats, LLC
How this is done in RAT-STATS





The sampling parameters are first developed
by the auditor
RAT-STATS is used to compute sample size
RAT-STATS used to generate random
numbers
Pull the sample and test
Enter results in RAT-STATS to compute
estimates
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
Probe sample
Statistical measure
Excel formula
Managing the business risk of fraud
EZ-R Stats, LLC
Step 1 – Develop sampling parameters
1.
2.
3.
4.
Number of strata
Size of population
Average value
Standard deviation
Managing the business risk of fraud
EZ-R Stats, LLC
Step 2 – Obtain the random numbers



Done by entering info into RAT-STATS
Multi-stage random numbers
Output can be a variety of sources:
– Text File
– Excel
– Microsoft Access
– Print File
Managing the business risk of fraud
EZ-R Stats, LLC
Step 3 – Pull the sample
 Each
random number selected
corresponds with an item in a
strata
 Put the selected item on a
separate schedule
Managing the business risk of fraud
EZ-R Stats, LLC
Step 4 - Test each selected item
 Generally
requires reviewing
documents
 Example data contains both
“examined” and “audited” value.
Managing the business risk of fraud
EZ-R Stats, LLC
Step 5 – Compute the results


Enter summary information into RAT-STATS
Output can be in a variety of formats:
–
–
–
–
–
Excel
Microsoft Access
Text File
Print File
Printer
Managing the business risk of fraud
EZ-R Stats, LLC
That’s It!
 Now
we’ll see an actual demo
using the RAT-STATS software
 Excel population of 5,000 invoices
 Divided into three strata
 Audited values stored in the
worksheet
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2 - Summarized






Understand the framework for managing the business risk of
fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential sampling and
other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is cost reduction …
Managing the business risk of fraud
EZ-R Stats, LLC
Techniques for cost reduction


Optimize sample size
(most “bang” for the
buck)
Skip sampling – review
100% of transactions
using computer
assisted audit
techniques (CAATs)
Managing the business risk of fraud
EZ-R Stats, LLC
Sample optimization

Sequential sampling
Managing the business risk of fraud
EZ-R Stats, LLC
University of Hawaii

Banana aphids
Managing the business risk of fraud
EZ-R Stats, LLC
Sequential sampling

Banana aphids
Managing the business risk of fraud
EZ-R Stats, LLC
100% test using CAATs




Provides complete coverage
Best practice
Basis for continuous monitoring
Repeatable process
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3 - Summarized






Understand the framework for managing the business risk of
fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential sampling and
other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is Yellow Book and
SAS 56 …
Managing the business risk of fraud
EZ-R Stats, LLC
Yellow book standards
Standards regarding statistical sampling and IT
Managing the business risk of fraud
EZ-R Stats, LLC
General standards

3.43 Technical Knowledge and competence
“The staff assigned to conduct an audit or attestation
engagement under GAGAS must collectively
possess the technical knowledge, skills, and
experience necessary to be competent for the type
of work being performed before beginning work on
that assignment.
The staff assigned to a GAGAS audit or attestation
engagement should collectively possess: “
Managing the business risk of fraud
EZ-R Stats, LLC
Stat sampling and IT
Skills appropriate for the work being performed.
For example, staff or specialist skills in

(1) statistical sampling if the work involves
use of statistical sampling;

(2) information technology
Managing the business risk of fraud
EZ-R Stats, LLC
SAS 56 – Analytical procedures



Requires use of analytic review procedures
for:
Audit planning
Overall review stages
Managing the business risk of fraud
EZ-R Stats, LLC
SAS 56 – Analytical Review


Encourages use of analytical review
Provides guidance
“A wide variety of analytical
procedures may be useful for
this purpose.”
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4 - Summarized






Understand the framework for managing the business risk of
fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential sampling and
other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is linear
regression …
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
7 - Trends
Trend Busters
Does the pattern make sense?
30,000
25,000
20,000
15,000
10,000
5,000
0
Sales
M
ar
n0
Ja
-0
7
M
ay
-0
7
Ju
l-0
Se 7
p07
N
ov
-0
7
Ja
n08
M
ar
-0
M 8
ay
-0
8
Employee Count
7
Amount
ACME Technology
Date
Managing the business risk of fraud
EZ-R Stats, LLC
7 – Trends
Trend Busters



Linear regression
Sales are up, but cost of goods sold is
down
“Spikes”
Managing the business risk of fraud
EZ-R Stats, LLC
7 – Trends
Purpose / Type of Errors




Identify trend lines, slopes,
etc.
Correlate trends
Identify anomalies
Key punch errors where
amount is order of
magnitude
Managing the business risk of fraud
EZ-R Stats, LLC
7 – Trends
Linear Regression
 Test
relationships (e.g.
invoice amount and sales
tax)
 Perform multi-variable
analysis
Managing the business risk of fraud
EZ-R Stats, LLC
7 – Trends
How is it done?




Estimate linear trends using “best
fit”
Measure variability (standard
errors)
Measure slope
Sort descending by slope,
variability, etc.
Managing the business risk of fraud
EZ-R Stats, LLC
7 – Trends
Trend Lines by Account - Example
Results
Account
N
Slope
Std Err
32451
18
1.230
0.87
43517
17
1.070
4.3
32451
27
1.023
0.85
43517
32
1.010
0.36
43870
23
0.340
2.36
54630
56
-0.560
1.89
Generally the trend is gently sloping
up, but two accounts (43870 and
54630) are different.
Managing the business risk of fraud
EZ-R Stats, LLC
Scatter plot with prediction and
confidence intervals
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 5 - Summarized






Understand the framework for managing the business risk of
fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential sampling and
other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Next is data mining …
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 6
Basis for Pattern Detection




Analytical review
Isolate the “significant few”
Detection of errors
Quantified approach
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2
Understanding the Basis





Quantified Approach
Population vs. Groups
Measuring the Difference
Stat 101 – Counts, Totals, Chi
Square and K-S
The metrics used
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2a
Quantified Approach
Based on measureable
differences
 Population vs. Group
 “Shotgun” technique

Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2a
Detection of Fraud Characteristics

Something is different than expected
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2b
Fraud patterns
Common theme – “something is
different”
 Groups
 Group pattern is different than
overall population

Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2c
Measurement Basis
 Transaction
counts
 Transaction
amounts
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
A few words about statistics
(the “s” word)




Detailed knowledge of statistics not
necessary
Software packages do the “numbercrunching”
Statistics used only to highlight
potential errors/frauds
Not used for quantification
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
How is digital analysis done?





Comparison of group with population as a
whole
Can be based on either counts or amounts
Difference is measured
Groups can then be ranked using a selected
measure
High difference = possible error/fraud
Managing the business risk of fraud
EZ-R Stats, LLC
Demo in Excel of the process


Based roughly on the Wake County
Transportation fraud
Illustrates how the process works, using
Excel
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
Histograms


Attributes tallied and categorized into “bins”
Counts or sums of amounts
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
Two histograms obtained

Population and group
Population
700
Group
80
70
60
50
40
30
20
10
0
600
500
400
300
200
100
0
Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07
Managing the business risk of fraud
Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07
EZ-R Stats, LLC
Objective 2d
Compute Cumulative Amount for each
Count by Month
Cum Pct
80
120.0%
70
100.0%
60
Count
50
80.0%
40
60.0%
30
20
40.0%
10
20.0%
Managing the business risk of fraud
ov
-0
7
N
Se
p07
Ju
l-0
7
07
M
ay
-
07
M
ar
-
Month
0.0%
Ja
n07
Ja
n0
Fe 7
bM 07
ar
-0
Ap 7
r-0
M 7
ay
-0
Ju 7
n0
Ju 7
lAu 07
g0
Se 7
p0
O 7
ct0
No 7
v0
De 7
c07
0
EZ-R Stats, LLC
Objective 2d
Are the histograms different?




Two statistical measures of
difference
Chi Squared (counts)
K-S (distribution)
Both yield a difference metric
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
Chi Squared



Classic test on data in a table
Answers the question – are the
rows/columns different
Some limitations on when it can be
applied
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
Chi Squared





Table of Counts
Degrees of Freedom
Chi Squared Value
P-statistic
Computationally intensive
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
Kolmogorov-Smirnov
Two Russian
mathematicians
 Comparison of distributions
 Metric is the “d-statistic”

Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d
How is K-S test done?

Four step process
1.
2.
3.
4.
For each cluster element
determine percentage
Then calculate cumulative
percentage
Compare the differences in
cumulative percentages
Identify the largest difference
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2d - KS
Kolmogorov-Smirnov
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2e
Classification by metrics










Stratification
Day of week
Happens on holiday
Round numbers
Variability
Benford’s Law
Trend lines
Relationships (market basket)
Gaps
Duplicates
Managing the business risk of fraud
EZ-R Stats, LLC
Objective e
Auditor’s “Top 10” Metrics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers / Variability
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2
Understanding the Basis





Quantified Approach
Population vs. Groups
Measuring the Difference
Stat 101 – Counts, Totals, Chi Square
and K-S
The metrics used
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 2 - Summarized
1.
2.
3.
4.
5.
Understand why and how
Understand statistical basis for quantifying
differences
Identify ten general tools and techniques
Understand examples done using Excel
How pattern detection fits in
Next are the metrics …
Managing the business risk of fraud
EZ-R Stats, LLC
It’s that time!
Session Break!
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3
The “Top 10” Metrics




Overview
Explain Each Metric
Examples of what it can detect
How to assess results
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3
Trapping anomalies
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3
Fraud Pattern Detection
Round Numbers
Market Basket
Benford’s Law
Stratification
Gaps
Target Group
Trend Line
Univariate
Duplicates
Holiday
Day of Week
Managing the business risk of fraud
EZ-R Stats, LLC
1 - Outliers
Outliers / Variability
Outliers are
amounts which
are significantly
different from
the rest of the
population
Managing the business risk of fraud
EZ-R Stats, LLC
1 - Outliers
Outliers / Variability




Charting (visual)
Software to analyze “z-scores”
Top and Bottom 10, 20 etc.
High and low variability (coefficient
of variation)
Managing the business risk of fraud
EZ-R Stats, LLC
1 - Outliers
Drill down to the group level


Basic statistics
– Minimum, maximum
and average
– Variability
Sort by statistic of interest
– Variability (coefficient
of variation)
– Maximum, etc.
Managing the business risk of fraud
EZ-R Stats, LLC
1 - Outliers
Example Results
Provider
N
Coeff Var
3478421
3,243
342.23
2356721
4,536
87.23
3546789
3,421
23.25
5463122
2,311
18.54
Two providers (3478421 and
2356721) had significantly more
variability in the amounts of their
claims than all the rest.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
2 - Stratification
Unusual stratification
patterns
Do you
know how
your data
looks?
Managing the business risk of fraud
EZ-R Stats, LLC
2 - Stratification
Stratification - How
Charting (visual)
 Chi Squared
 Kolmogorov-Smirnov
 By groups

Managing the business risk of fraud
EZ-R Stats, LLC
2 – Stratification
Purpose / types of errors




Transactions out of the ordinary
“Up-coding” insurance claims
“Skewed” groupings
Based on either count or amount
Managing the business risk of fraud
EZ-R Stats, LLC
2 – Stratification
The process?
1.
2.
3.
4.
5.
Stratify the entire population into
“bins” specified by auditor
Same stratification on each group
(e.g. vendor)
Compare the group tested to the
population
Obtain measure of difference for each
group
Sort descending on difference
measure
Managing the business risk of fraud
EZ-R Stats, LLC
2 – Stratification
Units of Service Stratified Example Results
Provider
N
Chi Sq
D-stat
2735211
6,011
7,453
0.8453
4562134
8,913
5,234
0.7453
4321089
3,410
342
0.5231
4237869
2,503
298
0.4632
Two providers (2735211 and
4562134) are shown to be much
different from the overall population
(as measured by Chi Square).
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
3 – Day of Week
Day of Week



Activity on weekdays
Activity on weekends
Peak activity mid to late week
Managing the business risk of fraud
EZ-R Stats, LLC
3 – Day of Week
Purpose / Type of Errors



Identify unusually high/low
activity on one or more days of
week
Dentist who only handled
Medicaid on Tuesday
Office is empty on Friday
Managing the business risk of fraud
EZ-R Stats, LLC
How it is done?






Programmatically check entire population
Obtain counts and sums by day of week
(1-7)
Prepare histogram
For each group do the same procedure
Compare the two histograms
Sort descending by metric (chi square/dstat)
Managing the business risk of fraud
EZ-R Stats, LLC
3 – Day of Week
Day of Week - Example Results
Provider
N
Chi Sq
D-stat
2735211
5,404
12,435
0.9802
4562134
5,182
7,746
0.8472
4321089
5,162
87
0.321
4237869
7,905
56
0.2189
Provider 2735211 only provided
service for Medicaid on Tuesdays.
Provider 4562134 was closed on
Thursdays and Fridays.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
4 – Round Numbers
Round Numbers
It’s about….
Estimates!
Managing the business risk of fraud
EZ-R Stats, LLC
4 – Round Numbers
Purpose / Type of Errors




Isolate estimates
Highlight account numbers in
journal entries with round
numbers
Split purchases (“under the radar”)
Which groups have the most
estimates
Managing the business risk of fraud
EZ-R Stats, LLC
4 – Round Numbers
Round numbers




Classify population amounts
– $1,375.23 is not round
– $5,000 is a round number – type 3 (3
zeros)
– $10,200 is a round number type 2 (2
zeros)
Quantify expected vs. actual (d-statistic)
Generally represents an estimate
Journal entries
Managing the business risk of fraud
EZ-R Stats, LLC
4 – Round Numbers
Round Numbers in Journal
Entries - Example Results
Account
N
Chi Sq
D-stat
2735211
4,136
54,637
0.9802
4562134
833
35,324
0.97023
4321089
8,318
768
0.321
4237869
9,549
546
0.2189
Two accounts, 2735211 and 4562134
have significantly more round number
postings than any other posting
account in the journal entries.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made up numbers
Made up Numbers
Curb stoning
Imaginary numbers
Benford’s Law
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made Up Numbers
What can be detected

Made up numbers –
e.g. falsified inventory
counts, tax return
schedules
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made Up Numbers
Benford’s Law using Excel



Basic formula is “=log(1+(1/N))”
Workbook with formulae available at
http://tinyurl.com/4vmcfs
Obtain leading digits using “Left”
function, e.g. left(Cell,1)
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made Up Numbers
Made up numbers






Benford’s Law
Check Chi Square and d-statistic
First 1,2,3 digits
Last 1,2 digits
Second digit
Sources for more info
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made Up Numbers
How is it done?






Decide type of test – (first 1-3 digits, last
1-2 digit etc)
For each group, count number of
observations for each digit pattern
Prepare histogram
Based on total count, compute expected
values
For the group, compute Chi Square and
d-stat
Sort descending by metric (chi square/dstat)
Managing the business risk of fraud
EZ-R Stats, LLC
5 – Made Up Numbers
Invoice Amounts tested with
Benford’s law - Example Results
Store
Hi Digit
Chi Sq
D-stat
324
79
5,234
0.9802
563
89
4,735
0.97023
432
23
476
0.321
217
74
312
0.2189
During tests of invoices by store, two
stores, 324 and 563 have significantly
more differences than any other store
as measured by Benford’s Law.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
6 – Market Basket
Market Basket






Medical “Ping ponging”
Pattern associations
Apriori program
References at end of slides
Apriori – Latin a (from) priori
(former)
Deduction from the known
Managing the business risk of fraud
EZ-R Stats, LLC
6 – Market basket
Purpose / Type of Errors



Unexpected patterns and
associations
Based on “market basket” concept
Unusual combinations of diagnosis
code on medical insurance claim
Managing the business risk of fraud
EZ-R Stats, LLC
6 – Market basket
Market Basket
JE Accounts
 JE Approvals
 Credit card fraud in Japan –
taxi and ATM

Managing the business risk of fraud
EZ-R Stats, LLC
6 – Market basket
How is it done?




First, identify groups, e.g. all
medical providers for a patient
Next, for each provider, assign a
unique integer value
Create a text file containing the
values
Run “apriori” analysis
Managing the business risk of fraud
EZ-R Stats, LLC
6 – Market basket
Apriori outputs



For each unique value, probability of
other values
If you see Dr. Jones, you will also see
Dr. Smith (80% probability)
If you see a JE to account ABC, there
will also an entry to account XYZ
(30%)
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
8 - Gaps
Numeric Sequence Gaps
What’s there is
interesting, what’s not
there is critical …
Managing the business risk of fraud
EZ-R Stats, LLC
8 – Gaps
Purpose / Type of Errors



Missing documents (sales, cash,
etc.)
Inventory losses (missing receiving
reports)
Items that “walked off”
Managing the business risk of fraud
EZ-R Stats, LLC
8 – Gaps
How is it done?




Check any sequence of numbers
supposed to be complete, e.g.
Cash receipts
Sales slips
Purchase orders
Managing the business risk of fraud
EZ-R Stats, LLC
8 – Gaps
Gaps Using Excel



Excel – sort and check
Excel formula
Sequential numbers and dates
Managing the business risk of fraud
EZ-R Stats, LLC
8 – Gaps
Gap Testing - Example Results
Start
End
Missing
10789
10791
1
12523
12526
2
17546
17548
1
Four check numbers are missing.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
9 - Duplicates
Duplicates
Why is there more
than one?
Same, Same, Same, and
Same, Same, Different
Managing the business risk of fraud
EZ-R Stats, LLC
9 – Duplicates
Two types of (related) tests


Same items – same vendor, same invoice
number, same invoice date, same amount
Different items – same employee name,
same city, different social security number
Managing the business risk of fraud
EZ-R Stats, LLC
9 - Duplicates
Duplicate Payments

High payback area
 “Fuzzy”

logic
Overriding software
controls
Managing the business risk of fraud
EZ-R Stats, LLC
Fuzzy matching with
software





Levenshtein distance
Soundex
“Like” clause in SQL
Regular expression
testing in SQL
Vendor/employee
situations
Managing the business risk of fraud
9 - Duplicates
Russian
physicist
EZ-R Stats, LLC
9 - Duplicates
How is it done?



First, sort file in sequence for
testing
Compare items in consecutive
rows
Extract exceptions for follow-up
Managing the business risk of fraud
EZ-R Stats, LLC
9 - Duplicates
Possible Duplicates - Example Results
Vendor
Invoice Date
Invoice
Amount
Count
10245
6/15/2007
3,544.78
4
10245
8/31/2007
2,010.37
2
17546
2/12/2007
1,500.00
2
Five invoices may be duplicates.
Managing the business risk of fraud
EZ-R Stats, LLC
Next Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Outliers
Stratification
Day of Week
Round Numbers
Made Up Numbers
Market basket
Trends
Gaps
Duplicates
Dates
Managing the business risk of fraud
EZ-R Stats, LLC
10 - Dates
Date Checking
If we’re closed, why
is there …
Adjusting journal entry?
Receiving report?
Payment issued?
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Holiday Date Testing

Red Flag indicator
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Date Testing challenges


Difficult to determine
Floating holidays –
Friday, Saturday,
Sunday, Monday
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Typical audit areas


Journal entries
Employee expense
reports

Business telephone calls
Invoices
Receiving reports

Purchase orders


Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Determination of Dates



Transactions when business is
closed
Federal Office of Budget
Management
An excellent fraud indicator in
some cases
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Holiday Date Testing


Identifying holiday
dates:
– Error prone
– Tedious
U.S. only
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Federal Holidays




Established by Law
Ten dates
Specific date (unless
weekend), OR
Floating holiday
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Federal Holiday Schedule




Office of Personnel Management
Example of specific date – Independence
Day, July 4th (unless weekend)
Example of floating date – Martin Luther
King’s birthday (3rd Monday in January)
Floating – Thanksgiving – 4th Thursday in
November
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
How it is done?




Programmatically count holidays for
entire population
For each group, count holidays
Compare the two histograms (group
and population)
Sort descending by metric (chi
square/d-stat)
Managing the business risk of fraud
EZ-R Stats, LLC
10 – Dates
Holiday Counts - Example Results
Employee
Number
N
Chi Sq
D-stat
10245
37
5,234
0.9802
32325
23
4,735
0.97023
17546
18
476
0.321
24135
34
312
0.2189
Two employees (10245 and 32325)
were “off the chart” in terms of
expense amounts incurred on a
Federal Holiday.
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3
The “Top 10” Metrics




Overview
Explain Each Metric
Examples of what it can detect
How to assess results
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 3 - Summarized
1.
2.
3.
4.
5.
Understand why and how
Understand statistical basis for quantifying
differences
Identify ten general tools and techniques
Understand examples done using Excel
How pattern detection fits in
Next – using Excel …
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Use of Excel




Built-in functions
Add-ins
Macros
Database access
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Excel templates

Variety of tests
–
–
–
–
Round numbers
Benford’s Law
Outliers
Etc.
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Excel – Univariate statistics





Work with Ranges
=sum, =average, =stdevp
=largest(Range,1),
=smallest(Range,1)
=min, =max, =count
Tools | Data Analysis | Descriptive
Statistics
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Excel Histograms



Tools | Data Analysis | Histogram
Bin Range
Data Range
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Excel Gaps testing




Sort by sequential value
=if(thiscell-lastcell <>
1,thiscell-lastcell,0)
Copy/paste special
Sort
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Detecting duplicates with Excel



Sort by sort values
=if testing
=if(=and(thiscell=lastcell, etc.))
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Performing audit tests with macros






Repeatable process
Audit standardization
Learning curve
Streamlining of tests
More efficient and effective
Examples http://ezrstats.com/Macros/home.html
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Using database audit software







Many “built-in” functions right off the shelf
with SQL
Control totals
Exception identification
“Drill down”
Quantification
June 2008 article in the EDP Audit &
Control Journal (EDPACS) “SQL as an
audit tool”
http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4
Use of Excel




Built-in functions
Add-ins
Macros
Database access
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 4 - Summarized
1.
2.
3.
4.
5.
Understand why and how
Understand statistical basis for quantifying
differences
Identify ten general tools and techniques
Understand examples done using Excel
How Pattern Detection fits in
Next – Fit …
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 5
How Pattern Detection Fits In



Business Analytics
Fraud Pattern Detection
Continuous monitoring
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 5
Where does Fraud Pattern Detection fit in?
Right in the middle




Business Analytics
Fraud Pattern Detection
Continuous fraud pattern
detection
Continuous Monitoring
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 5
Business Analytics


Fraud analytics -> business
analytics
Business analytics -> fraud
analytics
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 5
Role in Continuous Monitoring (CM)



Fraud analytics can feed (CM)
Continuous fraud pattern detection
Use output from CM to tune fraud
pattern detection
Managing the business risk of fraud
EZ-R Stats, LLC
Objective 6 - Summarized






Understand the framework for managing the business risk of
fraud
Plan, perform and explain statistical sampling in audits
Reduce audit costs using data mining, sequential sampling and
other sampling techniques
Apply SAS 56, the new SAS suite and the revised (2007) Yellow
Book.
Run, hands-on, the most productive analytic technique
(regression analysis).
Use data mining to introduce greater efficiency into the audit
process, without losing effectiveness.
Managing the business risk of fraud
EZ-R Stats, LLC
Links for more information





Kolmogorov-Smirnov
http://tinyurl.com/y49sec
Benford’s Law http://tinyurl.com/3qapzu
Chi Square tests http://tinyurl.com/43nkdh
Continuous monitoring
http://tinyurl.com/3pltdl
Managing the business risk of fraud
EZ-R Stats, LLC
Market Basket




Apriori testing for “ping ponging”
Temple University http://tinyurl.com/5vax7r
Apriori program (“open source”)
http://tinyurl.com/5qehd5
Article – “Medical ping ponging”
http://tinyurl.com/5pzbh4
Managing the business risk of fraud
EZ-R Stats, LLC
Excel macros used in auditing


Excel as an audit software
http://tinyurl.com/6h3ye7
Selected macros http://ezrstats.com/Macros/home.html

Spreadsheets forever http://tinyurl.com/5ppl7t
Managing the business risk of fraud
EZ-R Stats, LLC
Questions?
Managing the business risk of fraud
EZ-R Stats, LLC
Contact info
 Phone:
(919)-219-1622
 E-mail:
[email protected]
 Blog: http://blog.ezrstats.com
Managing the business risk of fraud
EZ-R Stats, LLC