Transcript Credit Scoring Development and Methods

Credit Scoring
Development and Methods
James Marinopoulos
Head of Retail Decision Model
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
1
Alan Greenspan:
President, Federal Reserve Board
May 1996
“… We should not forget that the basic economic function of these
regulated entities (banks) is to take risk. If we minimise risk
taking in order to reduce failure rates to zero, we will, by
definition, have eliminated the purpose of the banking
system.”
Retail Decision Models
Group Risk - Retail Risk
Risk Families
We are managing different groups of Risk
Customer fails
to pay
Change in
Losing money
market
Wrong Strategy
prices
Processing failures and
frauds
Regulatory compliance
Retail Decision Models
Group Risk - Retail Risk
Retail Decision Models Responsibilities

Policy
– Set Group policy on Decision Models
– Approve Decision Model policy changes

Monitor, Validate and Approve
–
–
–
–
–

New Scorecard Developments
Existing Scorecard Functionality
Proposed changes to Decision Models Processes
New Decision Models Systems functionality
Decision Models Systems functionality changes
Governance
– Monitoring
– Undertake bank validations, reports and presentations for APRA

Risk Measurement
– Set risk benchmarks for scorecards
– Risk grading models

Advise
– Worlds best practice in Decision Models
– Risk related issues surrounding Decision Models
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
4
RDM Structure and Responsibilities
Head of
Retail Decision Models
James Marinopoulos
Senior
Decision Model Manager
(Developments)
Quyen Pham-Nguyen
Graduate
Janet Long
Senior
Decision Model Manager
(Validations)
Nicholas Yannios
Senior
Systems Assurance Manager
Graeme Judd
Manager
Decision Model Validation
Kathy Zovko
Manager
Decision Model Monitoring
Valentina Dragan
Graduate
Maria Demetriou
17/07/2015
Relationship
Ongoing Validations
Developments
Monitoring
Change Requests
Data Analysis
Retail Decision Models
Group Risk - Retail Risk
Systems
5
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
6
What is credit scoring?





A statistical means of providing a quantifiable risk factor for a given
customer or applicant.
Credit scoring is a process whereby information provided is
converted into numbers that are added together to arrive at a score.
(“Scorecard”)
The objective is to forecast future performance from past behaviour.
Credit scoring developed by Fair & Isaac in early 60s
– Widespread acceptance in the US in early 80s and UK early
90s
– FICO scores make 75% of US Mortgage loan decisions
– Behavioural scoring accepted as more predictive than
application scoring
Decision Models are used in many areas of industries:
– Banking and Finance
– Insurance
– Retail
– Telecommunications

17/07/2015
Retail Decision Models
Group Risk - Retail Risk
7
Application Scoring

Application scoring is a statistical means of assessing risk at the
point of application for credit
– The application is scored once

Application scoring is used for:
– Credit risk determination
– Loan amount approval
– Limit setting
Credit
Decision
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
8
Behavioural Scoring

Behavioural scoring is a statistical means of assessing risk for
existing customers through internal behavioural data
– Customers/accounts scored repeatedly

Behaviour scoring is used for:
–
–
–
–
Authorisations
Limit increase/overdraft applications
Renewals/reviews
Collection strategies
Debit
$1344. 12
Debit $234.
$1344.
Debit
01 12
Debit $234.
$1344.
12
Debit
Debit
$987.56 01
Debit
$234.
Debit $6543.22
$987.56 01
Debit
Debit $6543.22
$987.56
Debit
Debit
$32423.11
Debit$32423.11
$6543.22
Debit
Total
$2556.00
Debit
$32423.11
Total
$2556.00
Total
$2556.00
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
Risk
Grading
9
Sample scorecard characteristics

Characteristics used in scorecards are similar to those used in
traditional judgemental lending, e.g.:
 Application
 Financial

Purpose of loan

Assets

Deposit

Liabilities

Security

Monthly repayment

Total Monthly income
Character
 Bureau


No. of bureau defaults

Time at current employment

Adverse ANZ behaviour

Residential status

Time at current address
The difference being that attributes within these characteristics are given
formal weights (scores) and added to produce a resulting score
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
10
Scorecard points (example)
Residential status
Owner
+25
Renter
-30
LWP/Other
+10
Time in employment (years)
<2
2
3-4
10
5-6
15
7+
25
Total monthly income
0
0
<$500
15
<$1000 <$1500 <$2000 <$3000 >$3000
25
31
37
43
48
Total defaults
No Defaults
0
17/07/2015
1
-70
2+
-250
Retail Decision Models
Group Risk - Retail Risk
11
Other Types of Scoring






17/07/2015
Attrition
Authorisations
Recovery
Response
Profitability
Customer
Retail Decision Models
Group Risk - Retail Risk
12
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
13
Good/Bad Odds





A scoring system does not individually identify a good performer
from a bad performer, it classifies an applicant in a particular
“Good/Bad odds” group.
An applicant belonging to a 200 to 1 group, appears pretty safe and
profitable.
If the applicant belongs to a 4 to 1 risk group, we would no doubt
find the risk unacceptable.
There is a “cut-off” point where it is not profitable for the bank to
accept a certain Good to Bad ratio
Based on the above, it is accepted that there will be some “bads”
above the cut-off level set, and some “goods” below the cut-off level
set.
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
14
'Good/Bad' Discrimination

The objective of a scorecard is to have characteristics which
discriminate between Good and Bad accounts with a sufficiently
high probability.
–


Some characteristics are legally or ethically not used
The score will be a measure of the probability of being a Good or
Bad performer.
If the scorecard is performing well then the average scores of ‘Bads’
are lower than the average scores of the ‘Goods’.
Goods
N um ber
O f C lie n ts
800
760
720
680
640
600
560
520
480
440
400
360
320
280
240
200
160
120
80
40
0
B ads
S c o re
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
15
Performance Charts
Goods
Number
Of Clients

The Good/Bad Odds at
each score can be
determined and plotted
onto a Performance chart
8
Bads
800
760
720
680
640
600
560
520
480
440
400
360
320
280
240
200
160
120
80
40
0
1
Score
16400
14
Graph 2 - Log Odds Performance Chart
12
8
128
6
25
3
2 to 1
8 to 1
Retail Decision Models
Group Risk - Retail Risk
800
720
680
640
600
560
520
480
440
400
360
320
280
240
200
160
0
120
0
80
2
40
5
0
17/07/2015
4
Log GBOs (Base 2)
10
645
760
Good/Bad Odds
3250
16
Application Scorecard Construction
Flow Chart
Outsourcing
Data Integrity
•External Data Source
•Scorecard Vendor
•Product Identification
•File Data Availability
•Sampling
•Data Extraction/Cost
Generic Scorecard
Validation
Statistical Analysis
•Characteristic Analysis
•Multivariate model build
•Reject Inference
Set cut-off Score
Implementation
Customised Scorecard
Scorecard Monitoring
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
17
Model Build


Once the characteristics have been selected a statistical
model can be developed.
Multivariate statistical methods include
– Logistic Regression
– Stepwise methods
– Residual analysis

Not all predictive characteristics are used in the model.
– An inter-correlation effect may exist between variables.
– For example, age may be correlated with time at current
employment and therefore only one is necessary in the model.
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
18
Models




Expert Systems
Decision Trees
Linear Regression
Logistic Regression has the following form:
 p 
k
   j 0  j x j
ln
1 p 

p

exp  j 0  j x j
k


1  exp  j 0  j x j
k

Neural Networks
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
19
Model Build

The model is built on dichotomous data. In this case a 1 for “Good”
customers and a 0 for “Bad” customers.
1
0.8
0.6
0.4
0.2
0
0
17/07/2015
200
400
600
Retail Decision Models
Group Risk - Retail Risk
800
1000
20
Logistic Regression

The logistic regression fits the probability better than Linear
regression.
1
0.8
0.6
0.4
Good/Bad Probability
0.2
Logistic
Linear (Good/Bad Probability)
0
0
17/07/2015
200
400
600
Retail Decision Models
Group Risk - Retail Risk
800
1000
21
Reject Inference and Validation

Reject Inference
– Reject Inference is only necessary for scorecards were there is no
performance information for rejected applications
• Applications that are rejected must be included in the final model.
– Behavioural scorecards deal only in existing customers, therefore
do not require reject inference.

Validation
– A randomly selected control group (hold out sample) or proxy
portfolio to test the model.
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
22
Measures of discrimination

Receiver Operating Curve (ROC)
– The Receiver Operating Curve is the area under the curve generated when
the cumulative Bads are plotted against the cumulative goods (Lorenz
Curve).

Gini coefficient (G)
– This discrimination measure is geometrically defined as the ratio of the area
A of the shaded semi-circular area to the area B of the triangle in the Lorenz
diagram.
1
ROC  (G  1)
2

PH (percentage Good for 50% Bad)
– This is defined as the cumulative proportion of Goods up to the median
value of the Bads.
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
Gini.xls
23
Measures of discrimination – (I)
17/07/2015
Lorenz Curve
1
0.9
0.8
Cumulative Bads
• Scorecard performance can be
judged on the level of
discrimination
• Two measure that can be used are:
Gini (or ROC)
PH - % of Goods below 50% of
bads
• 1% of PH could mean an additional
3% approvals
• 1% of PH could mean an reduction
of 0.2% bad debts
Gini=0.62%
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
Retail Decision Models
Group Risk - Retail Risk
0.2
0.4
0.6
0.8
1
10%
Cumulative Goods
24
Measures of discrimination –(II)

Discrimination measures should be determined for discrete
attributes
2
– Chi-Squared
–
(Obs  Exp)
 Exp
 Gi 
Fico (Kullback Divergence)100 (Gi  Bi ) ln
 B 
 i
Based on a book by Solomon
Kullback
“Information Theory and Statistics”
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
25
Issues for Successful Implementation



Cultural Change
Requires top management support
Operational process
– Redesign to minimise manual intervention and maximise cost
savings.

Data Integrity
– Quality of the overall decisions, and subsequently the Portfolio, is
dependant upon the accuracy of the data input. The first time!

Setting the Cut-off score correctly
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
26
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
27
Business Objectives

Increase consistency of lending decisions
– Consistent & unbiased treatment of applicant
• Customers with the same details get the same score
– Total management control over credit approval systems
• Allows for loosening or tightening of lending through credit cycles
• Potential increase in approvals

Reduce operating costs
– Increase in automated processing

Improve customer service
–
–
–
–
17/07/2015
Fast and consistent decisions at application point
More appropriate limit and authorisation decisions
Reduction in collection actions on low risk accounts
Risk based allocation of credit limits and issue terms
Retail Decision Models
Group Risk - Retail Risk
28
Business Objectives (cont)

Improved portfolio management
– Manage credit portfolios more effectively and dynamically
•
•
•
•
Better prediction of credit losses
Management ability to react to changes fast & accurately
Ability to measure & forecast impact of policy decisions
Quick and uniform policy implementation
– Improved Management Information Systems (MIS)
• Permits MIS to be developed to assist business needs and marketing
activities
• MIS can be fed back into future scorecard developments and collection
activities
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
29
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
30
World Banks


ANZ
European Banks
– Banking market in Europe is restructuring
– Banks are merging across country boundaries

UK bank visits
–
–
–
–

Bank A - bank with many recent acquisitions
Bank B - bank dealing with mainly credit cards
Bank C - ex building society now owned by bank
Bank D - large diverse bank
National Australia Bank
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
31
World Banks
UK Banks
AUS Banks
Mortgages
-
Y
Y
-
Y
Y
Personal Loans
Y
-
-
Y
Y
Y
Current Accounts
Y
-
Y
-
Y
Y
Credit cards
Y
-
-
Y
Y
Y
LMI
-
In House
In House
-
External
External
Retail FUM
?
£58b
£47b
£8b
$100b+
$60b
Scorecards
20
-
70
?
50 (12)
New
New
All
All
Best 40%
> 6 months on
Existing
Books
Product
Just Developed
? App Scrds
1 Beh Scrds
Under
Application scorecards
New
Behavioural scorecards
Existing
-
Adequate
Good
Good
Good
Good
Average
B&W
B&W
B&W
B&W
Black (Credit
Black (Credit
(Equifax)
(Equifax)
(Experian)
(Experian)
Advantage)
Advantage)
20+
3
30+
?
40+
15
Data Storage
Bureau
Scoring Modelling Staff
17/07/2015
Development
Retail Decision Models
Group Risk - Retail Risk
32
Bureaus

Fair Isaac is the main bureaus in USA
– “White” and “Black” data is supplied to and from all financial institution

Fair Isaac (Equifax) and Experian are the two main bureaus in UK
– “White” data is supplied to a financial institution if the supply to bureau
– Currently few banks supply and receive “white” data
• Mergers are leading most banks to look at this option
– Fair Isaac is trying to beat Experian in having bureau scores in the UK
• This is only possible when all banks supply “white” data

Credit Advantage is used in Australia
– Provides “Black” data only
– Linked with Decision Advantage (previously Equigen)
– Bureau scores used for ANZ Small Business
• We could use Dunn & Bradstreet for over $250k lending

Baycorp is used in New Zealand
– Provides “Black” data only
– Baycorp is also a collections agency
– NZ puts the smallest amount lost as a default

Baycorp and Credit Advantage have just merged
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
33
Credit Scoring & Bureaus Around the World
“We are not alone!”
Country
No Scoring
Data Collection/
Centralisation
Generic
Scorecards
Application
Scorecards
Only
UK
USA
Canada
South Africa
Spain
Australia
New Zealand
Italy
Germany
France
Belgium
Czech Republic
Hong Kong
Singapore
Thailand
India
Korea
Lebanon
Saudi Arabia
17/07/2015
Behavioural
Scorecards Product Based
Behavioural
Scorecards Customer
Based
Customer
Relationship
Management
Bureau
W&B
W&B
W&B
BB
BB
BB
BB
BB
BB
BB
-
Retail Decision Models
Group Risk - Retail Risk
34
BASEL - The New Accord








The New Accord will give banks with sophisticated risk
management capabilities increased flexibility
More emphasis on bank’s internal measures of risk, supervisory
review and market discipline
Decision support technology has an important role to play
Incentivise better risk management
Data warehouses are fundamental to addressing many of the
requirements
The New Basel Capital Accord
SMB sector will be key
More risk sensitive
Competitive equality
Pillar 1 :
Minimum capital
requirement
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
Pillar 2 :
Supervisory
review
process
Pillar 3 : Market
discipline
35
Paul%20Russell%2013a[1]
Pillar 1 : credit risk

Internal Rating Based (IRB) approach
– Foundation
• Bank sets Probability of Default (PD)
• Standard Exposure At Default (EAD)
• Standard Loss Given Default (LGD)
– Advanced
• Banks sets PD, EAD & LGD


Better recognition of credit risk mitigation techniques
Behavioural scoring
– Internal
– External

Data storage
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
36
Future direction of scoring


“Adaptive Control” first implemented 1985 in USA
– Champion/Challenger processes for determining actions
based on scores
– Required 10 years to be widespread in US
Customer Relationship Management
–
–
–
–


Profitability (NIACC)
Attrition
Propensity to Buy (Cross Sell)
Life time revenue
Recovery scorecards
Operations Research Methods
– Simulation modelling
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
37
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
38
Monitoring Examples

1. Operation Stability Reports
– The four types of front end monitoring reports:
1.1 Approval Statistics Report
1.2 Population Stability Report
1.3 System Rules Referral Report
1.4 Portfolio Statistics Report
– Operational statistics can be obtained as soon as an automated
decision process is implemented
– Early warning indicators of decision functionality error and
scorecard validity
– Should be produced by Business Units or MIS
Retail Decision Models
Group Risk - Retail Risk
Loan Approval/Declines by Score
Approva/Declinal Rates by Score
100%
Auto Declined
Manually Declined
Manually Approved
Auto Approved
90%
80%
Percentages
70%
60%
50%
40%
30%
20%
>1000
951-1000
901-950
851-900
801-850
751-800
701-750
651-700
601-650
551-600
501-550
0%
<=500
10%
Score Bands
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
40
Population Stability

Compare each characteristic and attribute
– over time
– against benchmarks


Plot score distributions over time for potential change
Indicates potential drift in performance
Dec-96
Mar-97
Jun-97
Sep-97
Dec-97
Mar-98
Jun-98
Sep-98
Dec-98
Mar-99
Jun-99
Sep-99
Dec-99
Benchmarks
17/07/2015
NO
25%
23%
24%
22%
21%
19%
19%
22%
20%
20%
18%
18%
17%
29%
YES
75%
77%
76%
78%
79%
81%
81%
78%
80%
80%
82%
82%
83%
71%
Population Stability
90%
80%
70%
60%
50%
40%
Dec-96
Mar-97
Jun-97
Sep-97
Dec-97
Mar-98
Jun-98
Sep-98
Dec-98
Mar-99
Jun-99
Sep-99
Dec-99
30%
20%
10%
0%
NO
Retail Decision Models
Group Risk - Retail Risk
YES
41
Monitoring Requirements

2. Performance Analysis
– The two types of back end monitoring are:
2.1 Scorecard Performance Report
2.2 Characteristic Analysis Report
2.3 Dynamic Delinquency Report
– Performance Analysis is undertaken once a certain level of
customer maturity has been established
– Should be produced by BU and Group Risk
Retail Decision Models
Group Risk - Retail Risk
Loans - Approval & Delinquency Rates
% Approved (LHS)
Loans Approval & Delinquency Rates
Delinquency Rates (RHS)
100%
25%
80%
20%
70%
60%
15%
50%
40%
10%
30%
20%
Delinquency Rates
Approval Rates
90%
5%
10%
0%
0%
1-300
301350
351400
401450
451500
501550
551600
601650
651700
701750
751800
>800
Score

Even with manual assessment below the cut-off score of 350 the
delinquency rates are higher
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
43
Scorecard Performance

Scorecard performance based on 30+ delinquency
– Good/Bad odds increase as expected by score
Non Delinq
Delinq
HL GB Odds
Score Distribution & G/B Odds
4000
40.0
3500
35.0
3000
30.0
2500
25.0
2000
20.0
1500
15.0
1000
10.0
>1000
951-1000
901-950
851-900
801-850
751-800
701-750
651-700
601-650
0.0
551-600
0
501-550
5.0
<=500
500
Score
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
44
Presentation Topics
Overview of scoring
Scorecard Modelling
Business Objectives
World Banks
Monitoring
Future Direction
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
45
Future Direction


Modelling
Experimental Design
– Champion/Challenger Strategies
– Hypothesis testing (uni & multi- dimensional)

Quality Control Techniques
– Control Charts

Operations Research
– Optimisation techniques
– Simulation Models
– Stress Testing
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
46
Conferences


Fair Isaac and Experian are the two main credit scoring companies
world wide
Fair Isaac (Every year, alternating in Europe and USA)
–
–
–
–

Main bureau and FICO Scores in USA
Equifax in UK
Systems included TRIAD
Conference was mainly selling FICO products and systems (but also
Technical)
Experian (Every year, in Europe)
– Formerly CCN
– Systems include Transact and Hunter
– Conference on world wide banking, financial, telecommunications and
predictive modelling usage (Business and/or Management)

University of Edinburgh (Every 2 year in Edinburgh)
– Very technical academic papers
– Proposal to run alternate years in a USA university
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
47
Three Portfolio Dimensions:
Volume, Loss, and Profit
E [ Volume ]
high
Low
cutoffs
High
cutoffs
low
high
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
48
Efficient Frontiers in two dimensions
Low
Cutoffs
0.6
OP
High
Cutoffs
0.0
0.2
0.6
E[Volume]
0.9
E[Loss]
Low
Cutoffs
OP
E[Profit]
E[Loss]
0.6
0.2
0.2
High
Cutoffs
Low
Cutoffs
E[Volume]
0.9
OP
Efficient Frontier
0.0
0.2
High
Cutoffs
17/07/2015
Retail Decision Models
E[Profit]
0.6
Group Risk - Retail Risk
49
Improved portfolio performance
0.6
Low
Cutoffs
0.6
Combined
Scores
OP
High
Cutoffs
Combined
Scores
0.0
0.2
0.6
0.9
E[Volume]
E[Loss]
Low
Cutoffs
Combined
Scores
OP
E[Profit]
E[Loss]
Single Score
0.2
0.2
Single Score
High
Cutoffs
Low
Cutoffs
E[Volume]
0.9
Single Score
OP
Efficient Frontier
0.0
0.2
High
Cutoffs
17/07/2015
Retail Decision Models
E[Profit]
0.6
Group Risk - Retail Risk
50
Best Practices

Combining Application & Behavioural scores (Bayesian estimates)
s
t
Reject set with
combined scores
Accept set with
combined scores
Equal- odds
line c (s,
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
t)
51
Other Techniques



Customer Relation Management
Survival Analysis
Multiple Indicator Multiple Cause
Proportional Hazards.ppt
Measuring Customer Quality.doc
17/07/2015
Retail Decision Models
Group Risk - Retail Risk
52