Transcript שקופית 1

Statistics Seminar
Forecasting Loads for a
Telephone Call Center
Sivan Aldor - Noiman
Advisors:
Prof. Paul Feigin
Prof. Avishai Mandelbaum
1
14.5.2006
Outline
• Call Centers Overview
• The Cellular Phone Company and Data Description
• The Mixed Model
• Evaluation Methodology
• Goodness of Fit tests
• Predicting Average Service Times
• Load Forecasts and Staffing Formulae
2
14.5.2006
Call Centers - Overview
• Call Centers are the primary contact point between service
providers and their customers in modern businesses.
• Call Centers comprise human
agents who provide services, as
well as an automated
interactive facility which often
handles the initial phase of the
customer interaction, and which
may provide complete service
to a significant proportion of the
incoming and outgoing calls.
3
14.5.2006
Call Centers - Overview
There are approximately 2.68 million operator positions
in over 50,000 call centers in the US with some locations
employing over 1000 agents.
A typical call center spends between 60% and 70%
of its annual budget on staff salary
Call center investment in workforce optimization
technologies will exceed $1 billion by 2006.
4
14.5.2006
Project Data MOCCA
Goal of Data MOCCA (MOdels for Call Center Analysis)
Designing and Implementing a (universal) database and
interface for storing, retrieving, analyzing and displaying callby-call information from Call Centers
Enable studying the behavior of:
• Customers
Wait Time, Abandonment, Retrials
• Service Providers / Agents
Service Time, Activity Profile
• Managers / System
Queues Lengths, Loads, Trends
5
14.5.2006
Project Data MOCCA
•Israeli Bank
•Israeli Cellular
Phone Company
•Large US Bank
• The Wharton
School Team ,
University of
Pennsylvania
6
• Technion Team
including the
Statistical
Laboratory
group
14.5.2006
The Cellular Phone Company
The Company’s Call Center
• Working hours are 7AM-11PM Sunday-Friday
• 750 agents are employed on a regular weekday
• Handles 50,000-60,000 calls on regular weekdays
• Provides services to 21 different types of queues. The
three largest are:
 Private queue
(30% of incoming calls)
 Business queue
(15% of incoming calls)
 Technical Support queue ( 10% of incoming calls)
7
14.5.2006
The Cellular Company
Current Forecasting Procedure:
• An automatic “black box” system predicts weekly forecasts.
• Each Thursday the following procedure is carried out by the
planning group:
1. Use 6 weeks of past data as the learning data
2. Predict the week which begins ten days later
3. Use subjective experience to correct the predictions
4. Derive the demand and required staffing for that week
8
14.5.2006
The Cellular Company
Sunday
Monday
Tuesday
Thursday
Friday
Saturday
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Learning
Period
9
Wednesday
Prediction
Lead Time
Forecast
Period
14.5.2006
Data Description
• Arrival counts to the Private queue
• Each day divided into 33 half-hour intervals between 7AM-11:30PM
• The learning data includes working weekdays ranging between
mid-February, 2004 and December 31, 2004
• Irregular days and Holidays were removed
• All together, there are 254 days
10
14.5.2006
Data Description
300
200
100
Number of arrivals per half hour
400
Arrivals to the Private Queue: July 6, 2004
0
5
10
15
20
25
30
35
Period
11
14.5.2006
Data Description
•
The call center has four different billing cycles
•
Each of its customers belongs to one of the cycles.
•
Each billing cycle is characterized by two periods:
1. Delivery Period (1-2 days)
2. Billing Period (usually one day),
about a week later
12
14.5.2006
Data Description
August 2004
Sunday
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
1-B
2-D
3-D
4
5
6
7
8-D
9-D
10-B
11
12
13
14-B
15-D
16-D
17
18
19
20
21-B
22
23-D
24-D
25
26
27
28
29
30-D
31-D
Cycle 1
Cycle 2
Cycle 3
B=Billing
Period
D=Delivery
Period
Cycle 4
We model each cycle using two indicators
one for each period
13
14.5.2006
Billing cycles Selection
Do we really need all 8 indicators?
We employ the following Poisson regression on the
daily arrivals to determine the important indicators:
Daily
Arrivals
N i ~ Poiss (i ) i  1,.., 254
log(i )  Wqi 
Weekday
effect

jcycles
Bi j 
Billing
effect

jcycles
Di j
Delivery
effect
We tried different options…
14
14.5.2006
Characterizing Billing Cycles
Using the LR statistic, comparing to the full
model, we arrived at the following two options:
1. Weekdays, Four delivery period indicators and
Cycle 1 billing period indicator
2. Weekdays, One global delivery period indicator and
Cycle 1 billing period indicator
LR Chi-Square statistics P-value
15
Option 1 5.99
0.1121
Option 2 11.33
0.078
14.5.2006
The Mixed Model
Model assumptions:
1. The arrival process follows an inhomogeneous Poisson
process.
2. During half-hour intervals the rate remains constant
We first use the following variance-stabilizing transformation to “change” the
Poisson data into approximately normal data :
For large enough
dk
the transformed data
16
since
N dk ~ Poisson(dk )
1
ydk  Ndk 
4
follows
ydk ~ N ( dk , 0.25)
14.5.2006
The Mixed Model
ydk   dk   dk
Fixed Effects
 dk  Vd   q  pq
d
d
,k
  d  dk
with  dk ~ N (0, 0.25) i.i.d
Random
Effects
(V1 ,...,VD )T ~ N D (0, G )
(d 1 ,...,dK )T ~ N K (0, R)
 1


G   V2  d D1,1

  d D ,1

d
1,2
d
D 1,2
d
D ,2



d
...  D1,D 
...
1 
...
d
1, D
d ij - No. of days between day i and day j
17
14.5.2006
The Benchmark Model
ydk   dk   dk
 dk   q  pq
d
d
,k
  d   dk
with  dk ~ N (0, 0.25) i.i.d
( d 1 ,..., dK ) ~ N K (0,  I )
T
18
2
14.5.2006
Evaluation Methodology
To test the prediction models use the out-of-sample
performance measures. Predict each week (6 days) between
April 12 and December 24, 2004. There are 203 days.
•
For each week use the same data and lead time as the cellular
company.
• At the end of the process we have for each period k during each
day d:
19
( Ndk , Nˆ dk )
k  1,...,33 d  1,..., 203
14.5.2006
Evaluation Methodology
Evaluate the model using the following measures:
1. For each period k during day d compute:
SE dk  ( Nˆ dk  N dk ) 2
| Nˆ dk  N dk |
REdk  100
N dk
Coverdk  I ( N dk  ( Lowerdk ,Upperdk ))
Width dk  Upperdk  Lowerdk
2. For each day summaries:
K
 SE
RMSE d 
k 1
K
dk
K
APE d 
 RE
k 1
K
Coverd 
20
 Cover
dk
k 1
K
dk
K
K
Width d 
Width
dk
k 1
K
14.5.2006
The Mixed Model – Billing cycles
RMSE
4 Deliveries+Bill 1
1 Global Delivery+ Bill 1
Benchmark
Lower Quartile
34.26
33.33
33.23
Median
40.62
40.46
41.39
Mean
45.51
44.57
46.05
Upper Quartile
53.21
51.72
54.36
4 Deliveries+Bill 1
1 Global Delivery+ Bill 1
Benchmark
Lower Quartile
8.35
8.15
8.3
Median
10.13
9.98
10.46
Mean
11.25
11.06
11.24
Upper Quartile
13.81
13.37
13.49
APE
21
14.5.2006
The Mixed Model – Billing Cycles
Coverage Probability
4 Deliveries+Bill 1
1 Global delivery+ Bill 1
Benchmark
Lower Quartile
0.85
0.88
0.39
Median
0.94
0.94
0.52
Mean
0.92
0.92
0.51
Upper Quartile
1
1
0.61
4 Deliveries+Bill 1
1 Global delivery+ Bill 1
Benchmark
Lower Quartile
134.87
138.15
51.57
Median
156.87
157.22
59.56
Mean
161.61
161.71
63.2
Upper Quartile
184.79
184.48
70.07
Width
22
14.5.2006
The Mixed Model – Weekdays
Weekdays patterns:
Scaled Intra-day arrival patterns
not significantly different
during 76% of the periods.
0.5
3. All other weekdays are
Scaled Arrivals
2. Sundays are different
1.0
1.5
1. Fridays are different
Is it important to have
6 different patterns?
Sun
0
5
Mon
10
Tue
15
Wed
20
Thu
25
Fri
30
periods
23
14.5.2006
The Mixed Model – Weekdays
RMSE
Three-Pattern
Multi-Pattern
Lower Quartile
32.65
33.33
Median
38.25
40.46
Mean
43.3
44.57
Upper Quartile
50.94
51.72
APE
24
Three-Pattern
Multi-Pattern
Lower Quartile
7.83
8.15
Median
9.68
9.95
Mean
10.8
11.06
Upper Quartile
12.86
13.37
14.5.2006
The Mixed Model – Weekdays
Coverage Probability
Three-Pattern
Multi-Pattern
Lower Quartile
0.88
0.88
Median
0.97
0.94
Mean
0.94
0.92
Upper Quartile
1
1
Width
25
Three-Pattern
Multi-Pattern
Lower Quartile
141.53
138.15
Median
160.76
157.22
Mean
164.33
161.71
Upper Quartile
186.79
184.48
14.5.2006
The Mixed Model – Within Periods
Comparing different within-periods covariance
structures
1. AR(1)
2. ARMA(1,1)
3. Toeplitz
26
1
 1

R   2 
 K 1

 1

R   2 
 K  2

K 1 


1 
K  2

1
K 3
 2
1

R
  K 1  K  2

K  2 

1




 K 1 
2




14.5.2006
The Mixed Model – Within Periods
Coverage Probability
RMSE
AR(1)
ARMA(1,1)
Lower Quartile
32.65
32.39
Median
38.35
38.47
Mean
43.40
Upper Quartile
50.94
APE
Median
Is ARMA really worth
Mean
43.23
it?
50.74
Upper Quartile
•AR(1) Procedure
takes 1 hour
AR(1)
ARMA(1,1)
Lower Quartile
7.83
7.83
Median
9.68
Mean
10.8
10.73
Upper Quartile
12.86
12.83
27
Lower Quartile
AR(1)
ARMA(1,1)
0.88
0.88
0.97
0.97
0.93
0.93
1
1
Width
AR(1)
ARMA(1,1)
141.53
138.98
160.76
158.76
Mean
162.84
161.48
Upper Quartile
185.4
161.7
Lower Quartile
•ARMA(1,1) takes
6-7
Median
9.54
hours
14.5.2006
The Mixed Model – Daily Effect
How important is the
Vd
random daily effect if we are
predicting 10-days-ahead?
We might expect it to have a very small influence on the
predictions…
We test two models:
1. The Three-Pattern Model with the random daily effect
2. The Three-Pattern Model without the random daily effect
28
14.5.2006
The Mixed Model – Daily Effect
Coverage Probability
RMSE
Lower Quartile
Median
Mean
Upper Quartile
Lower Quartile
Median
Mean
Upper Quartile
29
Daily
No Daily
32.65
32.03
Lower Quartile
Daily
No Daily
0.88
0.85
When we Median 0.97
Mean
0.93
43.40
44.53 carried out
50.94
54.12
one-day- Upper Quartile 1
ahead
APE
Width
predictions
Daily
No Daily
Daily
there were Lower Quartile 141.53
7.83
7.86
bigger
Median
160.76
9.68
10.09
Mean
162.84
10.8
11.03 differences
38.35
12.86
38.63
13.2
Upper Quartile
185.4
0.94
0.91
1
No Daily
136.12
151.61
154.63
174.95
14.5.2006
Testing for Goodness of Fit
The Mixed Model Normal Assumption
First we check the QQ-plot of the prediction residuals to see if it is
normally distributed.
30
14.5.2006
Testing for Goodness of Fit
The Mixed Model
ydk   dk   dk
Is the estimated
 dk  Vd   q  pq
d
d
,k
  d  dk
with  dk ~ N (0, 0.25) i.i.d
value of this
variance
T
(V1 ,...,VD ) ~ N D (0, G )
(d 1 ,...,dK ) ~ N K (0, R)
T
close to 0.25?
We can use ARMA(1,1) to our
advantage and check this out
31
14.5.2006
Testing for Goodness of Fit
The Mixed Model
 1


G   V2  d D1,1

  d D ,1

d
1,2

d D1,2
d
D ,2



d
...  D1,D 
...
1 
...
d
1, D

 R2  (1  )   2

*
The mean ˆ 2  0.309
 1

1
K  2 

*
2 
1
K 1
R   R*  1

 




R   2    K  2  K 3

1





K 2
 K 1

 
2   2
32

1 
2
 2
   2
14.5.2006
Operational Regimes
• Efficiency Driven
• Quality Driven
• QED (Quality-Efficiency Driven)
S  R R
R    E ( Service)
33
14.5.2006
QED Regime
To predict the load (R)
1. We need to predict the arrival rate
2. We need to predict the average service
34
14.5.2006
Average Service Time Patterns
We fitted a quadratic regression based on the weekday and period
The final model has the following components:
Period
Weekday
Period*Period
Weekday*Period
Date
35
14.5.2006
Average Service Time Patterns
The Predicted Average Service patterns for a typical week:
3.0
2.5
Average Service time (min)
3.5
4.0
Predicted Service Time Patterns
2.0
Sunday
Monday
Tuesday
Wednesday
Thursday
Friday
0
36
5
10
15
20
period
25
30
14.5.2006
Predicting Staffing Levels
Given the user supplied
follows:
ˆ
ˆ


Sˆ   
ˆ
ˆ


S  



we estimate as
predicted no. of required agents
actual no. of required agents
ˆ 
ˆ 

  

ˆ
Sˆ  S
ˆ Sˆ 
   S 
 ˆ   1
QED
 ˆ








37
14.5.2006
Predicting Staffing Levels
No. of Agents Difference vs. Period
1.0
Averaging the difference Ŝ  S
+
separately for each period
0.5
over the days we get the
+
-0.5
++
+
+
-1.0
+
+ +
+
+
+
+
+
+
+ +
+
++
+ +
+ +
+ +
+
+
+
-2.0

 Sˆ  S

+
+
+
-1.5
No. of Agents
0.0
following graph:
QED 
+
0
5
10
15
20
25
30
Period
38
14.5.2006
Predicting Staffing Levels
10
0
-10
-30
-20
Difference in number of agents
20
30
No. of agents difference boxplots vs. period
1
39
3
5
7
9
11 13 15 17 19 21 23 25 27 29 31 33
Period
14.5.2006
Work in Progress
1. Comparison with other models
2. Comparison with industry models
3. Trying to answer: “What is the effect of different
time resolutions on prediction performance?”
4. Tying up some loose ends…
40
14.5.2006
NOT
41
14.5.2006
The Prediction Lead time
10.0
9.9
9.8
40.0
9.7
39.6
39.2
0
42
mean APE
10.4
10.2
41.2
41.6
mean RMSE
mean APE
40.4 40.8
mean RMSE
42.0
42.4
10.6
42.8 43.2
10.8
What is the influence of the lead time on the APE
The average RMSE and APE vs. the Prediction Lead time
and RMSE?
1
2
3
4
5
6
Lead timelag(Days)
7
8
9
10
14.5.2006