Transcript Shanti Majithia - University of Reading

Terms of Reference
Historic
Basics
1. Focus on illustrating how statistical methods are used to solve business
problems and how statisticians interact with colleagues and clients to
Models
achieve this.
2. Descriptions of past and on-going case studies
3. Short introductions to their organisations and to the diverse roles of the
organisation’s statisticians,
WEATHER
from Met Office
(Actual and forecast)
Reading University
RSS
15th June 2005
Shanti Majithia
Forecasting Development Manager
Wokingham, Berks
UK Transmission
Agenda





My Background
Company Background
Application of Statistical techniques within the Company
University and Project work
Conclusion
My Background









Further education in London Maths Stats and Computing
Market Electricity Load research, Manpower planning
Operational Forecasting (Short Time Scale)
Liaison with students and Uni. to assist in data and direction
Presentations: Research paper and Forecasting conferences
Wind Energy, Climate Change, Heating and Cooling Load ( Air Con)
Risk management
Short term Gas Demand and Supply Forecasting
Translating data, analysis and information into decision making tools
National Grid Transco
- principal activities in regulated electricity and gas industries
UK:
E&W transmission
GB Gas Transportation
LNG
GridCom
USA:
NEESCom
Zambia:
38.6% CEC
(Copperbelt transmission)
Argentina:
27.6% Transener
Australia:
Basslink (Interconnector
to Tasmania)
National Grid - UK : Electricity
Over 13,000 circuit km
of 400 & 275kv
transmission lines and
cables
Over 21,000
Transmission Towers
300
substations
Fibre optics
Electricity
Balance generation and
demand efficiently
Ensure quality and security
Non stop process
Keeping the lights on
Electricity Transmission Elements
Power Station
23kV
132kV
Large Factories,
Heavy Industry
33 kV
Generator
Transformer
400kV
Transformer
Medium
Factories,
Light Industry
11 kV
240 V
}
To Small Factories,
Farms,
Residential Areas
and Schools
96/29355 ISSUE A SH. 1 OF 1 30-04-99
The UK Gas Industry Model
Competitive
Gas supply
Monopoly
Independent transmission
• Producers
• DFO’S
TRANSCO
• Storage Operators
• Shippers
• Traders
40 % of Distribution
TRANSCO & IPGTS
Suppliers
Energy Companies
Regulated Systems
Gas: National Transmission System (NTS)
• 6,600km 450-1220mm diameter pipeline
• High strength steel X65-X80
• Operating pressure design
70-94bar
• 7 Transco terminals
• 24 compressor stations
• 400 above ground installations (AGI)
St. Fergus
6300 km Pipelines
Terminals
Compressors
Regulators
Teesside
Barrow
Key Stats
• Max demand 02/03
• Peak Demand (1/20)
• Energy Supplied
Burton Point
205 GW
240 GW
1150 TWh/yr
Rough
Easington
Theddlethorpe
Bacton
Gas: From Beach To Meter
Salt
cavity
storage
Terminal
High pressure
storage
LDZ Offtake
Compressor
Power Station
LNG
storage
0
0
1
2
6
5
C
U
B
I
C
F
E
E
T
M
e
t
e
r
Regulator
Station
Industry
Governor
Low
pressure
storage
Real Time System Operation in Gas and
Electricity…..





Balance supply and demand efficiently
Facilitate the market
Ensure quality and security
Maximise system capacity
Non stop processes
BUT


Gas can be stored => daily balancing
Electricity can’t => real-time balancing
Application of Statistical Techniques within NGT



Data collection - live metering, market intelligence and field
measurement
Data mining e.g. Kohonen SOMs, Genetic Algorithms.
Forecasting Methods
Regression, Box-Jenkins, Bayesian, Neural Network (MLP & ALN),
Curve fitting and Holts-Winters, Arch and Garch



Probability and Risk Management
Liaison to keep abreast of modern methods e.g. Statistical methods
Management Information System
Area of Application of Statistical Techniques








Forecasting Energy Demand
Trading advice
Minimising of volatility
Management of probability and risk
Calculating and calibrating climate sensitivity
Health of the assets in terms of the return period
Simple use of statistical methods in plant reliability
Responses on the efficiency of the equipment
Electricity Forecasting Techniques








Multiple linear regression
Last 3 years of historic data
Summer (BST) and winter (GMT)
Weekdays / Sat / Sun
Special days excluded
‘Conventional’ and ‘Trend’ models
~ 120 models per annum
Interpolation between cardinal points for half hourly resolution
Forecasting Tools









Oracle database
Weather and demand feeds
StatGraphics
EViews
SAS
PREDICT & Forecaster
Clementine
NN and ALN
Genetic Algorithm Library (MIT)
The Forecasting Process
Weather Input
Mathematical
& Statistical
Models
Historical Demand
Input
The
Forecast
Demand - Influences
GW 55
Seasons/ Weather
50
Maximum Winter Day
45
“Exceptional events”
40
Typical Winter Day
35
Typical Summer Day
TV
30
25
Minimum Summer Day
20
15
00:00 02:00 04:00 06:00 08:00 10:00
12:00 14:00 16:00 18:00 20:00 22:00
24:00
NGC System demand during 3 minute silence on 14 September 2001 in
memory of the tragedy in America
38500
36500
The start of the Drop, just before 11:00
36000
Remembrance Day 1999
gave a 750 MW drop
37000
2700MW
Highest ever
drop in
Demand.
36500
36000
35500
35500
35000
3 minute silence
Previous Drop on the
11 August 1999
- 2200MW
The Solar Eclipse
Remembrance
day 1999
Eclipse 11-08-99
34500
34000
33500
13 minute duration
35000
10
30
10
32
10
34
10
36
10
38
10
40
10
42
10
44
10
46
10
48
10
50
10
52
10
54
10
56
10
58
11
00
11
02
11
04
11
06
11
08
11
10
11
12
11
14
11
16
11
18
11
20
11
22
11
24
11
26
11
28
11
30
Demand 2001
37500
33000
Demand 1999
38000
The Effect of Temperature on Demand
7000
6000
Demand Effect (MW)
5000
4000
3000
COLD
High Demand
HOT
High Demand
2000
Comfortable
1000
0
Temperature
Degrees Centigrade
The Effect of Illumination on Demand
4500
DULL
High Demand
4000
Demand Effect (MW)
3500
3000
2500
2000
1500
1000
BRIGHT
Low Demand
500
0
0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160
Logarithmic Function of Illumination
Four Weather Variables
•
Average Temperature [TO]: average of 4 spot hourly
temperatures up to current hour
•
Effective Temperature [TE]: TO lagged to 50% with
TE from 24 hours previous
•
Cooling Power of the Wind [CP]: empirical
combination of temperature and wind speed
•
Effective Illumination of the Sky [EI]: (EI=MI-ID),
where ID is a function of visibility, numbers and types
of cloud layers and amounts of precipitation and MI is
maximum illumination. In the logarithmic domain.
Winter Week Day Peak Demand Modelling
Multiple Regression Model Of Demand
An econometric regression model of the weekday darkness peak is
determined on the four previous winters demand & weather data
Weekday Darkness Peak Demand = Mean Darkness Peak Demand
+ Weather Dependant Demand
+ Day of Week
+ Seasonal trends
+ error terms
The days affected by Christmas & New Year holidays are excluded from the sample
Weather Dependant Demand
Weather Dependant
Demand function
 TEt+2TEt2+EIt+CPt
Example of a Weather forecast data
Example 1: MG3 file for receiving Forecast Data (Sample
file format for 0800 bulletin.)
‘A00’,’MG3’,’19082002’,’073030’,007
‘D00’, ‘19082002’, ‘080000’
‘D10’, 1,,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’ ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘D01’, ‘LON’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘19082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘20082002’,
‘19082002’,
‘090000’,
‘110000’,
‘130000’,
‘150000’,
‘170000’,
‘190000’,
‘210000’,
‘230000’,
‘010000’,
‘030000’,
‘050000’,
‘070000’,
‘090000’,
‘110000’,
‘130000’,
‘150000’,
‘170000’,
‘190000’,
‘210000’,
‘230000’,
‘010000’,
‘030000’,
‘050000’,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
15.0,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, 04,
0,,,
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
0, ‘SSW’
Gas Forecasting
- suite of models using different techniques
Inday (Simple regression)
D
STF (Complex regression)
D-1
D
Neural network
D-1
D
ALN (Adaptive logic network)
D-1
D
Profile (ARIMA)
D-1
D
Bayes (Complex regression)
D-1
Box 1 (Box Jenkins)
D-1
Box 2 (Box Jenkins)
D-1
Sumest (Complex regression)
D-1
Wintest (Complex regression)
D-1
Average
weighted according to
performance over last 7 days
(Combination). Further
adjustment made based on
recent combination error
(CAM)
What Does a Gas Model Look Like?
PROFILE – WITHIN DAY MODEL
PROFILE model uses the Box Jenkins technique to forecast within day gas demand. There are two
different models in the program. Model 1 is usually used for the 10am forecast and model 2 for the rest
of the day. However, if the 9am temperature is greater than either the 1pm or 3pm temperature then
model 1 is used for the 1pm and 4pm forecasts.
Model 1 (at hour k) (used for 10:00 forecast)
7Dt(h) = w07Tt(3) + w17Tt(6) + w27Tt(9) + w37Dt(k) + (1-1B) (1-7B7) at
Model 2 (at hour k) (used for forecasts at other times)
7Dt(h) = w07Tt(h-1) + w17Dt(6) + w21k7Dt(j) + (1-1B) (1 - 7B7)at
where
Tt(h) is the temperature at hour h on day t,
Dt(h) is the corresponding hourly demand on day t,
at is the error in the forecast demand for hour h on day t,
B is the backward shift operator i.e. Byt = yt-1
w0, w1, w2, w3, 1, 7 are model parameters.
.
NTS Supply Forecasting Model types
When
For
Every
What
Horizon
How - Model type
Every
hour
supply
point
End of
day
Within
day
Regression of DFNs & AT-Link noms
Every
hour
Nat
End of
day
Within
day
Regression of DFNs & AT-Link noms
End of
day
Day
ahead
Regression of change of supply
Every
hour
7 days
ahead
Holts-Winters (Time series)
Every
Every
hour
Once
per
day
supply
point
Every
supply
point
What is a Holts-Winters Model?
Yt
Lt  
 1   Lt 1  bt 1 
S t s
bt   Lt  Lt 1   1   bt 1
Yt
St  
 1   S t  s
Lt
where s is the length of the seasonality. L is the smoothed level of the series, b
is the trend of the series and S is the seasonality component.
The Forecast uses the following equation;
Ft  Lt  bt St  s
Understanding Data Questions





What to look for in the data before preparing forecasts
How to treat data when “problems” are recognised
How to prepare forecasts using different models and techniques
When each forecasting model is appropriate
How to use forecasts effectively after they are prepared
Key Questions






Why is a forecast needed?
Who will use the forecast, and what are their specific
requirements?
What level of detail or aggregation is required and what is the
proper time horizon?
How accurate can we expect the forecast to be?
Will the forecast be made in time to help decision making process?
Does the forecaster clearly understand how the forecast will be
used in the organisation?
Projects and Case Studies
The seasonal forecast of electricity demand: a
simple Bayesian model with climatological
weather generator
Sergio Pezzulli, Patrizio Frederic, Shanti Majithia,
Data mining --- Clustering of Electricity Profiles
Coloured areas are clusters, each with a distinctive daily
demand profile. Red text is their interpretation.
Clustering of Gas Profiles
Kohonen Network (SOM) Analysis
Jan &
Dec
Jan Feb Mar
& Nov
Apr May
& Oct
Yellow-ish areas indicate similar profiles,
Red-ish areas indicate more varying profiles.
June July
Aug & Sept
North Thames LDZ, Early Jan 2003
Fore cast De mand
London
1 Day Lag Forecast
Real Demand
2.5
MCM
2
1.5
1
209
201
193
185
177
169
161
153
145
137
129
121
113
105
97
89
81
73
65
57
49
41
33
25
17
9
1
0.5
Error
0.3
0.2
0.1
0
1
-0.1
-0.2
-0.3
10
19
28
37
46
55
64
73
82
91
100 109 118 127 136 145 154 163 172 181 190 199 208
New up coming Challenges
Windpower





Variable
Uncertain
Uncertain uncertainty
Danger: possibility of sudden loss
Weather differences can be at finer geographic resolution
Volatility and Uncertainty
How best to model? Ensemble forecasts?
How to make operational decisions?
Site Clustering




Site clustering can be used to produce
a more accurate national prediction by
taking local conditions into account
The main way of achieving this is to
have a ‘reference’ farm which is
representative of the cluster
It is possible to then use cluster
predictions as inputs to a national
model or simply upscaled
One further thought is to forecast both
a reference farm and a cluster
separately and use them to create a
more stable regional prediction
Daily Load Forecasting using ARIMA-GARCH
and Extreme Value Theory
University of Loughborough EPSRC Project
Application



Climate Change Impacts on Electricity demand can be categorised into
a long term (monthly) and short term (daily and hourly) load forecast.
Long term load forecast using the multiple regression approach
completed. The results are satisfactory. 80 years projection requires the
UKCIP scenario and BESEECH data (population, GDP, consumer
spending).
Short term load forecast using Box Jenkins and Extreme Value Theory
is also completed. Waiting for hourly climate data from BADC and CRU
before we can extend our daily/hourly projections to 2080s.
ARIMA (p, d, q) Model


The AutoRegressive Integrated Moving Average (ARIMA) model is a
broadening of the class of ARMA models to include differencing.
Reason: daily and hourly pattern are volatile and shows a strong seasonal
pattern. p: no. of autoregressive terms, d: the number of non-seasonal
differences and q = no of lagged forecast errors in the prediction equation.
p
q
d
yˆ t   C   i yt  i     j t  j    k X t , k    t



 j 1 


i 1
k 1
auto regressive
moving average
ARIMA(1,1,1) is used
regressor
white noise
Probability Distributions


nt is a standardized, independence, identically distributed (iid)
random draw from some probability distributions.
3 distributions are used for this purpose:a) Normal
b) Student-t
c) Extreme Value Distribution

For quantiles > 0.95, extreme value distribution is used.
Example of Scenario Forecasting
(with max and min scenarios)
33000
31000
29000
MW
27000
25000
23000
Forecast issued on Friday 12:00hrs
21000
Maximum risk scenario
Minimum risk scenario
19000
Actual
HALF-HOURLY
23
00
17
00
18
30
20
00
21
30
11
00
12
30
14
00
15
30
93
0
80
0
63
0
50
0
33
0
20
0
30
17000
Combination of Distribution-- Example
Link between Annual Peak
and Weekly Peak
Density Traces
(X 0.0001)
3
Winter ACS Median
density
2.5
2
12% Area cut off
Weekly Peak
Distribution
1.5
1
0.5
0
Variables
47
Simulated Weekly Peak Demand
Simulated Winter Peak Demand
48
49
50
51
52
53
54
55
56
57
58
59
60
GW
The density traces shows how the median of
the simulated winter peak distribution cuts off
an area of about 12% on the corresponding
distribution of simulated weekly peak demands.
Probability Distribution
%
99
94
89
84
79
74
69
64
60
55
50
45
40
35
30
25
20
15
10
5
58000
57000
56000
55000
54000
53000
52000
51000
0
MW
62000
61000
60000
59000
Conclusions

Various statistical applications demonstrated

Wide variety of Statistical method used in data rich Energy business

Opportunity for Statistician/Business Analysis