Transcript Time series methods - WWW-Docs for TU

Systems Research Institute
Polish Academy of Sciences
Koelpinsee 2010: Modelierung und Simulation
Forecasting of the hydraulic load
of communal wastewater networks
Jan Studzinski
Marcin Stachura
IBS PAN Warszawa
[email protected]
IBS PAN Warszawa
Contents:
•Introduction
•Time series methods
•Neuronal nets
•Fuzzy set models
•Results of modeling
•End conclusions
2
IBS PAN Warszawa
Introduction
A water and sewage system consists usually of three basic objects: of water supply
system, wastewater network and of sewage treatment plant.
They are connected each other in series and the work quality of one of them affects the
functioning of the following one:
• the water production for the water net has an influence on the hydraulic load of the
wastewater net and it decides of the raw sewage inflow entering the sewage treatment
plant
• the sewage inflow affects the quality of sewage purification and makes worse the
treatment plant control in case of fast and big inflow changes
It is important to know in advance the inflow changes to have the opportunity to prepare
the plant controllers on the oncoming events.
A method to predict the sewage inflow changes is to model them mathematically.
3

At the Systems Research Institute of Polish Academy of Sciences an integrated
information system for complex management of communal waterworks is under
development.

It consists of three subsystems for the water supply system, wastewater network
and the sewage treatment plant.

Each of the subsystems has the modular structure and the component modules are
GIS, SCADA, optimization algorithms and mathematical models improving the
management of the basic waterworks objects.

These mathematical models are the hydraulic models of water- and wastewater nets,
the physical model of sewage treatment plant and the models to forecast the
hydraulic loads of the water- and wastewater nets.

To improve the control of the sewage treatment plant there is recommended to
have the models with which the raw sewage inflow entering the treatment plant
could be predicted.
IBS PAN Warszawa
Time series methods
The main methods for the time series modeling are based on the classical least squares
method.
Its advantage is the big simplicity and efficiency and also the clarity of its mathematical
description.
The calculation task of the time series methods consists in general in solving a system
of linear algebraic equations, regarding the model parameters.
In the modeling calculations three time series methods have been used: the least
squares Kalman method, the generalized least squares Clarke method and the
maximum likelihood method.
5
IBS PAN Warszawa
The general descriptions of the process investigated and of the model are:
1
M
yn   A( z ) yn   B( z 1 ) xin  vn
i 1
M
yˆ n   Aˆ ( z 1) yn   Bˆ ( z 1) xin
i 1
respectively, with n = 1, 2, ..., N, - number of measurements data.
The process and model equations can be formulated in the matrix form:
y     v
yˆ    c
6
IBS PAN Warszawa
The estimator c of the process parameters is calculated by minimizing the following
residual sum:
N
N
Sv (c)   ( yn  yˆ n )   vˆn2
2
n 1
n 1
with - the estimator of correlated noise v.
The Kalman estimator resulted while minimizing the sum is:
c  ( T  ) 1 T y
and it is asymptotically biased.
It would be asymptotically unbiased when
v 
i.e. for the uncorrelated noise.
7
IBS PAN Warszawa
The Clarke method is the least squares method applied to the process equation with an
additional description of the noise correlation in form of the relation:
(1  D( z 1 )) vn   n
The noise equation in the matrix form is:
v   
The idea of this method is the transformation of the model equation in such the way that
the correlated noise v in it will be changed into the uncorrelated one.
The parameters estimator would be then asymptotically unbiased.
8
IBS PAN Warszawa
The maximum likelihood method is the least squares method applied to the process
equation with an additional description of correlated noise v, which is different from this
one in the Clarke method.
To the process equation the following noise equation:
vn  (1  D( z 1))  n
is added.
9
IBS PAN Warszawa
Neuronal nets
The artificial neuronal nets try to imitate the operation of the biological neuronal
nets of human beings.
This imaging is very rough because of large quantitative limitations of artificial
nets.
The brain of human being consists of more than 10 billion neurons which are
combined each other with more than several thousand connections.
In an artificial net there are usually not more than several hundred neurons
and not more than several dozen connections between two selected
neurons.
Another essential difference between a real and an artificial neuronal net is
the division of this last one into layers on which the neurons are placed.
Such the structure simplifiers the formulation of the mathematical net
description.
10
IBS PAN Warszawa
To each neuron on a layer the signals from the neurons located on the anterior
layer are transmitted.
The signals entered a neuron are multiplied by weight coefficients and
accumulated.
If the sum resulted is higher than the critical threshold value attached to the
neuron than the neuron ignition succeeds.
The totaled signal is converted by using a transition function related to the
neuron.
The signal value computed by the transition function means the output signal
of the neuron.
The mostly used transition function in neuronal nets is the nonlinear sigmoidal
function.
11
IBS PAN Warszawa
By calculating neuronal nets a learning process is realised during which the
end structure of the net is formulated.
From the net the inter-neuronal connections are eliminated for which the values
of the weight coefficients are stated as zero or close to zero.
During the learning process the data from the learning set are used to model
the network.
By the modeling the error generated by the network is minimized.
This error is calculated using an error function and it is usually the squares
residual sum.
The minimization of the squares residual sum occurs usually with an algorithm
of gradient optimization.
12

An essential problem while learning a neuronal net is gaining by it the ability
for generalization, i.e. for making right forecasting with the use of other
data sets.

Very often the correct neuronal net resulted from the learning set gains
wrong results with other calculation data.

This event is called the net overlearning.

A neuronal net with a big number of weight coefficients or hidden layers has
usually a large tendency to an excessive adjustment to the data instead of
ignoring their insignificant changes.

Complex neuronal nets reach almost at all times the smaller calculation
errors than the simpler ones but it shows rather on the overlearning effect
of them than on the good quality of the models.

To avoid the overlearning of neuronal nets a validation approach is used.

It consists in realizing a simulation run with the model resulted from the
learning iteration with another data set.

If the quality of the model won by the learning process and the quality of the
results won by the validation approach are similar then the model is
correct.

When the error resulted while learning the neuronal net is growing smaller in
successive iterations and the error resulted from the validation runs is
growing higher then the neuronal net is going to be overlearned.

Then it has to be simplified by reducing the number of its hidden neurons or
its hidden layers.

In order to improve the valuation of the neuronal net a third data set: testing
set, is isolated from the initial measurements data.

Then the model calculated with the use of the learning set and verified
using the validation set is tested additionally using the testing set.

The testing calculation is done only once after the whole learning process is
finished.
Fuzzy set models
Zadania
realizowane
obecnie przez have
programy
systemu,
Fuzzy models
of Takagi-Sugeno-Kanga
been applied
for c.d.:
the modeling.
•
Zadania realizowane obecnie przez programy systemu, c.d.:
•Lokalizacja
awarii
(SCADA)

Lokalizacja
awarii (SCADA)
The algorithm of modeling consists of three steps:

Lokalizacja
awarii
+ MOSUW,
wariant I) wariant I)
••Lokalizacja
fuzzyfication
of
the (SCADA
inputs
data
awarii
(SCADA
+ MOSUW,
• fuzzyficated

Obliczanie conclusion
wysokości węzłów (GIS + KRIPOS)
•Obliczanie wysokości węzłów (GIS + KRIPOS)
• defuzzyfication of the ouput signal

Kalibracja modelu hydraulicznego (MOSUW + REH)
•Kalibracja modelu hydraulicznego (MOSUW + REH)
map rozkładów
ciśnień
i przepływów
(MOSUW +are
KRIPOS)
On theWykreślanie
step of fuzzyfication
the
adherence
functions
used. Their values are from 0
up
to 1 and theymap
are usually
in form
of trapezoid.
•Wykreślanie
rozkładów
ciśnień
i przepływów (MOSUW + KRIPOS)

Dobór punktów monitoringu (MOSUW)
•Dobór
punktów
monitoringu
(MOSUW)
On the step
of defuzzyfication
the classical
linear time series models are used.
By the modeling the initial are are divided into two equal sets: the learning and the
testing set.
IBS PAN Warszawa
Results of modeling
The data used for the calculations has been got from the waterworks in Rzeszow.
They are daily measurements data series concerning:
o the water production for the communal water net
o raw sewage inflow reaching the sewage treatment plant
o rainfalls data for the city Rzeszow and
o the water level values in the river flowing through the city.
The number of measurements in each of the data series is equal to 974.
17
P roduk c ja wody
D a ta
D opły w ś c ie k ów do oc z y s z c z a lni
2007-04-01
2007-02-01
2006-12-01
2006-10-01
2006-08-01
2006-06-01
2006-04-01
2006-02-01
2005-12-01
2005-10-01
2005-08-01
2005-06-01
2005-04-01
2005-02-01
2004-12-01
2004-10-01
2004-08-01
2004-06-01
2004-04-01
2004-02-01
2003-12-01
2003-10-01
2003-08-01
2003-06-01
Dopływ ścieków, produkcja wody i
różnica, m 3/d
120000
100000
80000
60000
40000
20000
-20000
0
-40000
-60000
-80000
p o m i a ru
R óż nic a
To evaluate the models the following criteria have been used:
1 N
  ( xˆt  xˆ ) 2
N t 1
 xˆ 

  xˆ
x
MSE 
x 
1 N
  ( xt  x ) 2
N t 1
1
1
2
 (xt  xˆt )  SSE
N
N
N
R
 ( xt  x )( xˆt  xˆ )
t 1
N
N
t 1
t 1
2
2
 ( xt  x )   ( xˆt  xˆ )

 SSE 
AIC  2,83788771  ln 
  N  2 L p
N



IBS PAN Warszawa
On the first stage of modeling the time series methods of Kalman (K), Clarke (C) and
of the maximum likelihood (ML) have been used.
The models:
• with three inputs, i.e. with the water production (WP), rainfalls data (R) and with the
water level values in the river (WL)
• with two inputs, i.e. with WP and WL or with WP and R
• with only one input, i.e. with WP or with WL or with R
have been developed.
As the single output of the models the raw sewage inflow has been taken at all times.
20
IBS PAN Warszaw
Table 1. Time series models with three inputs.
Model/order/
inputs
 x̂

K/6/WP-WL-R
C/6/WP-WL-R
ML/3/WP-WL-R
3.164
3.164
3.822
0,47
0,49
0,60
Evaluation criteria
MSE
R
100,1
0,87
100,1
0,87
146,1
0,80
AIC
18,518
18,530
18,858
21
IBS PAN Warszawa
Table 2. Time series models with two inputs.
Model/order/
inputs
K/6/WP-WL
C/6/WP-WL
ML/3/WP-WL
K/6/WP-R
C/6/WP-R
ML/3/WP-R
 x̂

4.081
4.081
4.354
3.320
3.320
3.902
0,64
0,64
0,68
0,52
0,52
0,61
Evaluation criteria
MSE
R
166,5
0,77
166,5
0,77
189,6
0,74
110,2
0,85
110,2
0,85
152,3
0,79
AIC
19,000
19,002
19,106
18,598
18,600
18,893
22
IBS PAN Warszawa
Table 3. Kalman models with one input.
Model/order/
inputs
K/6/WP
K/6/R
K/6/WL
 x̂

4.272
3.624
4.333
0,67
0,57
0,68
Evaluation criteria
MSE
R
182,5
0,74
131,3
0,83
187,8
0,75
AIC
19,075
18,754
19,103
23
Modeling results for the best Kalman, Clarke and maximum likelihood
methods.
data
data
data
model
model
model
day
day
day
First conclusions:
The best results of modeling have been got with the Kalman model of sixth
order and with all three inputs considered.
The simplest Kalman model is the best but the other ones got with other
methods are in general not much worse when they are compared each other
quantitatively in view of the criteria values as well as qualitatively in view of
their diagrams.
On the second stage of modeling the neuronal nets of type MLP have been
used.
In this case also the models with only one input (WP or WL or R), with two
inputs (WP and WL or WP and R) and with three inputs (WP and WL and R)
have been tested.
Different time delays in the data series introduced onto the inputs have been
defined.
The numbers of the time delays are equivalent to the orders of difference
operators by the time series methods.
The output of the neuronal nets is one at all times and it means the raw
sewage inflow to the sewage treatment plant.
A neuronal net marked in the following as MLP/1/3/3-6-1 means the MLP net
with the delay (shift) in the data equal to 1 day, with 3 inputs, with 3 neurons
on the input layer, 6 neurons on the hidden layer and with 1 neuron on the
input layer.
Table 4. The best neuronal net models.
Model
MLP/1/1/1-2-1
MLP/2/1/2-3-1
MLP/3/1/3-3-1
MLP/1/2/2-3-1
MLP/2/2/4-3-1
MLP/2/2/4-4-1
MLP/4/3/12-5-1
Criterion R
learning
test
validation
1 input – daily water production WP
0,10
0,15
0,17
1 input – daily water level in the river WL
0,67
0,45
0,61
1 input – daily rainfalls R
0,49
0,59
0,53
2 inputs (without R)
0,76
0,73
0,78
2 inputs (without WL)
0,51
0,62
0,55
2 inputs (without WP)
0,68
0,46
0,64
3 inputs (WP and WL and R)
0,83
0,76
0,79
AIC
30,1
30,2
32,0
33,1
37,6
43,9
17,5
The best neuronal net model MLP/4/3/12-5-1.
data
model
day
Second conclusions:
The neuronal model with the full inputs set (MLP/4/3/12-5-1) turned out to be the
best.
In this model the measurements data in the input series are shifted of four days what
corresponds with the difference operator order equal to 4 in the time series methods.
On the third stage of modeling the fuzzy sets have been used.

In this case the models with with two inputs (WP and WL or WP and R)
and with three inputs (WP and WL and R) have been tested.

The models have the autoregressive structure like the time series and the
neuronal models.
Figure. The best neuronal net model MLP/4/3/125-1.
The fuzzy set model TSK-WP-R-WL for the learning and testing data.
The fuzzy set model TSK-WP-WL for the learning and testing data.
.
The fuzzy set model TSK-WP-R for the learning and testing data.
.
Table 5. The fuzzy set models.

Model
Model TSK-WP-WL-R
Model TSK-WP-WL
Model TSK-WP-R

Learning
Testing
Learning
Testing
4 290
4 006
4 784

4 034
5 253
5 827

0,671
0,626
0,748
0,631
0,821
0,911
Model
(Mittelwert)
(Mittelwert)
Model TSK-WP-WL-R
Model TSK-WP-WL
Model TSK-WP-R
4 162
4 630
5 306
0,65
0,72
0,83
MSE
x 105
177,7
223,8
303,5
R
(Mittelwert)
0,60
0,54
0,48
Third conclusions:

The neuronal model with the full inputs set (MLP/4/3/12-5-1) turned out to
be the best.

In this model the measurements data in the input series are shifted of four
days what corresponds with the difference operator order equal to 4 in the
time series methods.
IBS PAN Warszawa
End results
Table 7. Comparison of the best time series, neuronal net and fuzzy set models.
Model

K/6/WP-WL-R
MLP/4/3/12-5-1
0,47
0,63
Model
TSK-WP-WL-R

0,65
Evaluation criteria
R
0,87
0,79
AIC
18,52
17,50
R
0,60
36
The results of modeling the raw sewage inflow into a sewage treatment plant
have been presented.
The models are developed with the time series methods, the neuronal nets
and fuzzy set methods.
The results show that the simplest method of Kalman gets better models
than the other more complicated time series methods.
The Kalman method is also better than the more complex method of
neuronal nets and of fuzzy sets.
But the differences between the results of different methods are not essentially
big.
The sewage inflow models are meant for the forecast goals.
They are to be included into an information system improving the control of
the sewage treatment plant and the management of the communal
waterworks.
Thank you for your attention
38