Transcript Presentation

Data Transformation and
Uncertainty in Geostatistical
Combination of Radar and Rain
Gauges
CSI 991
Dan Basinger
Abstract
• Quantitative Precipitation Estimates (QPE) can
be performed using methods from
geostatistics.
• Estimates are currently made by combining
radar and rain gauge data.
• Problems arise, however, because of skewed
and heteroscedastic nature of rainfall.
• Trans-Gaussian kriging techniques have been
applied to correct for these problems.
Abstract (Contnued)
• Utilizing Kriging with External Drift (KED) in
concert with the Box-Cox Power Transform has
been applied to combined radar and rain
gauge data.
. Hourly precipitation data was collected in
several places in Switzerland
. KED measurements were performed with
and without transformation.
Abstract (Continued)
• The effect of the transformation was examined in
terms of accuracy of both point and probabilistic
estimates.
• As a result, data transformation via kriging
improved the analysis of the samples vis-à-vis the
model
• Problems persist, however, in the case of dry
gauges.
• In general, caution is emphasized in overutilizing
the transformation due to positive bias.
Introduction
• Real-time radar-gauge combinations have
been analyzed using different methodologies.
– Interpolation with deterministic weights
– Interpolation with splines
– Objective analysis
– Bayesian conditioning
– For this study, a stochastic interpolation algorithm
known as kriging is utilized.
Kriging for Geostatistical Analysis
• Variations of kriging have been used in the past for
radar-rainfall gauge combinations.
–
–
–
–
Basic kriging
Kriging with external drift (which is utilized in this study)
Cokriging
Indicator kriging
• The choice of these methods can vary depending on
the study scenario.
• Kriging with external drift (KED) has been selected for
this study.
• The spatial covariance is derived from the data itself.
Kriging for Geostatistical Analysis
(Continued)
• Kriging applied to radar-gauge combinations
has resulted in improved point estimates.
• Probabilistic precipitation estimates have
been generated in the form of a probability
density function (PDF).
– These probabilistic estimates drive the stochastic
simulation of QPE estimates.
– These in turn would find application in future
hydrological forecasts.
Kriging for Geostatistical Analysis
(Continued)
• The problem with geostatistical radar-gauge
combinations is the fact that the underlying
stochastic structure is based on the Gaussian
distribution and the stationarity of:
– Mean
– Spatial covariance
• Rainfall is scattered in terms of wet and dry
regions and thus, results in a skewed distribution.
• The Box-Cox Power Transform has been added to
correct for problems related to this skewness.
Trans-Gaussian Kriging with the
Box-Cox Transform
• Kriging with a power transform such as the Box-Cox
method is called trans-Gaussian kriging.
• Similar applications include:
– Mapping of hake distributions
– Mapping of lead magnitudes in soils
• For radar rainfall gauge combinations throughout
Switzerland, trans-Gaussian kriging is utilized to account
for the non-Gaussian nature of the data.
• Kriging with external drift (KED) techniques are performed
on the data and compared with results when no transform
is applied.
• Hourly rainfall measurements from radar and rain gauges
are utilized in this study.
Combination of Radar and Rainfall
Gauges
• Analysis methods are based on the concept that
precipitation data are perceived as a singular
realization of a multivariate random variable.
• This variable is assumed to have a Gaussian
distribution.
– Mean also known as the drift of trend [deterministic]
– Covariance function which describes the spatial
dependence between the residuals [random]
• This distribution for precipitation generates a
probabilistic estimate of the variable at any
location.
Combination of Radar and Rainfall
Gauges (Continued)
• The kriging process computes the expected
value also known as the point estimate as
well as the variance which is the prediction
uncertainty within the distribution,
• Kriging can produce maps of the following:
– Point estimates
– Quartiles of the probabilistic estimates
– Uncertainty estimates
– Simulated realizations based on the observations
Combination of Radar and Rainfall
Gauges (Continued)
• On this study, kriging with external drift (KED) has been
selected.
• Mathematically, we have:
• Where Prec is precipitation with i,j being indices, alpha and
beta are the trend parameters, the intercept and radar
coefficient, and Z is a random process modeling the
deviations of precipitation from the scaled radar field
(Page 1334)
Combination of Radar and Rainfall
Gauges (Continued)
• The data is collected at 75 gauge stations
throughout Switzerland.
• Most of the time, a large fraction of these
stations report dry conditions.
• The method of Restricted Maximum
Likelihood (REML) is utilized to estimate trend
and covariance parameters in the context of a
relatively limited sample size.
Combination of Radar and Rainfall
Gauges (Continued)
• Every data pair (radar and rainfall gauge) is
examined individually.
• The radar- rainfall gauge matching is
performed using the Nearest Neighbor
Algorithm.
– The radar value at a particular gauge location is
derived from the radar pixel (1 km X 1 km).
– This experiment is performed in such a way so as
to decrease inconsistencies in spatial sampling
between radar and rainfall gauges.
Application of Kriging with External
Drift (KED)
• KED presupposes the random-process stochastic
Z (i,j) to be Gaussian and stationary.
– Constant Mean
– Constant Variance
– Constant Covariance
• As it turns out, the deviations from the radarrainfall gauge relation are not Gaussian, but
rather, positively skewed.
– Precipitation values are truncated to 0
– Variance of the deviations is not constant, but
increase with the precipitation values.
Application of Trans-Gaussian Kriging
• The algorithm utilized to correct for the
deviations is trans-Gaussian kriging.
– The method is used to transform the variable along
with the covariates with a monotonic function.
– The requirements for the Gaussian distribution are
better fulfilled.
• Kriging is then performed on these transformed
values.
• A back transformation is subsequently
performed on the predictions resulting from the
kriging process.
The Box-Cox Power Transform
• A useful set of transformations is the Box-Cox
Power Transform.
• It is applicable to nonnegative data and has the
format:
Where Y and Y* are the original and transformed
variables and λ is the transformation parameter.
(Page 1334)
The Box-Cox Power Transform
(Continued)
• Values of λ > 0 and λ < 1 are useful in correction for
positive skewness.
• The case in which λ = 0 is not useful because of the
preponderance of dry conditions.
• For positive values of λ, the minimum for Y* is
• -1/λ.
• For trans-Gaussian KED, this Box-Cox Power Transform
is applied to both radar and rainfall gauge data.
• This is intended to correct for the skewness within both
variables.
The Box-Cox Power Transform
(Continued)
• The same value for λ is used for both sets of variables.
• Application of KED on the transformed data generates
a probabilistic estimate at each location.
– This yields a genuine Gaussian distribution with a mean
value given by the point estimate.
– This also yields a new variance corresponding to the
kriging variance.
• The new Gaussian distribution is subsequently
subjected to the Inverse Box-Cox Transform in order to
obtain the probabilistic estimates from the original
untransformed data.
The Box-Cox Power Transform
(Continued)
• The point estimate for the precipitation values
is approximated by the mean of the backtransformed quantiles.
• In the case of the lowest of the quantiles, the
transformed values are set to -1/λ.
• Physically, these correspond to the dry
precipitation cases.
• These quantiles are subsequently set to 0.
The Box-Cox Power Transform
(Continued)
• The KED process is performed with four settings
of L in the Box-Cox Transform.
– If λ = 1, we have the classical application without
transformation.
– If λ = 0.5, we have the square root transformation
which has been determined to be very suitable for
radar-rainfall gauge combinations.
– If λ = 0.1, we have an expression which is very close to
a logarithmic transformation and not very suitable for
radar-rainfall gauge combinations.
– In a fourth case, we set λ = λ which is estimated
case by case individually.
e
The Box-Cox Power Transform
(Continued)
• The fourth case deals with an estimate that
turns out to work out very well to hourly
precipitation values.
• There are several criteria that drive the
selection of a value for case-dependent λ :
– Gaussian distribution of residuals
e
– Relative linearity of the radar and rainfall gauge
relationship.
The Box-Cox Power Transform
(Continued)
• Applications of the Box-Cox transform over
the course of a year have yielded values of 0.2
to 0.53 for λ .
e
• Estimates of λ vary only slightly between
seasons.
e
• Values of λ vary more distinctly with rainfall
variances e
The Box-Cox Power Transform
(Continued)
• Large values of λ (> 0.3) usually occur in hours
e
of even and widespread rainfall.
Small values of λ ( < 0.22) usually occur in
e
relatively limited and/or variable precipitation
patterns.
All general calculations are performed in “R” and the Box-Cox
algorithm is performed in the “geoR” package.
Evaluation of the Methodology
• Several parameters are calculated in this study to
access the quality of the point estimate.
• The ratio of total estimated to measured water
amounts (RTW) measures the overall bias.
• The parameter called SCATTER is a measure of
the error spread at wet locations.
• The Hansen-Kuipers Discriminant (HK) is utilized
to measure the ability to differentiate between
wet and dry areas.
Evaluation of the Methodology
(Continued)
• The gauge measurements are compared with the
probability density function applying the concept
of a probability integral transform.
• We then calculate the frequency of observations
that fall in between predefined quantiles.
• The Ratio of Observed to Expected frequencies
based on interquantile bins measures the
reliability of the kriging algorithm.
Evaluation of the Methodology
(Continued)
• The Stable Equitable Error in Probability Space (SEEPS) is a
triple category contingency value measuring the
correspondence of dry, light, and intense rainfall
conditions.
• Light rainfall is defined as the lower 67 %
• Intense rainfall is defined as the upper 33 %.
• The mean root transformed error is calculated as:
n
2
MRTE= (1/n)Sum (SQRT(pred)–SQRT(obs) )
(Page 1336)
i=1
i
i
where “pred” refers to the point estimate
“obs” refers to the gauge measurement.
Data Sources
• The source of the data for the trans-Gaussian
radar and rainfall gauge measurements is hourly
rainfall over the whole of Switzerland.
• A radar composite of hourly accumulated rainfall
(in mm) covering the whole of Switzerland is
placed on a grid space of 1 km X 1km.
• Rainfall gauge data is based on hourly
accumulated rainfall with stations distributed
evenly over the country.
Results
• Trans-Gaussian kriging has been utilized in the
analysis of combined radar and rainfall gauge
data.
• This was based on the hypothesis that data
transformation can correct for the non-Gaussian
nature of the data distributions.
• Kriging with external drift (KED) in conjunction
with the Box-Cox Power Transform was utilized
to modify the data distribution.
Results (Continued)
• Many improvements were observed when KED was
utilized to generate a proper transformation as
compared with the untransformed data.
• The underlying assumption of a Gaussian residual
distribution along with relative stationarity of residual
variance was better satisfied.
• Transformation changes the probabilistic estimate into
a positively skewed probability distribution function.
• This properly scales the precipitation quantities
correcting for unrealistic dependencies from KED
without the Box-Cox Transform.
Results (Continued)
• KED without the Box-Cox Transform yielded rainfall
distributions very similar to those from a parameter with
exponents between 0.2 and 0.5.
• External transformations close to the logarithmic function
(Box-Cox parameters less than 0.2) tended to produce a
positive bias.
• Thus, in the overall analysis, it is important to avoid
transformations close to the logarithmic.
• Other problems that the transformation has yet to rectify:
– Cases with a high number of zeroes
– Cases involving heteroscedasticity
• Cases in which the square root transform was derived
produced good results.
Results (Continued)
• Overall, the best reliability for the probabilistic
estimate was found in those case-dependent
values of the transformation parameter λ.
• These resulted in optimization of Gaussian
residuals.
• Case-by-case estimates can be generated on
alternative criteria such as the linearity of the
radar rainfall gauge relationship.
Conclusions
• Utilizing data transformations in the context of
geostatistical applications is one way of analyzing
the complex nature of rainfall data.
• Also there are currently limitations to this
approach.
• These limitations can be potentially overcome by
extending the trans-Gaussian kriging process by
introducing additional terms such as:
– Radar Quality
– Topography
– Precipitation Patterns and Structure
Selected References
• Box, G. E. P. and Cox, D.R. 1964: “An Analysis of
Transformations” J. Roy. Stat. Soc. 26A: 211-252
• Diggle, P.J. and Ribeiro, P.J. 2007 Model-Based
Geostatistics
--------------------------------Springer-Verlag
• Haberland, U. 2007 “Geostatistical Interpolation
of Hourly Precipitation from Rain Gauges and
Radar from a Large-Scale Extreme Rainfall Event”
J. Hydrology 332: 144-157
Selected References (Continued)
• Hoel, P.G., Port, S.C., Stone, C.J. 1971
Introduction to Probability Theory Houghton Mifflin
---------------------------------------------
Schabenberger, O. and Gotway, C.A. 2005
Statistical Methods for Spatial Data Analysis
---------------------------------------------------------Chapman and Hall/CRC
Selected References (Continued)
Schneider, S. and Steinacker, R. 2009 “Utilization
of Radar Information to Refine Precipitation Fields
by a Variational Approach”
Journal of Meteor.Atmos. Physics 103 137-144
Sinclair, S. and Pegram, G. 2005
“Combining Radar and Rain Gauge Rainfall
Estimates Using Conditional Merging”
Atmospheric Science Letters 6: 19-22