Transcript Analog forecasting of ceiling and visibility using fuzzy sets

Intelligent integration for nowcasting
Selected slides from a talk given at the 38th Annual Congress
of the Canadian Meteorological and Oceanographic Society.
For the complete powerpoint file see:
A Fuzzy Logic-based Analog Forecasting System for Ceiling
and Visibility, 38th Annual Congress of the Canadian
Meteorological and Oceanographic Society, May 31-June 3,
2004, Edmonton, Alberta.
http://arxt39.cmc.ec.gc.ca/~armabha/papers_and_presentations
Future Role of Operational Meteorology
Scientific and systematic
forecast process
Partnership with technology
How
?
Intelligent Weather Systems (RAP/NCAR)
Weather
Radar
Nowcasts
RAP, Thunderstorm Auto-Nowcasting,
www.rap.ucar.edu/projects/nowcast
Sensor
Systems
Real-Time Data
Preprocessing
Human
Input
(> 15 min)
GUI
Real-Time
Data
Algorithms
Fuzzy Logic
Integration
Algorithm
Quality
Control
Data Assimilation
Mesoscale Model
1
Product
Generator
Model
Output
Algorithms
IWS Design
• Expert system development framework
• Applies existing knowledge, techniques and algorithms
• Achieves intelligent integration of all relevant, real-time data
Selective
Climatological
Input
• Supports rapid development of useful, maintainable operational applications
1. RAP, Intelligent Weather Systems, www.rap.ucar.edu/technology/iws/design.htm
User
Intelligent Weather Systems (RAP/NCAR)
Fuzzy logic integration algorithm
For example, a fuzzy rule for forecasting radiation fog:
1
Human input
2
 Decision
If sky clear and wind light and humidity high and humidity increasing
For example, choice of
Then chance of radiation fog is high
data and fcst technique
Fuzzy
Rule
Base
Satellite image
W1
low med hi
Wind speed
low
W2 med
hi
Humidity
Matrix of fuzzy
rules covers
space of
all predictors
Humidity trend
System can
run continuously
to give real-time,
smart forecast
quality control.
For details,
see examples.
3
Chance of
radiation fog
(qualitative description)
1. RAP, Intelligent Weather Systems, www.rap.ucar.edu/technology/iws/design.htm
2. Jim Murtha, 1995: Applications of fuzzy logic in operational meteorology, Scientific Services
and Professional Development Newsletter, Canadian Forces Weather Service, 42-54
3. Meteorological applications of fuzzy, http://chebucto.ca/Science/AIMET/applications
Operational Meteorology
A Scientific and Systematic Forecast Process:
a partnership with technology! 1
Technology
Observation
Sat, radar, awos…
Analyses
4DVAR, AI…
Diagnoses
RDP, AI…
Prognoses
GEM, EPS, UMOS…
Products/
Services
SCRIBE/AVIPADS, etc.
W
O
R
K
S
T
A
T
I
O
N
Meteorologist
Reports from public
Pattern recognition
Conceptual models
Science, experience,
training
Decisions
Performance
Measures
1. Jim Abraham, 2004: Science-Operations Connection workshop, Meteorological Service of Canada, Toronto, 24-26 February 2004.
“Smart Alert” Concept
Impending
problem
Bust
St. John’s
Fit
Loose
Tight
| | | | l| | | |l|
| | | | l| l
| | | |
| | | | l| | |l| |
| | l| | l| | | | |
| | l| | l| | | | |
…
| | | | l| l
| | | |
Wind
00h 1215
01h 1314
02h 1412
...
12h 1408
Ceiling
Visibility
Direction
Speed
Time
…
Weather
Weather
00h R-L01h R-L02h L...
12h L-
Search
Make
Save
Send
100+
60
30
25
20
15
10
9
8
7
6
5
4
3
2
1
21 22 23 0 1 2 3 4 5 6 7 8 9 10 11 12
AMD TAF CYYT 270010Z 270024
1315KT 2SM -RA BR OVC006
TEMPO 0002 1/2SM -DZ FG OVC003
FM0200Z 14010KT 1/2SM -DZ FG OVC002
TEMPO 0224 1/4SM -DZ FG OVC001
RMK NXT FCST BY 06Z=
DECISION SUPPORT SYSTEMS *
official forecast
Battleboard
!
actual trend
0
time
ACTUAL
WEATHER
MAP
(animated)
HEADS-UP
ALERT &
DISPLAY
GUI leverages
FORECASTER
forecaster’s actions
raises
forecaster’s
situational
awareness
(interacts, intervenes)
awareness and knowledge
Graphic intervention
First resort
GUIDANCE
DISPLAY
(satellite,
NWP, etc.)
MODELLED
WEATHER
MAP
(editable)
Direct intervention
Last resort
DSS
(interaction with
integration and
prediction)
PRODUCT
DISPLAY
(editable)
POSTPROCESSING
METAR
DA
NWP
data
data
RADAR
REAL-TIME
OBS
SATELLITE
PRODUCTS
MODEL-BASED
WEATHER
ELEMENTS
information
FORECAST
INTEGRATION
data
TRANSLATION
PRODUCT
GENERATION
UPPER AIR
EXTRAPOLATION
PROJECTED
OBS
CONSISTENCY
CHECKING
CLIMATE
ARCHIVE
AI
RAW, QC’d
WEATHER
knowledge
data
PREDICTION
PRODUCT
SPECIFICATIONS
MODELLED
WEATHER
data
data and information
• up-to-the-minute
intelligent data fusion
• abstract features
• derived fields
• intelligently composed
“interest fields”
USER
information
• special interests
• cost-based
decision-making
models
VERIFICATION
* Forecaster Workstation User Requirements Working Group meeting notes, 2000: Decision support systems
for weather forecasting based on modular design, updated slightly for Aviation Tools Workshop in 2003.
Decision Support Systems Design
Generic: no-name, conceptual design that could link and
integrate the most useful elements of WIND, AVISA, MultiAlert,
SCRIBE, FPA, URP, and so on in evolving WSP application, NinJo.
Modular: shows where distinct sub-tools / agents can be developed.
Working in this way, individual developers could work on isolated
sub-problems and anticipate how to plug their results into a larger
shared system. As technology inevitably improves, improved modules
can be easily installed and quickly implemented.
User-centered: forecast decision support systems from forecaster's
point of view, designed to increase situational awareness.
Hybrid: combines complementary sources of knowledge, forecasters
and AI, to increase the quality of input data and output information.
Intelligent integration of data, information, and model output, and
use of adaptive forecasting strategies are intrinsic in this design.
Hybrid Forecast Decision Support Systems
Hybrid forecast system development is a current direction of the Aviation Weather
Research Program (AWRP) 1 and the Research Applications Program (RAP), 2
NCAR (the main organizers of AWRP R&D).
“If a statistical / analog forecast disagrees with a model forecast, or if different
sensors disagree about how C&V are measured, what should we do about it?
Fuzzy logic could simulate how humans might apply confidence factors to
different pieces of information in different scenarios.” 3
AWRP Terminal Ceiling and Visibility Product Development Team (PDT) project,
Consensus Forecast System, a combination of:

COBEL, a physical column model 4

Statistical forecast models, local and regional

Satellite statistical forecast model
1. Aviation Weather Research Program, http://www.faa.gov/aua/awr
2. Research Applications Program, http://www.rap.ucar.edu
3. Norbert Driedger, 2004, personal communication.
4. Cobel, 1-D model, http://www.rap.ucar.edu/staff/tardif/COBEL
Hybrid Forecast Decision Support Systems
AWRP National Ceiling and Visibility PDT research initiatives: 1

Data fusion: intelligent integration of output of various models,
observational data, and forecaster input using fuzzy logic 2, 3

Data mining, C5.0 pattern recognition software for generating
decision trees based on data mining, freeware by Ross Quinlan
(http://www.rulequest.com), like CART

Analog forecasting using Euclidean distance development of
daily climatology for 1500+ continental US (CONUS) sites

Incorporate AutoNowcast of weather radar in 2004-2005 4

Incorporate satellite image cloud-type classification algorithms 5
1. Gerry Wiener, personal communication, July 2003.
2. Intelligent Weather Systems, RAP, NCAR, http://www.rap.ucar.edu/technology/iws
3. Shel Gerding and William Myers, 2003: Adaptive data fusion of meteorological forecast modules,
3rd Conference on Artificial Intelligence Applications to Environmental Science, AMS.
4. AutoNowcast, http://www.rap.ucar.edu/projects/nowcast
5. Tag, Paul M., Bankert, Richard L., Brody, L. Robin. 2000: An AVHRR Multiple CloudType Classification Package. Journal of Applied Meteorology: Vol. 39, No. 2, pp. 125-134.
Hybrid Forecast National
Decision
Support
C&V
ForecastSystems
System
Current
FY 04
RUC20
Persistence
Eta Model
Improved C&V
Translation
Obs-Based
Techniques
C & V values
derived from
forecast
hydrometeor
and humidity
fields.
Statically
carries forward
current C & V
conditions.
Augments RUC
in CONUS and
will support
subsequent
Alaska product
Experimental
use of data
mining for
improved
translation.
First trials of
forecasts from
historical data
using obs
inputs.
EXPERT SYSTEM-BASED FORECAST MERGE PROCESS
(Weighted Simple Additive Model)
COBEL
Column Model
Column model
Practical
with initial
forecast
focus on fog
methods from
and low cloud
operations for
targeted locale. in NE.
Rule-Based
Methods
Future
Others TBD.
Hybrids
Future
methods
focused on
C & V.
FORECAST
COMPONENT WEIGHTS
BASED ON
PERFORMANCE
DATABASE.
Display: NCV web, ADDS,
Cockpit, Other.
Forecast of Ceiling,
Visibility & Flight
Category on RUC Grid
DATABASE OF
FORECAST COMPONENT
PERFORMANCE VS
WEATHER CONDITION.
Feedback Loop
Using FY03-04 Mods
1. Herzegh, P. H., Bankert, R. L., Hansen, B. K., Tryhane, M., and Wiener, G., 2004: Recent progress
in the development of automated analysis and forecast products for ceiling and visibility conditions,
20th Conference on Interactive Information and Processing Systems, American Meteorological Society.
Fuzzy Logic at Research Applications Program, NCAR
According to Richard Wagoner, Deputy Director at Research Applications
Program (“Technology Transfer Program”), NCAR: 1
• NCAR / RAP is now a “continuous set theory” [fuzzy set theory]
development center.
• Over 90% of systems developed use fuzzy logic [FL] as the
intelligence integrator. [ … P.S. It is now 100% 2 ]
• [FL offers] unprecedented fidelity and accuracy in systems development.
• Automatic FL-based systems now compete with human forecasts.
1. Richard Wagoner, 2001: Background briefing on post processing data fusion technology at NCAR,
online presentation, http://www.rap.ucar.edu/general/press/presentations/wagoner_21feb2001.pdf
2. John K. Williams, 2004: Introduction to Fuzzy Logic as Used in the NCAR Research Applications Program,
Artificial Intelligence Methods in Atmospheric and Oceanic Sciences: Neural Networks, Fuzzy Logic, and
Genetic Algorithms, Short Course, American Meteorological Society, 10-11 January 2004, Seattle, WA.
ftp://ftp.rap.ucar.edu/pub/AMS_AI_ShortCourse/Williams_AMS_ShortCourse_11Jan2004.pdf
Fuzzy logic
Since we can assign numeric values to
linguistic expressions, it follows that we can
also combine such expressions into rules
and evaluate them mathematically.
A typical fuzzy logic rule might be:
If temperature is warm and pressure is low
then set heat to high
A graphical illustration to fuzzy logic, http://www.mcu.motsps.com/lit/tutor/fuzzy/fuzzy.html
How Rules Relate to a Control Surface
A fuzzy associative matrix (FAM) can be helpful to be sure you are not missing
any important rules in your system. Figure shows a FAM for a control system
with two inputs, each having three labels. Inside each box you write a label of
the system output. In this system there are nine possible rules corresponding
to the nine boxes in the FAM. The highlighted box corresponds to the rule:
If temperature is warm and pressure is low then set heat to high
A graphical illustration to fuzzy logic, http://www.mcu.motsps.com/lit/tutor/fuzzy/fuzzy.html
Three Dimensional Control Surface
The input to output
relationship is precise
and constant. Many
engineers were initially
unwilling to embrace
fuzzy logic because of a
misconception that the
results were not
repeatable and
approximate. The term
fuzzy actually refers to
the gradual transitions
at set boundaries from
false to true.
A graphical illustration to fuzzy logic, http://www.mcu.motsps.com/lit/tutor/fuzzy/fuzzy.html
Intelligent integration for nowcasting
For more information, see:
A Fuzzy Logic-based Analog Forecasting System for Ceiling
and Visibility, 38th Annual Congress of the Canadian
Meteorological and Oceanographic Society, May 31-June 3,
2004, Edmonton, Alberta.
http://arxt39.cmc.ec.gc.ca/~armabha/papers_and_presentations