Transcript Gts Ingestion

Different types of ingestion

Ingestion for replication
– Essential data
– Oriented replication and subscription

Added services
– Integration of the legacy databases
– Parsing of the GTS flow
• Global products
• Geo-localisation products
GTS Ingestion
Simdat Meeting 05-02-08
Added Services 1: Access to GTS legacy database

Advantages
– Not mandatory to parse and crack the GTS files
– Use of the legacy databases behind the switch (outside
vgisc area)
o Database of flat files in folders mapped on the short header TTAAiiGGggg
o Database of flat files in folders mapped on the hour or the minute

Disadvantages
– It’s impossible to have a dynamic metadata state (no
notification)
– Catalogue Synchronization and replication impossible
– static representation of GTS products, no standard GTS
database

Existing sofware in Meteo France
– Database of flat files in folders mapped on the short header TTAAiiGGggg
– FTP access (Jakarta Java Classes)
GTS Ingestion
Simdat Meeting 05-02-08
Other
DB
GTS
Database
GTS
Database
GTS
Switch
GTS
Database
NWP
DB
GTS
Collections
Metadata
DR
NWP
Metadata
CN
Remote
Switch
GTS Ingestion
Simdat Meeting 05-02-08
Remote
CN
Added Services 2: GTS database in the VGISC area

Advantages
• The Data Repository owns its GTS Data Bases (Data ,
Metadata)
• Dynamic metadata state (what is present in the database)
• Individual messages database
• The real-time push harvesting of metadata is possible
• The GISC parser waits and stores. It works on solicitation.

Disadvantages
– Parsing and cracking the GTS files
– Management of the Metadata -> what structure ?
– Hard work with strong knowledge on GTS messages
GTS Ingestion
Simdat Meeting 05-02-08
<<actor>>
GTS information
package '2 Flux GTS System' {1/5}
GTS_FileManager
manage
GTS_File
FluxGTS
GetFile
// Alphanumeric Text
Extension file : a or ua
1..*
Heading
Document
header
1
TTACode
T1
T2
A1
A2
II
CCCC
YY
GGgg
BBB
Bulletin
Collection
Grib
TextBrut
// Liste of product :
SA, SI, SM, FC, FT
(To be completed)
// Binary format
Extension file : b or ub
1..*
_collectionOfMsg
IndividualMessage
METAR
ID_OACI
_doc
SYNOP
ID_METEO
TAF
ID_OACI
GTS Ingestion
Simdat Meeting 05-02-08
BUFR
Experience: GTS anomalies


Collections
– Stations that do not belong to the country
– A few TTAA headers not compliant with the tables (Analyses
for example)
– Different collections with the same header, BBB integrated
(need rules to choose the good one)
Individual messages
– The same messages (same station, same type, same time)
are different in different collections
GTS Ingestion
Simdat Meeting 05-02-08
Other
DB
GTS
Database
GTS
Database
NWP
DB
GTS GISC
Parser
GTS
Switch
GTS
Database
DR
NWP
Metadata
GTS
Collections
Metadata
GTS
Messages
Metadata
CN
Remote
Switch
Remote
CN
GTS Ingestion
Simdat Meeting 05-02-08