Transcript MOME: An advanced measurement meta

MOME: An advanced
measurement meta-repository
IPS-MoMe Workshop, Warsaw, Poland
March 14, 2005
Felix Strohmeier
Authors:
Pedro A. Aranda Gutiérrez, Antal Bulanza, Marek Dabrowski,
Baiba Kaskina, Jürgen Quittek, Carsten Schmoll,
Felix Strohmeier, Attila Vidacs, Kardos Sandor Zsolt
MOME 001990
http://www.ist-mome.org
Outline
|
Motivation
|
MOME approach
|
Meta data models
|
Database implementation
|
Current status
|
Conclusion
2
Motivation: The need for measurement data
|
|
|
|
|
Developing new measurement and monitoring methods
Developing architectures of measurement and monitoring
systems
Developing algorithms for network and traffic control
functions supported by measurements
Developing and validating realistic traffic models
Validating the offered level of QoS
Easy access to the
repositories of raw
measurement data is
highly desirable
Easy access to
information about
available measurement
tools is highly desirable
The ultimate goal: „understanding the Internet”
4
The MOME approach
|
Collecting and disseminating information about known
measurement tools and available measurement data
contact
methods
availability
platform
algorithms
maturity
interfaces
protocols
metrics
BGP topology
data
...
Data Analysis
Raw data
repositories
data type
MOME
DATABASE
(Data)
capture
scenario
data
format
analysis
results
network
type
capture
platform
WWW-GUI
E2E QoS
parameters
Meta-data Extraction
Packet
captures
interoperability
Search Functions
INTEROPERABILITY
DATABASE
(Tools)
licensing
URI Link
Raw data
5
Meta-database paradigm (1)
|
„Data about data”
|
|
|
|
Documentation of measurement environment and scenario
Statistical information about the captured traffic
Link to the repository, where data is actually stored
„Data about tools”
|
|
|
Category
Measured metrics
Hardware and software platform
MOME assists researchers in finding and retrieving the
measurement data and tools that are most appropriate
for their purposes
6
Meta-database paradigm (2)
|
Distributed storage of measurement data
|
|
Centralised 'directory' functionality to access the measurement data
|
|
As much storage as provided by the sources of the measurements
Directory stores data about the remotely stored measurement data
Projects contributing measurements can keep total control on them
|
When they withdraw data from public access
|
|
|
|
They delete the entry in the MOME database
The MOME database itself detects when registered measurements have been
withdrawn from the Internet
In a centralised design, this control would have been lost
Examples
|
|
CAIDA's Internet Measurement Database (work in progress)
GRID technology based solar observatory
7
Definition of the database model
|
A database model suited for heterogeneous measurements
Different kinds of measurements
| Measurements produced by different tools
|
|
Meta-data model
|
|
Describe the actual data of interest
Trade-off: keep it generic – keep it specific
Flow traces
Packet traces
HTTP traces
Monitoring
and
Measurement
data
Web-based
repositories
QoS results
Routing data
8
The MOME meta-data model (1)
Packet traces
Raw data
9
The MOME meta-data model (2)
Flow traces
Raw data
10
The MOME meta-data model (3)
QoS measurements
Raw data
11
The MOME meta-data model (4)
Routing data
Raw data
12
The MOME meta-data model (5)
HTTP trace
Raw data
13
Data analysis
|
|
|
Additional information about the characteristics of captured traffic
Automatic analysis by tools integrated in MOME
Results stored in the meta-database
14
Tools database
|
E.g. different tool categories
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Packet capturing
Traffic flow measurement
Packet monitoring
Connection monitoring
Service monitoring
Application-level monitoring
Intrusion detection
Sniffing
Performance measurement
Connectivity checking
Route detection
Topology detection
Traffic visualisation
Traffic generation
15
Implementation and deployment of the
MOME system
WEBSERVER
GUI
USER
wwwdata
http
www.ist-mome.org
surveyor
DATABASE
mome
http/ftp
data
analysis
scp/sftp
fileserver
|
|
THIRD PARTIES
mysql
shell.ist-mome.org
„assoctiated“
measurement
sites
Public
repositories
http://www.ist-mome.org
http://www.ist-mome.org/database/
16
Current state of the MOME databases (1)
|
Descriptions of exemplary datasets from public repositories
|
|
|
Descriptions of packet-level traces collected continuously at
the „associated” measurement site in FH Salzburg
|
|
|
Proof of concept for definition of the MOME meta-data model
Covers most important measurement sites
Traces stored locally at the MOME repository
So far about 480 1-hour long traces collected
Analysis tools
|
|
|
Implemented framework, which allows for incorporating new
analysis tools
Currently allows for basic analysis of packet traces (libpcap)
Additional analysis tools planned to be added
17
Current state of the MOME databases (2)
|
Framework
|
|
|
|
Tools descriptions
|
|
|
57 MOME tools currently listed in database
Description (category, metrics, OS support) allows user
the selection of useful tools for his/her purpose
Features
|
|
|
|
|
Implemented framework, which allows user registration and login
browsing and search open for all users
write access (add/modify) only available to registered users
User comments for listed tools
Automatic URL checker
Search function
Database statistics
Further improvements on the way…
18
Conclusion
|
Database online:
|
|
|
http://www.ist-mome.org/database/
Unregistered access to all data
Free registration allows
|
|
|
|
Demo-PC available
|
|
|
|
Adding new tools
Adding new meta-data
Start result analyses on own data
Refreshment area
Visitors welcome
Open for Co-operation
Encourage Projects to share measurement data
19
Thank you!
Questions?
MOME 001990
http://www.ist-mome.org