Transcript OSD Metadata Management

OSD Metadata Management
Xianbo Zhang
10/14/02
Region Concept
Client
Metadata Server
osd
Client
...
IP Networks
osd
Client
osd
osd
osd
osd
Client
Region Concept
Region 1
Client
Metadata Server1
osd
Client
...
Region 2
IP Networks
osd
Region 3
Metadata Server2
Metadata Server3
Client
osd
osd
osd
osd
Client
Region Concept (Cont.)
• Interested networks partitioned into regions
• Each region has local metadata server, osd devices and
clients
• Each metadata server has information about osd
devices, objects, and client policies in this region. Also
each metadata server has direct or indirect way to
communicate with other servers.
• All metadata servers form a decentralized system, each
server has the same(or similar) capabilities and
responsibilities
Region Concept (Cont.)
With the services and information provided by
metadata system, object can migrate or replicate
itself to the right region to make most of the
data request/access happen in the same local
region, minimizing network latency, reducing
WAN traffic, and providing good locality,
scalability, high performance, high availability,
… Special algorithms/data structures need to be
developed to achieve these goals.
Functionality Requirements for Region based
Infrastructure
• Region self-organization: Manage dynamic region
division, merging based on specified criteria.
• osd device registration: through simple
configuration, the device can register with any
known metadata server and will be managed by
the right region.
• Client registration: client registers with any known
server, and will be migrated to the nearest server
automatically along with its capabilities
Functionality Requirements for Region based
Infrastructure (Cont.)
Floating among regions:
• Client: Client with its policy may move among regions.
• Object: Object may be migrated or replicated among
regions according to object access pattern, object attribute
requirements such as data location sensitivity.
• Device: Moving osd device from one region to another
region is transparent to client. The transparency may be
achieved through object migration/replication.
Functionality Requirements for Region based
Infrastructure (Cont.)
Fault-tolerance (avoiding single point of failure):
• Server: three levels of failover support, that is,
server local backup/mirroring, server remote
backup/mirroring, for most important data, erasure
code can be used among servers for data disaster
recovery.
• Object: object replication among osd devices,
among different regions
Functionality Requirements for Region based
Infrastructure (Cont.)
Data access pattern monitoring:
• Servers monitor data requests within region.
According to the access pattern, object may be
migrated, replicated or stripped among osd
devices to achieve good locality, load balancing
and high scalability.
• Load balance among metadata servers can be
achieved based on this same data access
monitoring.
Functionality Requirements for Region based
Infrastructure (Cont.)
Common services provided:
• Object location : Locate the nearby copy of
requested object upon authenticated client request.
• Message routing : overlay network built among
servers, message can be routed efficiently for
resource discovery and system management. Here
each server acts as server, client and router.
Functionality Requirements for Region based
Infrastructure (Cont.)
Common services provided (Cont.):
• Distributed lock services for metadata and objects.
• Object consistency service among osd devices
according to object’s unique consistency
requirements
• Automatic backup based on object attributes,
supporting remote backup
• Remote mirroring
Schematic Explanation of Data Access within
a Region
•After authentication and authorization, client directly access data
with capability from metadata server
IP Networks
Client
Metadata Server
Client
osd
osd
osd
Schematic Explanation of Data Access outside
a Region
•Distributed lookup protocol find nearby object copy via the
distributed system formed by metadata servers
•Direct data access to that osd device with required data
Metadata Server
IP Networks
Client
osd
osd
Metadata Server
Client
osd
osd
osd
Enterprise vs. Globe/Internet
• Enterprise has one consistent administration
policy. Each client of global storage system has its
own.
• Enterprise has trusted servers. Global storage
system has untrusted servers.
• Enterprise has limited size when comparing to
Internet, object location could be different from
popular peer-to-peer global systems.