Transcript resource - Distributed Object Computing

Patterns for Effective Management
of Resources in Software Systems
Presented by Douglas C. Schmidt
Based on the POSA3 book by
Michael Kircher
Prashant Jain
Reproduced with permission from
Pattern-Oriented Software Architecture, Volume 3: Patterns for Resource Management,
Michael Kircher & Prashant Jain.
© 2004 John Wiley & Sons Ltd.
Patterns for Effective Management of Resources in Software Systems
1
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Agenda
• Introduction
• Resource Acquisition
• Resource Lifecycle
• Resource Release
• Case Studies
• Summary
Patterns for Effective Management of Resources in Software Systems
2
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
What is Resource Management?
Resource management in software systems is the process of
controlling the availability of resources to resource users.
Resource management ensures that resources are available when
needed, that their lifecycle is deterministic, & that they are released in a
timely manner to ensure the liveliness of the systems that use them.
• A resource is an entity that is available in limited supply
• e.g., memory, CPU, network bandwidth, etc.
• A resource user is an entity that acquires, accesses, or releases
resources
• e.g., an application or system program
• A resource provider is an entity that provides the resources on
request
• e.g., a virtual memory manager, thread scheduler, network
interface, etc.
Patterns for Effective Management of Resources in Software Systems
3
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Categorization
• By resource lifecycle
• Reusable: Resources once released can be acquired & used again
o Examples: Memory, threads, file handles
• Non-reusable: Resources are consumed, & therefore once acquired are
either not released, or their release is implicit
o Example: Processing time in a computing grid
• By resource usage
• Exclusive: Resources that can only be used by a single user at a time
• Example: Processing time of a service
• Concurrent: Resources that can be concurrently accessed by multiple
users
• Examples: Queues & databases
Reusable
Non-reusable
Exclusive
Memory
Processing-time
Concurrent
Read-only object
Patterns for Effective Management of Resources in Software Systems
4
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Forces behind Effective Resource Management (1 of 2)
• Performance
• High-performance systems require low latency & high throughput.
• Therefore: Unnecessary resource acquisitions, releases, & accesses that incur
processing overhead & delay must be avoided.
• Scalability
• Large & complex systems may require support for more users or higher
transfer volumes.
• Therefore: The way resources are managed & the way their lifecycle is
managed must be carefully controlled.
• Predictability
• In systems with real-time requirements, the maximum response time of
individual operations must be predictable.
• Therefore: Acquisition & release times must not vary.
Patterns for Effective Management of Resources in Software Systems
5
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Forces behind Effective Resource Management (2 of 2)
• Flexibility
• Ease of configuration & customization is a common requirement among
systems.
• Therefore: The mechanics of acquisition, release & lifecycle control of
resources need to be flexible, while still preserving performance, reliability, &
scalability requirements.
• Stability
• An important requirement of software systems is that frequent resource
acquisition & release should not make the system unstable.
• Therefore: Resource allocation & management must be done in a manner that
leads to efficient use of resources & avoids scenarios of resource starvation
that can lead to system instability.
• Consistency
• All resource acquisition, access, & release must leave a software system in a
consistent state.
• Therefore: Any inter-dependencies among resources must be well-managed to
ensure system consistency.
Patterns for Effective Management of Resources in Software Systems
6
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
What are Design Patterns?
A design pattern describes a solution
for a recurring problem that arises in a
particular context
• Patterns facilitate reuse of successful
software architectures & designs
• Patterns identify & specify abstractions
that are above the level of single
classes & instances
• Patterns capture the static &
dynamic structure & collaboration
among key participants in software
design
• Patterns provide a common vocabulary
& understanding for design principles
Patterns for Effective Management of Resources in Software Systems
7
AbstractService
service
Client
Proxy
service
Service
1
1
service
The Proxy Pattern
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
The Three Main Ingredients
• Context: a situation giving rise to a problem
• Describes when it makes sense to apply the
pattern
• Problem: the recurring problem arising in the
context
• Identifies the forces that must be resolved &
identifies any constraints that must be
considered
• Solution: a proven resolution of the problem
• Provides a means to balance most (if not all)
of the forces
• Describes the key steps required to address
the problem
Patterns for Effective Management of Resources in Software Systems
8
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
The Three Main Ingredients Applied to
Proxy
• Context: when developing a
distributed application
• Problem: need to alleviate
tedious & error-prone
differences in byte-orders &
alignment constraints across
heterogeneous platforms
AbstractService
service
Client
Proxy
service
• Solution: define a proxy that
performs (de)marshaling &
shields applications from
network programming details
Patterns for Effective Management of Resources in Software Systems
Service
1
1
service
The Proxy Pattern
9
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pattern-oriented Effective Resource Management
Caching
Eager
Acquisition
Performance
Predictability
Pooling
Scalability
Leasing
Stability
Evictor
Coordinator
Resource
Lifecycle
Manager
Lookup
Lazy
Acquisition
Consistency
Flexibility
Partial
Acquisition
Patterns for Effective Management of Resources in Software Systems
10
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Patterns for Resource Acquisition
“I find that a great part of the information I have
was acquired by looking up something and
finding something else on the way.”
Franklin P. Adams
Patterns for Effective Management of Resources in Software Systems
11
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lookup – Example
• A system consisting of multiple distributed services
• To access a service, client needs a reference to the object that provides
that service
• For example, a client would need a reference of the Transaction Manager
to participate in distributed transactions
How can a client obtain the initial reference to an object that the
client wants to access?
Client
get
object
reference
publish
object
reference
Transaction
Manager
Network
Patterns for Effective Management of Resources in Software Systems
12
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lookup
Context
• Systems where resource users need to find & access local & distributed resources
Problem
• How can a resource user find out on demand what resources are available in its
environment?
• How can a resource user obtain an initial reference to a resource provider that offers
the resource?
Solution
• Provide a lookup service that allows resource
providers to make resources available to
resource users
• The resource provider advertises resources via
the lookup service along with properties that
describe the resources that the resource
providers provide
• Allow resource users to first find the advertised
resources using the properties, then retrieve the
resources, & finally use the resources
Patterns for Effective Management of Resources in Software Systems
13
Lookup Service
Resource Provider
Resource User
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lookup – Example Resolved
• The server that creates the Transaction Manager should register the
reference of the created Transaction Manager with the Lookup Service.
• The server is therefore the resource provider.
• In CORBA, the Lookup Service is implemented by the Naming Service.
• A client can obtain the object reference of the Transaction Manager from
the Lookup Service.
: Resource User
: Resource
Provider
: Lookup Service
: Resource
lookup (properties)
reference
acquire
access
Patterns for Effective Management of Resources in Software Systems
14
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Known Uses: Lookup
• CORBA
• The Common Object Services Naming Service & Trading Service implement lookup
services.
• Jini
• Jini services are registered with Jini’s lookup service, & these services are accessed
by users using Jini’s discovery protocol.
• COM+
• The Windows Registry is a lookup service that allows resource users to retrieve
registered components based on keys.
• Universal Description, Discovery, & Integration protocol (UDDI)
• UDDI allows publishers of Web Services to advertise their service & clients to search
for a matching Web Service.
• Grid Computing
• Grid computing uses the Lookup pattern to find distributed resources.
• Eclipse plug-in registry
• The plug-in registry holds a list of all discovered plug-ins, extension points, &
extensions & allows clients to locate these by their identity.
Patterns for Effective Management of Resources in Software Systems
15
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lazy Acquisition – Example
• Picture Archiving & Communication System (PACS) that provides storage
of patient data
• Data includes patient details, such as address information, as well as
digitized images & is represented by business objects
• For a patient with a long medical history & several digitized images,
fetching all the data & creating the corresponding business object(s) can
take a lot of time
How can the PACS system be
designed so that retrieval of patient
information is quick regardless of the
number of images in the patient’s
record?
PACS Client
Patterns for Effective Management of Resources in Software Systems
16
PACS Server
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lazy Acquisition
Context
• A system with restricted resources that must satisfy high demands, such as
throughput & availability
Problem
• Reduction of the initial cost of acquiring the resources
• If systems were to acquire all resources up front, a lot of overhead would be
incurred & a lot of resources would be consumed unnecessarily
Solution
• Acquire resources at the latest possible time
• The resource is not acquired until it becomes unavoidable to do so
• Introduce a resource proxy that hides the lazy acquisition from
the resource user
Resource User
Patterns for Effective Management of Resources in Software Systems
Resource Proxy
17
Resource Provider
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lazy Acquisition – Example Resolved
• Do not fetch any image data, when fetching information for a
particular patient
• Create an image proxy for each image in the patient record returned
• When an image is actually viewed, it is fetched (lazily) from the
image store
• Specializations:
• Lazy Instantiation
• Lazy Loading
• Lazy State
: Resource User
: Resource
Provider
: Resource Proxy
: Resource
access
acquire
resource
access
Patterns for Effective Management of Resources in Software Systems
18
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Lazy Acquisition – Known Uses
• Singletons
• Objects that exist uniquely in a system, are usually instantiated using Lazy Instantiation
• J2EE
• EJB containers use Lazy Loading & Lazy Instantiation to ensure that only components
that are actually used by clients are active
• Java Server Pages (JSPs) are typically compiled into servlets only when they are
actually accessed, rather than when they are deployed
• .NET Remoting
• Singleton remote objects are instantiated only on first access, even though clients
might obtain references to them before the first access
• JIT compilation
• The compilation of the regular Java byte code into fast machine-specific assembler
code is done just-in-time
• Eclipse plug-in
• Actual logic of a plug-in, which is contained in Java archives (JAR), is loaded lazily on
first use of any of the plug-in’s functionality
Patterns for Effective Management of Resources in Software Systems
19
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Eager Acquisition – Example
• Embedded telecommunication application with soft real-time constraints,
such as predictability & low latency in execution of operations
• In most operating systems, operations such as dynamic memory
allocation can be very expensive
• Memory allocations are protected by synchronization primitives
• Memory management, such as compaction of smaller memory segments
to larger ones, consumes time
How can you achieve predictable &
fast memory allocations via
operations such as new() or
malloc()?
Patterns for Effective Management of Resources in Software Systems
Telecommunication
switch
CPU
board
20
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Eager Acquisition
Context
• A system that must satisfy high predictability & performance in resource acquisition
time
Problem
• Expensive resource acquisitions, such as dynamic acquisition of threads & memory,
can result in unpredictable time overheads
• How can resources be acquired in systems with soft real-time constraints while still
fulfilling the constraints?
Solution
• Eagerly acquire a number of resources before their actual use
• At a time before resource use, optimally
at start-up, the resources are acquired
• The resources are then kept
in an efficient container
Provider Proxy
Resource
Resource User
Patterns for Effective Management of Resources in Software Systems
Resource Provider
21
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Eager Acquisition – Example Resolved
• Implement a simple memory pool for fixed-size blocks of memory without
synchronization to be used by a single thread
• Eagerly acquire the memory for the pool
• Provide separate memory pools for different ranges of memory size
• Solution allows for a ’natural’ & flexible control flow, while still ensuring
predictability
class Fixed_Size_Memory_Pool {
public:
Fixed_Size_Memory_Pool (size_t max_blocks) {
// ... Eagerly allocate free list blocks ...
}
void *acquire (size_t size) {
// ...
void *acquired_block = free_list_.front ();
free_list_.pop_front ();
return acquired_block;
}
void release (void *block) {
free_list_.push_back (block);
}
};
Patterns for Effective Management of Resources in Software Systems
22
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Eager Acquisition – Known Uses
• Ahead-of-time compilation
• Ahead-of-time compilation is commonly used by Java virtual machines to avoid
compilation overheads during execution
• Pooling
• Connection or thread pools, typically pre-acquire a number of resources, such as
network connections, or threads, to serve initial requests quickly
• Application servers
• An element called <load-on-startup> can be specified for a servlet in the
deployment descriptor, causing the container to load that servlet at start-up.
• Eclipse plug-in
• Plug-in declarations that determine the visualization are loaded eagerly
• The actual logic, which is contained in Java archives (JAR), is loaded lazily on first use
of any of the plug-in’s functionality
• Hamster
• A hamster acquires as many fruits as possible before eating them in its burrow
Patterns for Effective Management of Resources in Software Systems
23
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Partial Acquisition - Example
• Network Management System (NMS) that manages several network
elements (NEs)
• NMS allows user to get details of one or more NEs
• Details of an NE may correspond to a large amount of data
• The details of all the NEs can be fetched at system start-up or recovery
GUI
Since obtaining all the details of the
network elements can have a big impact
on the time it takes for the application to
start up or recover, how do we make the
application start-up or recovery time more
efficient & predictable?
Network
Management
System
NE
NE
NE
Shelf
Shelf
Shelf
Card
Patterns for Effective Management of Resources in Software Systems
24
Card
Card
Card
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Partial Acquisition
Context
• Systems that need to acquire resources efficiently. The resources are characterized
by either large or unknown size
Problem
• In highly robust & scalable systems, how can it be ensured that resources are
acquired efficiently?
• How can acquisition of resources be influenced by parameters such as available
system memory, CPU load, & availability of other resources?
Solution
• Split the acquisition of a resource into two or more stages.
Resource Provider
• In each stage, acquire part of the resource
• The amount of resources to acquire at each stage
should be configured using one or more strategies
Resource User
• Eager Acquisition & Lazy Acquisition can be used to
determine when to execute one or more stages to
partially acquire a resource
Resource
Patterns for Effective Management of Resources in Software Systems
25
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Partial Acquisition – Example Resolved
• Acquisition of the details of the NEs, along with their components, can be
split into multiple stages
• In the initial stage only the details of the NEs are fetched
• In subsequent stages, the details of the components of the NE are fetched
• The details can be fetched in the ‘background’ using Active Object pattern
• Partial acquisition of the details of an NE can also trigger the retrieval of
the neighboring NEs
User Interface
:TopologyManager
Network Element A :
NetworkElement
: Scheduler
select "NE A"
get details for NE ("NE A")
«create»
FetchNEComponents :
Request
insert
NE details
can run
call
get shelves
shelves details
shelves details
get cards
select "shelf 1"
cards details
cards details
Patterns for Effective Management of Resources in Software Systems
26
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Partial Acquisition – Known Uses
• Incremental image loading in web browsers
• Browsers first download the text content of a Web page
• The images are downloaded incrementally, during which time the user can read the
text content of the page
• Socket input
• Multiple read operations are performed at a socket, where each read operation
partially acquires data from the socket & stores it in a buffer
• Audio/Video streaming
• When decoding audio & video streams the streams are acquired in parts
• Data-driven protocol-compliant applications
• An application handling CORBA IIOP requests typically reads the IIOP header in the
first step to determine the size of the request body
• It then reads the contents of the body in one or more steps
Patterns for Effective Management of Resources in Software Systems
27
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Patterns for Resource Lifecycle
“Seek not, my soul, the life of the immortals;
but enjoy to the full the resources
that are within thy reach.”
Pindar
Patterns for Effective Management of Resources in Software Systems
28
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Caching – Example
• Network management system that needs to monitor the state of many
network elements
• Establishing a new network connection for every network element
selected by the user & destroying it after use incurs overhead in the form
of CPU cycles
• The average time to access a network element can be too large
How can the average time to
access a network element be
optimized, while also reducing
the number of network
connections that are created &
destroyed?
Netw ork
Management
System
connections
Patterns for Effective Management of Resources in Software Systems
29
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Caching
Context
• Systems that repeatedly access the same set of resources & need to optimize for
performance
Problem
• Repetitious acquisition, initialization, & release of the same resource causes
overhead
• How can you reduce the acquisition, access, & release cost of frequently used
resources to improve performance?
Solution
• Temporarily store the resource in a fast-access buffer called a cache
• Use the cache to fetch & return the resource instead of re-acquiring it
• The cache identifies resources by their identity, such
as pointer, reference, or primary key
Resource Provider
Resource Cache
Resource
Resource User
Patterns for Effective Management of Resources in Software Systems
30
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Caching – Example Resolved
• Implement a cache of connections of network elements
• On user access of a specific network element, the connection to the
network element is acquired
• When the connection is no longer needed, it is not destroyed but is
added to the cache
• Later, when a new request for the connection arrives, the connection is
acquired from the cache, thus avoiding high acquisition cost
• Variants
• Transparent Cache
: Resource
Cache
: Resource User
: Resource
Provider
: Resource
acquire
• Read-Ahead Cache
resource
• Cached Pool
access
release (resource)
acquire
resource
Patterns for Effective Management of Resources in Software Systems
31
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Caching – Known Uses
• Data transfer object
• An object that is transferred by value in middleware technologies such as CORBA &
Java RMI is held locally in a cache & represents the actual remote object
• Web proxy
• A Web proxy is a proxy server located between a Web browser & Web servers
• The Web proxy caches a Web page that is fetched by a browser from a Web server,
& returns it when a subsequent request is sent for the same page
• .NET Data Sets
• .NET data sets are instantiated locally & filled by the results of an SQL query on the
database with one or more tables, using an SqlDataAdapter
• The data sets therefore provide caching of data, thus saving expensive database
queries
• Enterprise JavaBeans (EJB)
• EJB entity beans represent database information in the middle tier
• The entity beans therefore help avoid expensive data retrieval (resource acquisition)
from the database
Patterns for Effective Management of Resources in Software Systems
32
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pooling – Example
• Simple Web-based on-line commerce application providing a catalog
• Contents of catalog dynamically generated from database
• Servlet connects to database, executes a query & uses results to generate
HTML page
• Catalog can be accessed by hundreds of users simultaneously
How do we ensure that a reasonable number of database connections
are created & furthermore a database connection is available quickly
to service a user request?
Servlet A
Servlet B
connections
Database
Servlet Engine
Patterns for Effective Management of Resources in Software Systems
33
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pooling
Context
• Systems that continuously acquire & release resources of the same or similar type, &
have to fulfill high scalability & efficiency demands
Problem
• How can released resources be reused to avoid the overhead of re-acquisition?
• How can the acquisition time of a resource by a user be made predictable even
though direct acquisition of the resource from the resource provider can be
unpredictable?
Resource User
Solution
• Manage multiple instances of one type of
resource in a pool. This pool of resources allows
for reuse when resource users release
resources they no longer need
• The released resources are then put back into
the pool
• When a resource is released & put back into the
pool, it should be made to lose its identity
Patterns for Effective Management of Resources in Software Systems
34
Resource Pool
Resource Provider
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pooling – Example Resolved
• Database connections are pooled in a connection pool
• On every request by the user, the servlet acquires a connection from the
connection pool, which it uses to access the database
• After access, the connection is released to the connection pool
:Resource User
:Resource
Provider
:Resource Pool
acquire
acquire
«create»
:Resource
resource
resource
access
release (resource)
acquire
resource
access
release (resource)
release
«destroy»
Patterns for Effective Management of Resources in Software Systems
35
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pooling – Known Uses
• JDBC Connection Pooling
• Java Database Connectivity (JDBC) connections are managed using connection
pooling
• COM+
• The COM+ Services runtime, as part of the Windows operating system, supports
object pooling for COM+ objects & .NET applications
• EJB Application Servers
• Component-based Application servers use component instance pooling to manage
the number of component instances efficiently
• Web Servers
• Most Web servers use Thread Pooling to efficiently manage the threads
• Threads are reused after completing a request
Patterns for Effective Management of Resources in Software Systems
36
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Coordinator – Example
• Large-scale system distributed over multiple processing nodes
• Each node comprises of services that need to be frequently updated
• Successful update of some sub-systems but not all can leave the system
in inconsistent state
How do we ensure that either all the subsystems are successfully
updated or none are?
Sub-System-1
Sub-System-2
Sub-System-n
update
Central Admin
Component
Patterns for Effective Management of Resources in Software Systems
37
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Coordinator
Context
• Systems where a task needs to be coordinated among multiple participants
Problem
• How do we ensure that in a task involving two or more participants, either the work
done by all of the participants is completed or none is?
Solution
• Introduce a Coordinator that is responsible for
execution & completion of a task
• Split the work of each participant into two
phases: prepare & commit
• In first phase, Coordinator asks each participant
to prepare for the work to be done
• If prepare phase is successful, Coordinator
initiates commit phase, in which the actual work
is done by the participants
• If prepare phase fails, Coordinator aborts the
task
Patterns for Effective Management of Resources in Software Systems
38
Coordination
prepare()
commit()
abort()
Coordinator
beginTask()
register()
commitTask()
Participant
1
*
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Coordinator – Example Resolved
• Each sub-system acts as a participant & implements the Coordination
interface
• The task to be completed is software update
• Coordinator executes the two-phase execution of the task
: Client
: Coordinator
: Sub-System 1
: Sub-System 2
begin task
register
register
commit task
prepare
true
prepare
false
abort
Patterns for Effective Management of Resources in Software Systems
39
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Coordinator – Known Uses
• Java Authentication & Authorization Service (JAAS)
• Login Context serves the role of Coordinator
• Login Modules are participants that follow two-phase process for authentication
• Java Transaction Services (JTS)
• JTS Transaction Manager serves the role of Coordinator
• Software Installation
• In prepare phase, checks are performed, such as making sure that there is enough
disk space
• Databases
• Make use of two-phase commit to ensure data consistency
• Priest conducting a marriage
• Priest is the Coordinator, who follows a two-phase process
• Bride & groom are the participants
Patterns for Effective Management of Resources in Software Systems
40
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Lifecycle Manager – Example
• Distributed system that needs to service thousands of clients
• Clients establish connections to communicate with remote services
• The complexity in managing the lifecycles of connections can affect the
core functionality of the application
• Resource management can make its business logic hard to understand &
maintain due to the entangled connection-management code
How can the lifecycle of
interdependent resources be
managed while ensuring that
the business logic of a system
does not become too hard to
understand & maintain?
Server
Network
Clients
Patterns for Effective Management of Resources in Software Systems
41
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Lifecycle Manager
• Context
• Large & complex systems that require simplified management of lifecycle
of their resources
• Problem
• Resources in systems can be of many different types, such as network
connections, threads, synchronization primitives, servants & so on
• Resources of the same or different types might depend on each other
• How can the lifecycle of interdependent resources be managed such that it
does not make the business logic of the system hard to understand &
manage?
Resource User
• Solution
• Separate resource usage from resource management
Resource Lifecycle
• Introduce a separate Resource Lifecycle
Manager
Manager whose sole responsibility is to manage
and maintain the resources of a resource user
• The Resource Lifecycle Manager maintains
Resource Provider
the dependencies between resources
Patterns for Effective Management of Resources in Software Systems
42
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Lifecycle Manager – Example Resolved
• Introduce a component that is responsible for the resource lifecycle
management & hence free the application from this responsibility
• The clients then use the resource lifecycle manager to request new
connections to the server
• Once these connections are given to the clients, their lifecycle is managed
by the resource lifecycle manager
public ResourceLifecycleManager {
// resource group management
public ResourceGroup createGroup (String groupID,
Resource resources []) { //... }
public void addResourceToGroup (Resource resource,
ResourceGroup group) { //... }
public boolean release (ResourceGroup group) {//... }
// ...
}
Patterns for Effective Management of Resources in Software Systems
43
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Lifecycle Manager – Known Uses
• Component container
• A component container manages the lifecycle of application components &
provides application-independent services
• Further, it manages the lifecycle of resources used by the components
• Examples include: J2EE Enterprise JavaBeans (EJB) , CORBA Component
Model (CCM) , & COM+
• Remoting middleware
• Middleware technologies such as CORBA & .NET Remoting ensure the proper
lifecycle management of resources such as connections, threads,
synchronization primitives, & servants implementing the remote services
• Grid computing
• A resource lifecycle manager is typically used to manage the local resources of
computers participating in a grid
• Resource lifecycle manager ensures that resources in a grid are available &
fulfills local as well as distributed requests to those resources
Patterns for Effective Management of Resources in Software Systems
44
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Patterns for Resource Release
“I saw the angel in the marble and
carved until I set him free.”
Michelangelo Buonarroti
Patterns for Effective Management of Resources in Software Systems
45
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Leasing – Example
• Quoter service, which provides stock quotes to clients
• Reference of Quoter service is made available to clients via Lookup
service
• Suppose the Quoter service were to crash & not come up again
• Quoter service reference would become invalid but would still be available
via the Lookup service
How can the invalid Quoter service reference be automatically
removed at the Lookup service?
Client
query
object Network
reference
publish
object
reference
Quoter
Service
Lookup
Service
Patterns for Effective Management of Resources in Software Systems
46
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Leasing
Context
• Systems where resource use needs to be controlled to allow timely release of unused
resources
Problem
• How can resources that are not used by a resource user, or no longer available be
automatically freed as soon as possible to make them available to new resource
users?
• How can the system load caused by unused resources be minimized?
• How can it be ensured that a resource user does not use an obsolete version of a
resource when a new version becomes available?
Lease
Solution
• Introduce a lease for every resource that is held
by a resource user
• A lease is granted by a grantor & is obtained by
a holder
• A lease specifies a time duration for which the
resource user can hold the resource
• Once the time duration elapses, the lease is
said to have expired & the corresponding
resource is released
Patterns for Effective Management of Resources in Software Systems
47
Holder
Grantor
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Leasing – Example Resolved
• Server containing Quoter service is resource user & Lookup service is
resource provider
• When the Quoter service is registered at the Lookup service, a lease is
created
• The lease specifies the
duration for which the Quoter
service is available
• Once the lease expires, the
Lookup service automatically
removes the Quoter service
reference
: Resource
Provider
: Resource User
: Resource
acquire
resource available?
«create»
: Lease
reference, lease
access
timer
lease expired
release (reference)
release (resource)
expired
Patterns for Effective Management of Resources in Software Systems
48
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Leasing – Known Uses
• Jini
• Jini couples each service with a lease object that specifies the duration of time for
which a client can use that service
• Jini also associates a lease object with each registration of a service with the Jini
lookup service. If a lease expires & the corresponding service does not renew the
lease, the service is removed from the lookup service
• .NET Remoting
• Remote objects in .NET Remoting are managed by the Leasing Distributed Garbage
Collector
• Dynamic Host Configuration Protocol (DHCP)
• A DHCP lease is the time that the DHCP server grants permission to the DHCP client
to use a particular IP address
• Web-based e-mail accounts
• Many web-based e-mail accounts, for example MSN Hotmail or GMX become
inactive automatically if not used for a certain period of time
• The time duration can be regarded as a lease whose renewal requires use of the email account
Patterns for Effective Management of Resources in Software Systems
49
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Evictor – Example
•
•
•
•
A three-tier Application with large number of business objects
State of objects is persistent & so objects can be re-created
Objects need to be in memory to respond to client request quickly
However, keeping too many objects in memory can be expensive
How do we balance the conflicting requirements of keeping objects
in memory for efficiency & at the same time reducing
their number to minimize performance degradation?
NE
NE
NE
Shelf
Shelf
Shelf
Card
Card
Card
Patterns for Effective Management of Resources in Software Systems
Card
50
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Evictor
Context
• Systems that need efficient management of resources
Problem
• How can the lifecycle of a resource be controlled by the frequency of use of the
resource?
• How can release of resources be controlled by factors such as type of resource,
available memory & CPU load?
Solution
• Evictor monitors the use of a resource
• It then controls its lifecycle using one or
more strategies such as LRU
• Resources that are selected using the
strategies are evicted
EvictionInterface
Strategy
Evictor
Patterns for Effective Management of Resources in Software Systems
isEvictable()
info()
beforeEviction
51
1
*
Resource
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Evictor – Example Resolved
• NE, Shelf, & Card objects would implement EvictionInterface
• Each time an object is used, it would get marked with a timestamp
• Periodically, the Evictor would run & use a strategy such as LRU to
identify objects that have not been marked
• Objects that are not marked would be evicted & their corresponding
resources released
• Before being evicted, objects would be given a chance to do any cleanup
NE
NE
NE
Objects
Selected
Shelf Shelf Shelf
Card
Card
Card
Patterns for Effective Management of Resources in Software Systems
Card
52
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Evictor – Known Uses
• Enterprise Java Beans (EJB)
• Activation & deactivation mechanism is provided by the container to swap out
beans from memory to secondary storage
• Java Reference API
• JDK 1.2 introduced the concept of reference objects
• Soft, weak & phantom references
• CORBA
• Servant Manager keeps track of number of instantiated servants
• It evicts servants depending upon memory requirements
• Paging
• OS copies a certain number of pages from the storage device into main memory.
• When a program needs a page that is not in main memory, the OS evicts a page
from memory & copies it to the storage device
• It then copies the required page from the storage device into main memory
Patterns for Effective Management of Resources in Software Systems
53
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Resource Management Patterns Map
Resource
Lifecycle
Manager
Caching
Pooling
Partial
Acquisition
Eager
Acquisition
Evictor
Lazy
Acquisition
Patterns for Effective Management of Resources in Software Systems
Coordinator
Lookup
Leasing
54
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Case Study: Mobile Network
• Overview
• System Architecture
• Hot Spot Analysis
• Base Station
• Operation & Maintenance Center
• Pattern Map
Patterns for Effective Management of Resources in Software Systems
55
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
System Overview
Core
Network
Operation
Radio Network
Controller
&
Maintenance
Center
Base Station
Base Station
Patterns for Effective Management of Resources in Software Systems
56
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station
•
•
•
•
Responsible for communicating with mobile phones
Antennas are used to transmit digitized voice via radio signals
Mediates calls between mobile phones & RNCs
Operated & maintained by the OMC
OMC
RNC
Operation &
Maintenance
Call Processing
Base Station
Patterns for Effective Management of Resources in Software Systems
57
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Operation Maintenance Center (OMC)
• The Operation & Maintenance Center consists of
• User Interface
• Topological Tree
• Tree Management
RNC
Base
Station
CPU
Board
Base
Station
OMC
User Interface
Base Station
Topological Tree
Tree Management
Operation &
Maintenance
Database
Patterns for Effective Management of Resources in Software Systems
58
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
System Interaction – Call Establishment
Mobile phone
Base Station 1
RNC
Core Netw ork
establish call
establish call
verify number
establish call
Patterns for Effective Management of Resources in Software Systems
59
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
System Interaction – Call Hand-over
Mobile phone
Base Station 1
Base Station 2
RNC
OMC
measure signal
w eak signal detected
prepare handover
handover
handover complete
query
get established calls
get established calls
Patterns for Effective Management of Resources in Software Systems
60
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Hot Spot: Flexible Application Discovery
Problem
• Application components are encapsulated & provide service interfaces
• To connect to each other, how can the applications find each other’s service
interfaces?
Solution
• Service interfaces are registered with a lookup service (Lookup)
• Use the lookup service (Lookup) to find service interfaces of used components
3: use
Connection
Mgmt
<ref01>
Call
Processing
<ref02>
User Interface
<ref03>
Call Processing
Connection
Management
1: register
2: lookup
Lookup Service
Lookup Table
Patterns for Effective Management of Resources in Software Systems
61
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Hot Spot: Concurrency
Problem
• Many concurrent activities must be performed, such as call establishment &
performance management
• Multiple threads are created to guarantee responsiveness of the base station
• How can the number of threads that are created be controlled such that there is
optimal resource utilization?
Solution
• Use several threads,
but limit the number
of threads
• Reuse threads (Pooling)
: Operation &
Maintenance
: OMC
: Thread
: Thread Pool
measure performance
«create»
theAction : Long
Running Action
run (theAction)
execute
run (theAction)
return result
return result
«destroy»
Patterns for Effective Management of Resources in Software Systems
62
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Hot Spot: Connection Management
Problem
• Connections are not fixed & are also unreliable
• How can system resources be freed on time when connections are broken?
Solution
• Free resources associated with a connection automatically (Evictor)
OMC
RNC
Operation &
Maintenance
Call Processing
Base Station
Patterns for Effective Management of Resources in Software Systems
63
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Hot Spot: High Performance
Problem
• Interactions between CPU boards are expensive
• The system must guarantee stringent Quality of Service properties
• How can unpredictable overhead be avoided from dynamic resource
acquisitions, such as connection establishment & memory allocations?
Solutions
• Avoid expensive resource acquisitions at run-time (Eager Acquisition)
• Reuse once acquired resources (Caching)
Operation &
Maintenance CPU
Board
Call Processing
CPU Board(s)
Base Station
Patterns for Effective Management of Resources in Software Systems
64
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Hot Spot: Failure Handling
Problem
• OMC service interface may become unavailable & bind resources unnecessarily
• Sudden disconnections cannot be avoided
• How can the base station ensure that a reference to a service interface that is no
longer valid is not used?
Solution
• Lease reference to OMC service interface (Leasing)
• Prolong leases regularly
• In case of failure a new OMC may contact the base station
OMC
RNC
Operation &
Maintenance
Call Processing
Base Station
Patterns for Effective Management of Resources in Software Systems
65
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Base Station Pattern Map
Eager
Acquisition
Lookup
Pooling
Operations and
Maintenance
Call Processing
Caching
Base Station
Evictor
Connection
Management
Patterns for Effective Management of Resources in Software Systems
66
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: Station Discovery & Communication
Problem
• Base stations must be connected to the OMC
• Base station service interfaces are exposed via references
• How can the service interfaces be obtained since the IP addresses of base
stations are not enough to communicate with them?
Solution
OMC
• Fetch references of
application components
using a special protocol
• Store those references
and allow their retrieval (Lookup)
Lookup Service
get reference
Base Station
ftp login/get
reference file
service interface
get reference
lookup
service interface
access
Patterns for Effective Management of Resources in Software Systems
67
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: Network State Maintenance
• Problem
• Network connections are inherently unreliable
• How can the state of a network element be obtained if the network connection to that
element is broken?
• Solution
• Cache state of individual network elements (Caching)
• Represent network elements via proxies
• Inform operator that visible state represents possibly outdated information
RNC
Base
Station
Base
Station
CPU
Board
Operation &
Maintenance
Call Processing
Base Station
Patterns for Effective Management of Resources in Software Systems
68
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: High Performance
Problem
• User Interface must be highly responsive
• How can the OMC be made highly responsive to user input, such as when selecting
network elements & viewing their properties?
Solution
• Buffer data once it’s retrieved (Caching)
• Caching helps to improve performance, avoiding expensive data acquisition
: User Interface
: Tree
Management
: Topological Tree
: Element Cache
: Database
system startup
query (top level elements)
put (elements)
update view ()
user interaction
get element
get
update view
Patterns for Effective Management of Resources in Software Systems
69
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: High Scalability
Problem
• OMC manages thousands of network elements
• Network elements contain large state information
• How can the OMC be made scalable to cope with all the network elements it
manages?
Solution
• Evict node state information that is not needed any longer (Evictor)
• Acquire large state in steps, for example large alarm logs (Partial Acquisition)
RNC
Base
Station
Base
Station
CPU
Board
Patterns for Effective Management of Resources in Software Systems
70
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: Maintenance & Upgrade
Problem
• Software updates must be applied to network elements regularly
• How can it be ensured that all versions of the software are
compatible & consistent?
Solution
Radio Network
Controller
• Coordinate the download &
installation (Coordinator)
• Separate installation
from activation
Base Station
Operation & Maintenance
Center
Base Station
Patterns for Effective Management of Resources in Software Systems
71
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Hot Spot: Complexity
Problem
• Managing resources explicitly is tedious
• Lifecycles might be non-trivial & depend on each other
• How can the lifecycles of interdependent resources be managed?
Solution
• Introduce a Resource Lifecycle Manager to coordinate the lifecycle of complex
resources
Thread acquired
Connection
acquired
Thread
Memory
acquired
Connection
Thread released
Connection
released
Memory
released
Patterns for Effective Management of Resources in Software Systems
Memory
72
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
OMC Pattern Map
Pooling
Lazy
Acquisition
User Interface
Eager
Acquisition
Topological Tree
Partial
Acquisition
Evictor
OMC
Tree Management
Caching
Resource
Lifecycle
Manager
Softw are Distribution
Lookup
Base Station
Patterns for Effective Management of Resources in Software Systems
Coordinator
73
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Case Study: Ad Hoc Networking – Overview
• Ad hoc networking is based on the principle of spontaneous addition,
discovery, & use of services in a network
• Services can be of multiple types, such as
• Simple time services
• PowerPoint presentation services
• MP3 player services
• All ad hoc networking technologies
• Are either platform- or programming language-dependent,
• But they share a common underlying architecture
Patterns for Effective Management of Resources in Software Systems
74
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Motivation
• Consider a mobile network as the ad hoc network
• Network consists of mobile devices containing software that accesses
distributed services
• Services can include
• A stock quote service that delivers real time stock quotes
• A news service that delivers the latest news, customized according to user
preferences
• An advertisement service that delivers advertisements according to user preferences
& location
• Services themselves reside on the mobile phone & communicate with
back-end providers to obtain necessary information
However, devices such as mobile
phones are typically constrained by
memory, processing power, &
storage, & therefore can not host
several services simultaneously
Application
Software
Dynamically
loaded Service
Runtime
Environment
Mobile Phone
Patterns for Effective Management of Resources in Software Systems
75
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Forces
• Efficiency
• Unnecessary service acquisitions should be avoided.
• System load caused by unused services must be minimized.
• Simplicity
• Management of services used by a client should be simple.
• It should be optional for a client to explicitly release services it no longer needs.
• Availability
• Services not used by a client, or no longer available, should be freed as soon as
possible to make them available to new clients.
• Adaptation
• The frequency of use of a service should influence the lifecycle of a service.
• Service release should be determined by parameters such as available memory &
CPU load.
• Actuality
• A device should not use an obsolete version of a service when a new version
becomes available.
• Transparency
• The solution should be transparent to clients.
• The solution should incur minimum execution overhead & software development
complexity.
Patterns for Effective Management of Resources in Software Systems
76
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Solution
• Apply the resource management pattern language
• The following are the four main components that are responsible for
the key interactions in applying the resource management pattern:
• Service – A service is published so
that it can be discovered & used by
clients in the ad hoc network
• Lookup service – A lookup service
is used by service providers that
register services, & by clients to
discover the services
• Client – A client, such as a mobile
device, is an entity that makes use
of a service. The client finds a
service using the lookup service
• Service provider – A service
provider is an entity that provides a
service for clients by registering the
service, or only part of it, with the
lookup service
Lookup Service
«uses»
«uses»
«uses»
Client
Patterns for Effective Management of Resources in Software Systems
Service Provider
«uses»
«provides»
Service
77
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone – Solution Structure
Directory :
Lookup Service
Infrastructure Server
Registraton/Lookup Interface
Acquisition
Interface
Application
Software : Client
Stock Exchange :
Service Provider
Quoter Proxy
: Service Proxy
Quoter Service
: Service
Service Interface
Mobile Phone
Patterns for Effective Management of Resources in Software Systems
Server
78
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone
Lazy Acquisition
• The user of the mobile phone can decide any time to use a service
• The services are loaded lazily –
the service must not
be loaded in advance
• The directory is consulted on demand
Directory :
Lookup
Service
for services to be downloaded
on the mobile phone
Infrastructure Server
Registration/Lookup Interface
Application
Software : Client
?
?
Mobile Phone
Patterns for Effective Management of Resources in Software Systems
Server
79
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone
Lookup: Service Registration
• Stock exchange service provider acts as a resource provider, while the quoter
service is the actual resource that it provides
• Stock exchange creates & registers stock quoter service with the directory
• Stock exchange service provider submits a reference to itself along with a set of
properties that describe the services that it provides
• Stock exchange service provider registers quoter proxy with the lookup service
Stock Exchange :
Service Provider
«create»
Directory :
Lookup Service
Quoter Service
: Service
register (stock exchange, "Quoter", quoter proxy)
Patterns for Effective Management of Resources in Software Systems
80
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone
Lookup: Service Discovery
• Application software of mobile phone first finds the directory
• It then downloads the client proxy & installs it
• It finally acquires the quoter service & uses it to fetch stock quotes
Application
Softw are : Client
Stock Exchange :
Service Provider
Quoter Service :
Service
Directory :
Lookup Service
lookup ("Quoter")
reference to stock exchange, quoter proxy
install proxy
acquire service
quote ("SI")
Patterns for Effective Management of Resources in Software Systems
81
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone
Leasing, Evictor
• Once the mobile phone finds
a service, it acquires a lease
to gain access to the service
• A lease is a grant of
guaranteed access for a
specific time
• Each lease is negotiated
between the client & the
service provider
• References to service
providers are automatically
removed from the directory
that no longer provide any
services, & therefore whose
leases have expired
• Corresponding resources are
automatically evicted
Application
Softw are : Client
Stock Exchange :
Service Provider
Quoter Service :
Service
acquire service
create lease
lease
quote ("SI")
lease expired
prepare for eviction (quoter proxy)
Patterns for Effective Management of Resources in Software Systems
release service
evict proxy
82
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Example: Mobile Phone– Pattern Map
Leasing
Client
Service
Provider
Lookup Service
Lookup
Lazy
Acquisition
Evictor
Service
Patterns for Effective Management of Resources in Software Systems
83
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Summary
• The presented patterns
• Address non-functional properties, such as
• Scalability, Performance, Predictability, Stability, Consistency, & Flexibility
• Cover the complete lifecycle from acquisition to release
• Are independent of any application domain
• Are generalizations of more specialized patterns
• Not addressed topics in this tutorial
• Access to concurrent resources, see POSA2 & Doug Lea’s book “Concurrent
Programming in Java”
• Fault tolerance techniques, to be published in 2007
• Reference counting, see POSA 4/5
• Extension of the pattern language with new experiences in
• Grid computing – sharing & aggregation of distributed resources
• Autonomic computing – automated resource management anticipating
resource need
Patterns for Effective Management of Resources in Software Systems
84
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Other Literature
• The POSA3 patterns are tightly integrated with other pattern literature
Patterns for Effective Management of Resources in Software Systems
85
Douglas C. Schmidt, Prashant Jain, & Michael Kircher
Pattern-Oriented Software Architecture, Volume 3:
Patterns for Resource Management
Michael Kircher, Prashant Jain
John Wiley & Sons
ISBN 0-470-84525-2
http://www.posa3.org
Patterns for Effective Management of Resources in Software Systems
86
Douglas C. Schmidt, Prashant Jain, & Michael Kircher