A Framework for Detecting Performance Design and Deployment
Download
Report
Transcript A Framework for Detecting Performance Design and Deployment
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Detecting Performance
Design and Deployment Antipatterns in
Enterprise Systems
Trevor Parsons
Performance Engineering
Laboratory
University College Dublin
Ireland
Performance
Engineering
Laboratory
Performance Engineering Laboratory
1
www.perfenglab.com
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Presentation Outline
Description of purpose
Goal Statement
Solution Overview and Architecture
What new understanding, knowledge, methods, or technologies (i.e. contributions) will
this research generate?
Methodology
Performance issues in Enterprise Systems
Limitations in Current Performance Tools
Performance Design and Deployment Antipatterns
What experiments, prototypes, or studies will be done to achieve the stated goal?
Evaluation
Performance
Engineering
Laboratory
How will the cotributions be evaluated?
Performance Engineering Laboratory
2
www.perfenglab.com
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Performance Issues in Enterprise Systems
Enterprise Frameworks provide services to help developers meet performance
goals, however…
Often poor performance is caused by poor design, not poor code
Example:High inter-component communication -> poor performance
Complex distributed enterprise systems are not properly understood
Performance not guaranteed
Performance issues may still arise
Applications tend to be large
Complex execution envronment (e.g. application servers)
Peformance implications of design decisions may be unknown
Parts of application outsourced
Commmercial Off the Shelf (COTS) Components
Time to market constraints
Performance is often a major issue for enterprise systems
Performance
Engineering
Laboratory
Performance Engineering Laboratory
3
www.perfenglab.com
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Limitations of current tools
Generate massive amounts of data
Multi User Systems (1000’s of users)
May overwhelm developers trying to locate bottlenecks
e.g. Profilers
Give no reason for, or solution to, performance issues
Do not detect potential bottlenecks
Performance
Engineering
Laboratory
May occur if system properties changes
e.g. increase in workload
Performance Engineering Laboratory
4
www.perfenglab.com
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Performance Design and Deployment Antipatterns
Performance
Engineering
Laboratory
Design patterns – document recurring solutions
to standard software developent problems
Antipatterns – document common mistakes
made during software development
Suggest solutions/refactorings
Performance patterns/antipatterns solely focus
on performance concerns
Performance antipattern discovery helps
developers to locate and treat performance
issues within the system
Helps developers form a sense of performance
intuition
Performance Engineering Laboratory
5
www.perfenglab.com
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Antipattern Hierarchy
Antipatterns
…..
EJB
Performance
Deployment
Antipatterns
Performance
Engineering
Laboratory
Performance
Antipatterns
Antipatterns
associated
with other
software quality
attributes
EJB
Performance
Antipatterns
.NET
Performance
Antipatterns
EJB
Performance
Design
Antipatterns
EJB
Performance
Programming
Errors
Performance Engineering
Laboratory
Performance
6
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Solution Overview
Automatically Detect and Assess Performance Design and Deployment Antipatterns in Component Based
Enterprise Systems
•
Monitoring: Obtains performance information from a running application
•
Analysis: Uses analysis techniques(e.g. Data Mining) to make sense of large volumes of data
•
Detection: Rule Engine - Antipatterns are described in terms of rules
•
Assessment:Performance models applied to ranks antipatterns in terms of their performance impact on the system
•
Presentation: Creates UML models of the system augmented with performance data, highlighting discovered antipatterns. Diagrams are
created at different levels of abstraction (following the MDA paradigm).
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
7
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Architecture
Monitoring
JVMTI
JMX MEJB
Monitor
Call Path
Tracer
XML Parser
Custom
Monitoring Tool
Analysis
Determine System Structure
Data Mining
CustomAnalysis
Statistical Analysis
Detection
Rule Engine
Rules
Performance
Engineering
Laboratory
Assessment &
Ranking
Presentation
Micro
Performance
Models
Performance Engineering
Laboratory
Performance
8
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Problem
Network
Controller
Client
Entity 3
4 Remote Calls
1 : \getName\
View
Controller
2 : \getAddress\
3 : \getAge\
4 : \getId\
Entity1
Entity2
Entity3
Solution
Controller
Session Facade
Entity3
Client
Improved
Performance
Design
Network
1 Remote Call
View
1 : \getDetails\
Controller
2 : \getName\
3 : \getAddress\
Session Facade
4 : \getAge\
5 : \getId\
Entity1
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Entity2
Entity3
Performance
9
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
New Knowledge, Methods, Technologies
•
Technologies
–
Monitoring: Non-Intrusive Call Path Tracer for J2EE
•
•
•
Methods:
–
Analysis: Data Mining
•
•
•
•
–
–
•
Collects chain of component methods called for each user request
Maintains the order of calls across all tiers (web,business and database)
Association Rule Mining (patterns of interest)
Clustering(data reduction)
Sequential Rule Mining (patterns of interest)
Statistical analysis (data reduction)
Detection: Apply Expert Systems (Rule Engine) to Analyse Performance Design
Assessment: Micro Performance Models
Knowledge:
–
–
–
–
Performance
Engineering
Laboratory
Understanding of Enterprise Systems (from performance perspective)
Automatic Identification of Design and Deployment Issues (current tools focus on errors)
Identify Potential Problems
Data Reduction and Pattern Identification
Performance Engineering
Laboratory
Performance
10
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Methodology
• Prototype & Evalutaion
• Current Status
– Currently Implementing tool:
• Monitoring: COMPAS J2EE (on sourceforge)
• Analysis: Currently being Implemented
• Detection: Prototype has been implemented
• Micro Performance Models: Future work
– Access to a number of Very Large Real Enterprise Applications(with
100’s of software components)
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
11
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Evaluation
•
Criteria:
– Accurate
• How many antipatterns can we detect
• How many false positives/negative
– Data Reduction
• Measure volume of data produced
– Extensible Design
• Can custom monitoring/analysis/antipatterns/models be added
– Performant
• Who profiles the profiler
• Does the prototype perform reasonably well
– Scalable
• Do the above scale
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
12
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Future Work
• Complete Implementation
• Test on Real Systems!!
• Find Real Problems!
• Create and Apply Performance Models to
Problems.
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
13
www.perfenglab.com
Engineering
Laboratory
Detecting Performance Design and Deployment
AntiPatterns in Enterprise Systems
Trevor Parsons
[email protected]
Questions
Publications:
Performance
Engineering
Laboratory
Trevor Parsons, John Murphy. "Data Mining for
Performance Antipatterns in Component Based
Systems Using Run-Time and Static Analysis."
Transactions on Automatic Control & Control Science,
Vol. 49 (63), No. 3, May 2004, pp. 113-118 - ISSN
1224-600X.
Trevor Parsons, John Murphy. "A framework for
automatically detecting and assessing performance
antipatterns in component based systems using runtime analysis”. The 9th International Workshop on
Component Oriented Programming, part of the 18th
European Conference on Object Oriented
Programming. June 2004, Oslo, Norway.
Trevor Parsons. “A Framework for Detecting,
Assessing and Visualizing Performance Antipatterns in
Component Based Systems”. First Place at the
OOPSLA ACM SIGPLAN Student Research
Competition at The 19th Annual ACM Conference on
Object-Oriented Programming, Systems, Languages,
and Applications. October, 2004, Vancouver, Canada.
(Poster and 2 page abstract)
Performance Engineering Laboratory
14
www.perfenglab.com
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
COMPAS J2EE
Monitoring Tool:
• Non-instrusive End-to-End monitoring infrastructure
• Do not need to change application source code
• Completely portable across application
servers/databases
• Can Trace requests across the different tiers and
maintains the order of calls
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
15
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Antipattern Categories
Inter Component
Communication Antipatterns
Stateful Antipatterns
• Fine Grained Remote Calls
• Bloated Stateless Sessions
• Aggressive Loading
• Thin Sessions
• EJB Home Far Away
• Sessions A-Plenty
• ……
• Eager Iterator
• Bloated Stateful Sessions
•…..
Pooling Antipatterns
Performance
Engineering
Laboratory
•Incorrect Stateless Session Pool Size
Other Antipatterns
•Incorrect Stateful Session Cache
• Incorrect Transactions Size
•……..
•…..
Performance Engineering
Laboratory
Performance
16
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
COMPAS J2EE
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
17
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Advantages of Approach
•
Antipatterns clearly defined (categorised) and documented
–
–
•
Automatic detection and assessment ….
–
–
–
–
•
Takes onus away from developer having to sift through large volumn of performance data
Makes sense of large amount of performance data rather than merely presenting it to user
Developers may gain performance Intuition
Highlights potential bottlenecks/antipatterns
Developers Can Reason about their Performance Design & Deployment Settings
–
–
•
The better understood antipatterns become, the less likely developers will be to make such mistakes, and
the easier it will be for them to recognize the antipatterns if they exist.
Examples/Test Results included in antipattern description
previous tools analyse performance programming errors rather than design errors
allows developers to assess their design (Is my design optimal???)
New detection approach uses runtime (dynamic) analysis
–
–
Traditional design recovery/reverse engineering techniques use static analysis
source code often unavailable
•
•
–
Fits well with modern software development processes
•
–
Performance
Engineering
Laboratory
COTS components
Outsourcing
Require a running implementation of the system at each iteration of the development process (e.g. XP)
Need run-time performance data to reason about performance (performance antipatterns have dynamic
properties)
Performance Engineering
Laboratory
Performance
18
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Sample Rules and Facts
Antipattern: Stateless Sessions-a-plenty
If A is a Session Bean
& If A has no transactional methods
& If A has small (e.g. <3) no of methods
& If A does not require security settings
& If A does not have a relationship with any entities or DB
(Facts) Data Required:
• Component Types
• Components Methods
• Transactions
• Security Settings
• Component Relationships (from call paths)
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
19
www.perfenglab.com
Engineering
Laboratory
Detecting Performance AntiPatterns
in CBS
Design and Deployment
AntiPatterns in Enterprise Systems
Trevor
TrevorParsons
Parsons
[email protected]
[email protected]
Sample Rules and Facts
Example Association Rule:
(Do not confuse the Association Rule with the RULES in the Rule Engine)
employee.getName= > employee.getAddress , employee.getAge ,
employee.getID
Confidence 90%
Support 20%
Antipattern: Fine Grained Remote Calls
This antipattern describes a situation where fine grained remote calls are being used. Often Fine grained calls can be
eradicated using a session façade.
Rule which describes Antipattern (loaded into Rule Engine):
If A is an association rule
–
& If A has two or more calls
–
& If A has more than X% confidence
–
& If A’s method calls belong to remote components
RULE ENGINE FIRES =>fine grained remote calls Antipattern Detected !!!!!!!!
(Facts) Data Required:
•
Association Rules (from call paths)
•
Component Types
•
Component Methods
Performance
Engineering
Laboratory
Performance Engineering
Laboratory
Performance
20
www.perfenglab.com
Engineering
Laboratory