Transcript Talk
An Analytical Model for Multi-tier Internet Services and its Applications Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer, Asser Tantawi University of Massachusetts and IBM TJ Watson UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science Internet Applications Proliferation of Internet applications auction site online game online store Growing significance in personal, business affairs Focus: Modeling Internet applications UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 2 Why Model Internet Applications? Capacity provisioning How many servers does the application need? Performance prediction E.g., predict response time Application configuration Tune various application parameters Request policing Turn away excess requests during overloads UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 3 Internet Application Architecture search “moby” queries response Melville’s ‘Moby Dick’ Music CDs by Moby HTTP J2EE Database request processing in an online bookstore Multi-tier architecture Each tier uses services provided by its successor Session-based workloads Caching, replication UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 4 Existing Application Models Models for Web servers [Chandra03, Doyle03] Do not model Java server, database etc. Black-box models [Kamra04, Ranjan02] Unaware of bottleneck tier Extensions of single-tier models [Welsh03] Fail to capture interactions between tiers Existing models inadequate for multi-tier Internet applications UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 5 Talk Outline Motivation Application Model Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 6 Baseline Application Model clients application Model consists of two components Sub-system to capture behavior of clients Sub-system to capture request processing inside the application UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 7 Modeling Clients Z Client 1 Z Client 2 clients Client N application Z Q0 Clients think between successive requests Infinite server system to capture think time Z Captures independence of Z from processing in application UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 8 Modeling Request Processing p1 p3 pM=1 p2 S1 Q1 N tier 1 S2 SM Q2 QM tier 2 tier M Transitions defined to capture circulation of requests Request may move to next queue or previous queue Multiple requests are processed concurrently at tiers Processor sharing scheduling discipline Caching effects get captured implicitly! UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 9 Putting It All Together p1 p3 Z client Z client p2 S1 S2 Q1 Q0 N pM=1 tier 1 SM Q2 QM tier 2 tier M A closed-queuing model that captures a given number of simultaneous sessions being served UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 10 Model Solution and Parameter Estimation Mean Value Analysis (MVA) Algorithm Computes mean response time Visit ratios Equivalent to trans. probs. for MVA Vi ≈ λi / λreq ; λreq at policer, λi from logs Service times Use residence time Xi logged at tier i For last tier, SM ≈ XM Si = Xi – ( Vi+1 / Vi ) · Xi+1 Think time Measured at the entry point of application UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 11 Talk Outline Motivation Application Model Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 12 Evaluation of Baseline Model Auction site RUBiS One server per tier Apache 150 JBOSS 75 Mysql 30000 Avg resp time (msec) 25000 Observed Basic Model 20000 15000 10000 5000 0 0 100 200 300 400 Num sessions 500 Concurrency limits not captured UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 13 Handling Concurrency Limits Z Z N Q0 S1 S2 Q1 Q2 SM QM dropped requests Requests may be dropped due to concurrency limits Need to model the finiteness of queues! UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 14 Handling Concurrency Limits Z Z N Q0 drop p1 drop S1 S1 Q1 S2 SM Q2 drop Q1 QM drop pM drop SM drop QM Approach: Subsystems to capture dropped requests Distinguish the processing of dropped requests UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 15 Avg resp time (msec) Response Time Prediction 30000 25000 20000 15000 10000 5000 0 Observed Basic Model Enh Model 0 100 200 300 400 Num sessions 500 Enhanced model can capture concurrency limits UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 16 Query Caching at the Database Caching effects Bulletin-board site RUBBoS 50 sessions SELECT SQL_NO_CACHE causes Mysql to not cache the response to a query Avg resp time (msec) Captured by tuning Vi and/or Si 1400 More model enhancements 1200 Observed Model 1000 800 600 400 200 0 0 80 60 40 20 % queries cached 100 Replication at tiers Multiple session classes UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 17 Prototype Data Center 40+ Linux servers Gigabit switches Multi-tier applications Auction (RUBiS) Bulletin-board (RUBBoS) Apache, JBOSS (replicable) Mysql database UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 18 Dynamic Capacity Provisioning Auction application RUBiS Factor of 4 increase in 30 min Server allocations Workload 160 120 100 80 60 40 10 20 30 40 Time (min) 50 60 10 7000 Web servers App servers Resp time (msec) Number of servers Arrivals per min 140 20 0 Response time 12 8 6 4 2 0 0 10 20 30 40 Time (min) 50 60 6000 5000 4000 3000 2000 1000 0 0 10 20 30 40 Time (min) 50 60 Server allocations increased to match increased workload Response time kept below 2 seconds UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 19 Talk Outline Motivation Baseline Application Model Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 20 Summary and Future Work Analytical model for Multi-tier Internet Applications Mean-value analysis Concurrency limits, replication, caching, multiple classes Model validation using 3-tier applications Dynamic provisioning, request policing Future work Handling load imbalances at replicated tiers Handling more diverse workloads Handling other kinds of scheduling disciplines at servers UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 21 Thank you! More information at: http://www.cs.umass.edu/~bhuvan UNIVERSITY OF MASSACHUSETTS, AMHERST – Department of Computer Science 22