Transcript Capacity Planning and Infrastructure Best Practices for Microsoft

Capacity Planning and Infrastructure
Best Practices for Microsoft Office
SharePoint Server 2007
Meron Fridman
Microsoft Office System Regional Director
CTO – Bynet Software Systems
[email protected]
Agenda
Addressing Capacity Boundaries and Considerations
Workload Considerations
Collaboration, Web Publishing, Search
Capacity planning
Performance, High Availability, Storage
Additional infrastructure related best practices
Common Questions
How much hardware do we need?
Should we implement a server farm?
Do we need SQL Server?
How much data can we store?
How many users can our environment support?
How many sites can we run on our servers?
How do we validate our design?
What tools can we use to measure performance?
How to setup Development environment
Development environment best practices
• Each developer will use own SharePoint
• All members in the same farm (Backup only the farm)
• Work with host headers (on port 80)
• Point the host file in each developer machine to 127.0.0.1
• Use Integration server for deploy modules
• Use the same version in your all environments
- SharePoint 207
- SharePoint 207
- SharePoint 207
Active
Directory
SharePoint 2007 Index +
Managment
SharePoint
2007
Capacity Planning
The art of evaluating a technology against
the needs of an organization,
and making an educated
decision
about the procurement of Hardware to meet the
demands specific to a system being installed
Framework
Planning for Capacity and Performance
Components
Software Boundaries
Throughput Targets
Data Capacity
Hardware
Phases
Phase 1: Plan for Software Boundaries
Phase 2: Estimate Performance and Capacity Requirements
Phase 3: Plan Hardware and Storage Requirements
Phase 4: Test Your Design!
Performance and capacity planning: The process of mapping your solution design to a
farm size and set of hardware that will support your business goals.
Plan for Software Boundaries
Phase 1
Object Categories
Software Scalability vs. Hardware Scalability
RTM Test Results, Findings, and Recommendations from the
Product Group
Other Considerations
Plan for Software Boundaries
Object Categories
Site Objects
Site Collections, Web sites, documents, document libraries, list
items, document file size, etc.
People Objects
User profiles, security principals, etc.
Search Objects
Search indexes, Indexed documents
Logical Architecture Objects
Shared Services Providers, Site Collections, Content Databases,
Zones, etc.
Physical Objects
Servers: Index, Web FE, Database, Application, etc.
Product Considerations
Site Collection Max and Recommended Sizes
Database Max and Recommended Sizes
Backup and Operations Considerations
Server Throughput Considerations
User Type Considerations
Authentication Considerations
Plan for Software Boundaries
Software Scalability vs. Hardware Scalability
Software scalability
Recommendations for acceptable performance based on software
behavior and characteristics
Hardware scalability
Does not change/modify software behavior or characteristics…but
can increase overall throughput of a server farm and might be
necessary to achieve acceptable performance as the number of
objects approach recommended limits
“Plan for Software Boundaries”
(TechNet)
Plan for Software Boundaries
Other Considerations
Throughput vs. number of Web servers
Test findings showed plateau at 5:1 (Web FE : SQL)
Perform tests in your environment
Other Recommendations
Carefully plan your site hierarchy and design
Minimize # Web applications and application pools
Limit # of Shared Service Providers (SSP)
Plan for database growth
Follow data and feature best practices and suggested limits…
Growth patterns
Server performance
High availability
Applications
Data growth
Search
MOSS
Global Intranet Considerations
Centralized or Distributed
Global teams or local teams - Where to put the sites?
End user experience - Bandwidth/Latency?
Operational costs
Search
Offline
Network accelerators
3rd Party replication
Products
50 MB
50MB
Munich
Chicago
1 MB
Peking
2MB
35MB
2MB
2MB
N
W
E
S
backbone
0-100ms
101-200ms
201-300ms
>300ms
Data centers
Sites
Web Publishing
Characteristics
Fewer content creators (may be external agencies)
Large number of viewers (authenticated or not)
Small number of central sites for content publishing
Approval workflow
Staging and deployment
Database Characteristics
Publishing databases mainly read
Design considerations
Design for end to end performance
Page size matters! (html, .js,.css,…. Compression …)
More memory than CPU intensive (caching strategy!)
Not everything is deployable via content deployment
Search
Characteristics
Multiple sources
Security trimming
Indexing Characteristics
Network/CPU intensive
Querying Characteristics
CPU/Memory/IO Intensive
Database Characteristics
Property store in SQL (Search_DB)
Design considerations
50 Million items / catalog
Estimate Performance and Capacity
Phase 2 – Usage Profiles
Determine Usage Profile
Usage profile == User community’s behavior
Distribution of requests across content
Operation types and frequency (Any “heavy” Search patterns?)
Existing solution in place? Analyze IIS logs …
Leverage usage profiles provided in configurations tested by
Product Group as starting point:
Configuration tested by Product Group
Windows SharePoint Services collaboration environments
Portal collaboration environments
Search environments
Internet environments
InfoPath Forms Services environments
Determine resource requirements to support Excel Services
Estimate Performance and Capacity
Phase 2 – Other Factors
Configuration factors that can influence throughput targets
Indexing
Schedule indexing window off-hours?
Use Dedicated Web FE (Hosts file …)
Caching enabled?
Output Caching and Cache Profiles (Individual Page Level)
Object Caching (Individual Web Part control, field control, and content level)
Disk-based Caching for Binary Large Objects (Web FE Level)
Page customizations
Custom Web parts
<BlobCache location="C:\blobCache" path="\.(gif|jpg|png|css|js)$" maxSize="10" maxage="86400" enabled=“true"/> in the web.config file for a web application
Estimate Performance and Capacity
Phase 2 – Other Factors, Latency
Latency components
Server processing
SQL processing, # SQL round trips, security trimming
Client processing
Javascript, CSS, AJAX requests, HTML load, Client machine specs
Bandwidth, size of download
Recommendations
Primary cause of latency problems: custom web parts (Dispose)
Watch for:
SQL round trips, unnecessary data, excessive client side script
Re-use existing client code versus adding more
Design code for performance – (Use HTML and .Net best practices)
Plan Hardware and Storage
Phase 3 – How SharePoint Scales
Designed to grow with organization needs
Server resources: x32, x64, CPU, RAM, Storage
Recommend 64-bit for back end services (SQL) which can leverage
additional addressable memory
SQL: Storage configuration is critical!
Server Farm
Topology restrictions removed
WFE, Query, Index, Excel Calc, Project, SQL
Adopted WSS adage: content only limited by HW capability*
Sites: In WSS 3.0, Portals sites are "just another WSS site”
Plan Hardware and Storage
Phase 3 – 64-bit vs. 32-bit Hardware
WSS 3.0 and MOSS 2007 can work on both
64-bit Hardware Recommended
32-bit can directly address only a 4GB Memory Address Space
64-bit supports up to 1024GB Memory (Physical / Addressable)
32-bit will look better and perform better with 2GB RAM, but we
recommend 64-bit for future investments and scale
64-bit HW Prioritization
SQL Server  Index  Excel  Search  WFE
64-bit hardware can be mixed within a farm (on different tiers)
PDF IFilter for x64?
Adobe - http://labs.adobe.com/wiki/index.php/PDF_iFilter_8_-_64-bit_Support
FoxIT - http://foxitsoftware.com/pdf/ifilter/
Plan Hardware and Storage
Phase 3 – Storage Considerations
Primary Metric: Document Storage
Plan for 1.2 – 1.5 x file system size for SQL Server
Secondary Metric: Index Size
Index Server: ~5-12% of total size of all indexable content (WSS);
Reserve 2.5 x Index size!
Query Server: Same as Index Server
Search_DB: 4 x Index size!
Use Site Collection Quotas to manage DB size
Plan Hardware and Storage
Additional Storage Considerations
Use dedicated SQL DB for large Site Collections
Limit the number of Sites per Content DB (Web Application), or
Create the Content DB while creating the Site Collection (STSADM)
Use separate LUNs (Disks) for:
Search_DB
TempDB – Divide into multiple equal size DB files based on # CPUs
Use separate LUN for each DB file in case there is I/O bottleneck
Use a separate SQL instance for Search
SharePoint Client Payload
Background
SharePoint sends down a relatively large payload when you
visit the site; even though it is improved from SPS 2003
This payload comes from jScript files, IE Behaviors, and
images, in addition to the text content
Some elements continue to come down
Multiple times on a single page
Download again during same browser session on the same site
Files requested on first hit of the home page of a just
created publishing site
/Pages/Default.aspx
/_layouts/1033/core.js
/_layouts/1033/ie55up.js
/_layouts/images/gosearch.gif
/_layouts/1033/init.js
/_layouts/1033/styles/core.css
/PublishingImages/newsarticleimage.jpg
/WebResource.axd
/WebResource.axd
/_layouts/1033/search.js
/_layouts/images/navshape.jpg
2,924
/_layouts/1033/EditingMenu.js
/WebResource.axd
/_layouts/1033/styles/controls.css
1,448
/_layouts/1033/styles/HtmlEditorTableFormats.css 1,317
/_layouts/images/titlegraphic.gif
1,299
/_layouts/images/icongo01.gif
/_layouts/images/pagebackgrad.gif
/_layouts/images/stsicon.gif
/_layouts/images/itgen.gif
/_layouts/images/itdisc.gif
/_layouts/images/itlink.gif
/_layouts/images/ittask.gif
/_layouts/images/itcontct.gif
/_layouts/images/helpicon.gif
/_layouts/images/recycbin.gif
/_layouts/portal.js
/_layouts/1033/styles/HtmlEditorCustomStyles.css 642
/_layouts/images/itevent.gif
/_layouts/images/itdl.gif
/WebResource.axd
/WebResource.axd
/_layouts/images/siteTitleBKGD.gif
/_layouts/images/tvplus.gif
/_layouts/images/tvminus.gif
/_layouts/images/quickLaunchHeader.gif
/_layouts/images/topnavselected.gif
/_layouts/images/siteactionsmenugrad.gif
/_layouts/images/topnavunselected.gif
/_layouts/images/menudark.gif
/_layouts/images/whitearrow.gif
68
/_layouts/images/Menu1.gif
/_layouts/images/pageTitleBKGD.gif
/_layouts/images/navBullet.gif
/_layouts/images/tvblank.gif
/_layouts/images/lstbulet.gif
/_layouts/images/blank.gif
107,128
54,367
20,508
19,933
15,732
13,596
10,710
8,272
5,373
5,092
2,735
2,482
1,171
1,082
1,070
1,050
1,049
1,043
1,039
1,036
1,025
1,004
954
619
582
224
218
209
209
207
148
146
146
92
68
68
63
58
56
56
43
Total 288,361 Bytes (nearly 300K)
SharePoint Client Payload
Available Solutions
HTTP Compression (you decide …)
Enabled and turned on by default for Static files like .js, .css etc
.aspx pages need to be added
Compression improves latency and decreases payload size
Compression affects up to 15% throughput
Enable Caching at various levels
How to create a detached page that downloads the Core.js file
but that does not reference the Core.js file in a SharePoint
Server 2007 site - http://support.microsoft.com/?id=933823
Performance Testing and
Troubleshooting
Framework
Planning for Capacity and Performance
Phases
Phase 1: Plan for Software Boundaries
Phase 2: Estimate Performance and Capacity Requirements
Phase 3: Plan Hardware and Storage Requirements
Phase 4: Test your Design!!!
Poor Performance Troubleshooting
Poor throughput
SQL – use SQL best practices for performance especially disk
performance
Asynchronous operations and timer job “conflicts”
Resource contention: # web apps, app pools, DBs, etc.
Check ANY custom code for SQL round trips and payload
Look for mis-configured network components (NICs, routers, etc.)
Search consumes multiple resources
Poor end user perceived latency
Page payload
OOB 1st page download ~200-300KB; Should not be significantly higher
Use page compression where possible
Caching strategy: Turn on BLOB & Output caching whenever possible
Client machine hardware configuration
Mis-configured network (see above)
Additional Infrastructure Best
Practices
Best Practices
Office SharePoint Search
Index requires NTLM authentication - Use separate Web
Applications and Zones
Office SharePoint Server Search account Must NOT be a member
of the Farm Administrators group!
Alternate Access Mapping:
http://blogs.msdn.com:80/sharepoint/archive/2007/03/06/what-every-sharepointadministrator-needs-to-know-about-alternate-access-mappings-part-1.aspx
Domain Controllers and authentication overhead
4:1 ratio (Web FE : Domain Controller for NTLM authentication)
Web Application Security Policies – Scenarios
Full read – search crawling accounts, auditors, legal compliance
Deny all – security control, regulatory compliance
Deny write – extranet lockdown
Office SharePoint Server
Backup and Restore
Data Protection Manager 2007 and MOSS
System State
Internet
Information
Services (IIS)
“Front End”
“Farm” Config_DB
(SQL)
SharePoint VSS Writer
DPM 2007
Files
Content Servers (SQL)
Enterprise Search (index)
SharePoint Server 2007 Recovery
• The Entire Farm
“Farm” Config_DB
(SQL)
Entire Farm
DPM 2007
Content Servers (SQL)
Enterprise Search (index)
SharePoint Server 2007 Recovery
• The Entire Farm
• A Content DB
“Farm” Config_DB
(SQL)
Content DB information
DPM 2007
Content Servers (SQL)
Enterprise Search (index)
SharePoint 2007 Recovery
Content DB
• The Entire Farm
• A Content DB
• Site Collection
• A Site
“Recovery Farm”
(single server)
Temporary Staging Area
Complies with SharePoint Server design
Garbage scrubbed after restore
Might be a virtual machine (VM)
• Document
“Farm” Config_DB
(SQL)
DPM 2007
DPM handles restore thru Recovery
Farm to production Farm
Farm then redirects data to
appropriate content database and site
Content Servers (SQL)
Enterprise Search (index)
Summary
Best practices for capacity planning
Start big enough but not too big
Do proper monitoring in order to know what/when to grow
Go 64-bit where possible
Plan End to End user experience
Bandwidth
Latency
Payload
Related Content
Planning and Architecture for Office SharePoint Server 2007
– Plan for Performance and Capacity: http://technet2.microsoft.com/Office/enus/library/8dd52916-f77d-4444-b593-1f7d6f330e5f1033.mspx?mfr=true
– Design Server Farms and Topologies: http://technet2.microsoft.com/Office/enus/library/651be1bf-c354-4f6d-965d-d3385cea0a2d1033.mspx
– Plan for Software Boundaries: http://technet2.microsoft.com/Office/enus/library/6a13cd9f-4b44-40d6-85aa-c70a8e5c34fe1033.mspx
– Plan enterprise content storage: http://technet2.microsoft.com/Office/enus/library/9994b57f-fef8-44e7-9bf9-ca620ce207341033.mspx
– Working with large lists in Office SharePoint Server 2007:
http://technet2.microsoft.com/Office/en-us/library/6f03049f-5bfe-4807-b6090e2d4a9ec3b51033.mspx?mfr=true
– How Large for a single SharePoint content database:
http://blogs.msdn.com/joelo/archive/2006/08/01/how-large-for-a-single-sharepoint-contentdatabase.aspx
Watch the Blogs
–
–
http://blogs.msdn.com/sharepoint
http://blogs.msdn.com/joelo
More related Content
HP Performance Whitepaper
–
http://h71019.www7.hp.com/ActiveAnswers/cache/497613-0-0-0-121.html
Dell Case Study
–
http://blogs.msdn.com/sharepoint/archive/2007/05/04/dell-case-study-showcase-ofconsolidation-manageability-and-scale.aspx
How to defragment Windows SharePoint Services 3.0
database and SharePoint Server 2007 database
–
http://support.microsoft.com/kb/943345
Questions?
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.