Transcript Office 365 Network Topology and Performance Planning

Microsoft has invested $15 billion in infrastructure,
building over 100 datacenters and we are constantly
evaluating new locations
Our high-performing network is one of the top 3
in the world with public peering in 23 countries
with 1,500 ISPs.
Our Datacenters support over 20 Million businesses
and over 200 Online Services. Office 365 is sold in
131 markets, 43 languages, and 25 currencies.
Huge infrastructure scale enabling service growth
Quincy
Cheyenne
Dublin
Chicago
Amsterdam
Boydton
Beijing
Des Moines
Hong Kong
San Antonio
Servicing Office
365 customers
Other Microsoft
services
Japan
Shanghai
Singapore
Brazil
35+ factors in site selection:
 Proximity to customers
 Energy, Fiber infrastructure
 Skilled workforce
Australia


100+ datacenters
1+ million servers
Geo-Redundant
Service/Application Design
Top 3 Most Connected Networks
in the World
DC-to-Internet Backbone
Network Device Count Growth
2
3
4
5
6
7
• Peer with over 1,500 ISP’s globally
• Multiple Terabits,
• Over 50 Points of Presence globally
• Global backbone connecting MS Datacenter to the
Internet
DC-to-DC Backbone
• Multiple Terabits of Capacity
• Dark fiber based DC-DC backbone to enable high
bandwidth between Datacenters
Dark Fiber
• Tens of thousands of Route Miles of owned Dark
Fiber Backbone
• Million+ 10G DWDM Route Miles of capacity
deployed
Cache Node
• Hosting Services collocated at User location (metro)
Edge Nodes
• Multiple Terabits of Edge Interconnect capacity
• Directly connected to more than 1,500 networks
with over 4,000 BGP connections
Decoupled DCs
1
• All nodes active, all nodes stateless
IT Capacity Unit = STAMP
• Separation of CPU’s Storage, SQL Services
• DC Capacity Unit or Workload Appliance
The green circles
represent Microsoft
Edge nodes live for the
Office 365 Portal.
There are many other Microsoft edge nodes
that are not yet compliant with Office 365.
EDGE
TCP
connect
SSL
connect
San Antonio DC
TTFB
Reusing existing
connections
Edge reuses connections between
to further improve performance
TTLB
Without Edge, entire
request over ISP’s network
Content
Time
EDGE
With Edge, Microsoft’s network
brought closer to the user
Client RTT
Server RTT
App
Latency
•
•
•
Brisbane
Australia
Kuala Lumpur
Malaysia
Los Angeles
USA
Melbourne
Australia
Amsterdam
Netherlands
Miami
USA
Perth
Australia
Auckland
New Zealand
New York
USA
Sydney
Australia
Wellington
New Zealand
Palo Alto
USA
Vienna
Austria
Moscow
Russia
San Jose
USA
Luxembourg
Belgium
Singapore
Singapore
Seattle
USA
Sao Paulo
Brazil
Stockholm
Sweden
Montreal
Canada
Zurich
Switzerland
Toronto
Canada
Taipei
Taiwan
Prague
Czech Republic
London
UK
Paris
France
Ashburn
USA
Frankfurt
Germany
Atlanta
USA
Hong Kong
Hong Kong
Boston
USA
Dublin
Ireland
Chicago
USA
Milan
Italy
Dallas
USA
Turin
Italy
Denver
USA
Tokyo
Japan
Honolulu
USA
Seoul
Korea
Las Vegas
USA
•
•
•
•
•
Some cities have multiple peering
points
Peering agreements with 1,500+
ISPs
Peering locations may be on-net or
off-net
Peering may involve physical
connection and/or routing
advertisements
Data as of July 2014 from
PeeringDB and is subject to change
Outlook, Outlook Web Access, OneDrive for Business
Native clients on
tablets, PCs,
& desktops
Native clients
on mobile devices
Browser-based
Clients also cache
with HTML5
Browser-based
mobile clients
Office 365 offers a wide variety of options across devices for customers to access the service
OWA uses HTML 5 Offline Application Caching if enabled in Offline Settings
Server
workloads
Microsoft
network
Microsoft
edge nodes
Internet
peering and
routing
Content
delivery
network
Customer
Internet
connection
Rich client
applications
•
•
•
•
•
•
•
•
•
•
•
•
•
http://aka.ms/tune
Exchange Online
• Similar to on-prem
SharePoint Online
• Estimates rely upon on-prem baselines
views of webpages, uploads/downloads of content
•• HTTPS
Cached mode
Lync
Online reduces impact and provides for latency tolerance (<325 mSec)
•• Document
Perfmon, Netmon,
top,with
etc. to
baseline
editing
Office
Web Apps or Office
IM is bursty,
latency to
tolerant,
and
verySP1
small
••Use
Definite
advantages
Outlook
2013
your
pilot
to
predict
new
requirements
•• Bursty,
but
latency
tolerant
Voice usesBandwidth
RTAudio Calculator
• Exchange
•• Without
no
real
way toto
estimate
50kbpsbaseline,
low, 80kbps
high,
autodetermination
•• http://gallery.technet.microsoft.com/Exchange-Client-Network-8af1bf00
Assumes
enough
time
order upgrades if
• Netmon
Video depends
upon resolution
to baseline
needed
• 280kbps
4000kbps
dependent
upon resolution
• Will
ramp low,
up as
morehigh,
content
is loaded
into SharePoint, MySites,
• Desktop sharing depends upon desktop resolution
•• OneDrive
Start
with
at least 20% head room
for Business
Peer-to-Peer versus Client-Server
••• SharePoint
bandwidth
Lync
Bandwidth
Calculator calculator
Evaluate
options
to conserve bandwidth
•
http://www.microsoft.com/en-us/download/details.aspx?id=19011
• http://go.microsoft.com/fwlink/?LinkId=517364
•
•
•
•
•
SMTP Traffic
Migrations
• After hours test mailbox
migrations to baseline

• Bandwidth consumed
• Average migration rate

• For a migration event,
the total bandwidth
consumed is X
X>(MB+MB1+…MBx)
• Will go away at end of
migrations
• HTTPS, as inbound
requests (downloads
from the hybrids)
Baseline SMTP in and out
at the existing edge
Production will increase
 2*baseline in + 2*baseline
out

During coexistence,
between on-premises
and Office 365 will add
additional
Administration

Traffic is negligible




HTTPS traffic
Latency tolerant
Bursty and intermittent
DirSync noticeable only
on first run





HTTPS traffic
Only deltas after initial sync
Every 3 hours (+ runtime)
Variable based on deltas
More during the day than at night
Planning Recommendations
1.
2.
3.
4.
5.
6.
7.
8.
Consider your current growth rates
Based on your current utilization, order
larger circuits
20% headroom is small. 33% is a more
common. 50% is conservative.
Buy lower CIR with higher burst to
accommodate needs
Use your 50GB mailboxes, not your
archives
Use Outlook 2013 SP1 to take advantage
of MAPI over HTTP and avoid RPC
Move files to SharePoint Online
Pilot Lync video before use
Any users who cannot use modern clients
in cached mode should use OWA
Ensure best use of Internet egress and DNS
Conservation Strategies
1.
2.
3.
4.
5.
6.
7.
8.
9.
WAN accelerators
Some customer scenarios have seen improved performance
Never a silver bullet and other measures should be investigated
Required to be disabled for debugging or support
Firewall IP address exceptions and URLs
Recommend to use domain name based filtering
IP Addresses not as quickly updated
Proxies
PAC files, CONNECT, and are they helping or hurting






Expect 2 days of troubleshooting on site for the performance troubleshooting engineer
Meet with the network engineer and get details of the network topology
Have network engineers on standby at other customer network locations
There are nuances you learn on site
You can do multiple tests simultaneously if on site
Network topology is easier to visualize when on site
Stage 1 Performance troubleshooting engineer not required
1.
•
•
•
•
•
2.
•
•
3.
•
4.
•
Stage 2 Performance troubleshooting engineer is required
5.
6.
7.
8.
http://technet.microsoft.com/en-us/library/office-365-system-requirements.aspx
http://technet.microsoft.com/en-us/library/cc179175(v=office.15).aspx
http://na1-fasttrack.cloudapp.net
http://em1-fasttrack.cloudapp.net
http://ap1-fasttrack.cloudapp.net
OFC-B411 Office 365 Network Performance Troubleshooting
OFC-B343 SharePoint Online Performance: Designing Your Pages to Be Fast
•
•
•
•
http://aka.ms/tune
http://aka.ms/tune
http://aka.ms/tunemva
http://channel9.msdn.com/Events/TechEd
www.microsoft.com/learning
http://microsoft.com/technet
http://developer.microsoft.com