Transcript Latency Lags Bandwidth (last ~20 years)

Latency Lags Bandwidth (last ~20 years)
• Performance Milestones
10000
1000
Relative
BW
100
Improve
ment
Disk
10
(Latency improvement
= Bandwidth improvement)
1
1
10
• Disk: 3600, 5400, 7200,
100
10000, 15000 RPM
Relative Latency Improvement
Bandwidth Rocks (1)
Latency Lags Bandwidth (last ~20 years)
• Performance Milestones
10000
1000
Relative
Memory
BW
100
Improve
ment
• Memory Module: 16bit plain
DRAM, 16b Page Mode
10
DRAM, 32b, 64b, 64b
(Latency improvement
SDRAM, 64b DDR SDRAM
= Bandwidth improvement)
1
• Disk: 3600, 5400, 7200,
1
10
100
10000, 15000 RPM
Disk
Relative Latency Improvement
Bandwidth Rocks (2)
Latency Lags Bandwidth (last ~20 years)
• Performance Milestones
10000
1000
Network
• Ethernet: 10Mb, 100Mb,
1000Mb, 10000 Mbit/sec
• Memory Module: 16bit plain
Disk
DRAM, 16b Page Mode
10
DRAM, 32b, 64b, 64b
(Latency improvement
SDRAM, 64b DDR SDRAM
= Bandwidth improvement)
1
• Disk: 3600, 5400, 7200,
1
10
100
10000, 15000 RPM
Relative
Memory
BW
100
Improve
ment
Relative Latency Improvement
Bandwidth Rocks (3)
Latency Lags Bandwidth (last ~20 years)
• Performance Milestones
Processor
• Processor: ‘286, ‘386, ‘486,
Network
Pentium, Pentium Pro,
1000
Pentium 4
Relative
Memory
• Ethernet: 10Mb, 100Mb,
BW
100
1000Mb, 10000 Mbit/sec
Improve
ment
• Memory Module: 16bit plain
Disk
DRAM, 16b Page Mode
10
DRAM, 32b, 64b, 64b
(Latency improvement
SDRAM, 64b DDR SDRAM
= Bandwidth improvement)
1
• Disk: 3600, 5400, 7200,
1
10
100
10000, 15000 RPM
10000
Relative Latency Improvement
Bandwidth Rocks (4)
Rule of Thumb for Latency Lagging BW
• In the time that bandwidth doubles,
latency improves only by factors
of 1.2 to 1.4
(and capacity approximately triples in the
time that bandwidth doubles)
Bandwidth Rocks (5)
Improvement in Time for BW to Double
CPU
Memory LAN Disk
Latency / Year
1.17
1.07 1.10 1.09
Capacity / Year
--
Bandwidth / Year
1.50
1.27 1.39 1.18
Years for 2X Bandwidth
1.7
2.9 2.1 4.2
Capacity Improvement
--
Latency Improvement
1.3
1.52
3.4
-- 1.28
--
2.8
1.2 1.2 1.4
Bandwidth Rocks (6)
Recent Improvement in Time for BW to Double
(Last 3 Performance Milestones)
CPU
Memory LAN Disk
Latency / Year
1.22
1.06 1.13 1.09
Capacity / Year
--
Bandwidth / Year
1.55
1.30 1.78 1.24
Years for 2X Bandwidth
1.6
2.7 1.2 3.2
Capacity Improvement
--
Latency Improvement
1.4
1.49
2.9
-- 1.37
--
2.8
1.2 1.2 1.3
Bandwidth Rocks (7)
6 Reasons Latency Lags Bandwidth
1. Latency helps BW, but not vice versa
•
•
Spinning disk faster improves both
bandwidth and rotational latency
Higher linear density helps BW, not Latency
2. Bandwidth hurts latency
•
Longer queues help BW, hurt Latency
3. Distance limits latency
•
•
Speed of light and computers on network
Size of DRAM block => long bit and word
lines => most of DRAM access time
Bandwidth Rocks (8)
6 Reasons Latency Lags Bandwidth
4. Moore’s Law helps BW more than
latency
•
•
Smaller, faster transistors communicate over
relatively longer lines: limits latency
Faster transistors, more transistors,
more pins all help BW
5. Software overhead hurts latency
•
Long messages amortize overhead;
overhead more for short messages
6. Bandwidth is easier to sell
•
E.g., 10 Gbit/s vs. 10 msec latency Ethernet
Bandwidth Rocks (9)
4 Ways to Cope with Latency Lags Bandwidth
1. Caching (Leveraging Capacity)
•
Processor caches, file cache, disk cache
2. Replication (Leveraging Capacity)
•
Read from nearest head in RAID,
Distributed caches in SMP, Distributed ISPs
3. Parallelism and Out-of-Order
Processing (Leveraging Bandwidth)
•
Processors, Disk controllers, Messages
4. Prediction (Leveraging Bandwidth)
•
Branches, Prefetching: disk, caches
Bandwidth Rocks (10)
Conclusion
• If everything improves at the same rate,
then nothing really changes
– When rates vary, we see real dislocation
that in turn requires real innovation
• Hardware and software developers
should plan that performance advances
will emphasize BW over latency
Bandwidth Rocks (11)