Transcript ppt

Lecture 2: Review of
Performance/Cost/Power Metrics
and Architectural Basics
Prof. Jan M. Rabaey
Computer Science 252
Spring 2000
“Computer Architecture in Cory Hall”
JR.S00 1
Review Lecture 1
• Class Organization
– Class Projects
• Trends in the Industry and Driving Forces
JR.S00 2
Computer Architecture Topics
Input/Output and Storage
Disks, WORM, Tape
DRAM
Memory
Hierarchy
L2 Cache
L1 Cache
VLSI
Instruction Set Architecture
RAID
Emerging Technologies
Interleaving
Bus protocols
Coherence,
Bandwidth,
Latency
Addressing,
Protection,
Exception Handling
Pipelining, Hazard Resolution,
Pipelining and Instruction
Superscalar, Reordering,
Level Parallelism
Prediction, Speculation,
JR.S00 3
Vector, VLIW, DSP, Reconfiguration
Computer Architecture Topics
P M
P M
S
°°°
P M
P M
Interconnection Network
Processor-Memory-Switch
Multiprocessors
Networks and Interconnections
Shared Memory,
Message Passing,
Data Parallelism
Network Interfaces
Topologies,
Routing,
Bandwidth,
Latency,
Reliability
JR.S00 4
The Secret of Architecture Design:
Measurement and Evaluation
Architecture Design is an iterative process:
• Searching the space of possible designs
Design
• At all levels of computer systems
Analysis
Creativity
Cost /
Performance
Analysis
Good Ideas
Bad Ideas
Mediocre Ideas
JR.S00 5
Computer Engineering Methodology
Implementation
Complexity
Implementation
Evaluate Existing
Systems for
Bottlenecks
Analysis
Benchmarks
Technology
Trends
Implement Next
Generation System
Simulate New
Designs and
Organizations
Workloads
Design
JR.S00 6
Measurement Tools
• Hardware: Cost, delay, area, power estimation
• Benchmarks, Traces, Mixes
• Simulation (many levels)
– ISA, RT, Gate, Circuit
• Queuing Theory
• Rules of Thumb
• Fundamental “Laws”/Principles
JR.S00 7
Review:
Performance, Cost, Power
JR.S00 8
Metric 1: Performance
In passenger-mile/hour
Plane
DC to Paris
Speed
Passengers
Throughput
Boeing 747
6.5 hours
610 mph
470
286,700
Concorde
3 hours
1350 mph
132
178,200
• Time to run the task
– Execution time, response time, latency
• Tasks per day, hour, week, sec, ns …
– Throughput, bandwidth
JR.S00 9
The Performance Metric
"X is n times faster than Y" means
ExTime(Y)
--------ExTime(X)
=
Performance(X)
--------------Performance(Y)
• Speed of Concorde vs. Boeing 747
• Throughput of Boeing 747 vs. Concorde
JR.S00 10
Amdahl's Law
Speedup due to enhancement E:
ExTime w/o E
Speedup(E) = ------------ExTime w/ E
=
Performance w/ E
------------------Performance w/o E
Suppose that enhancement E accelerates a fraction F
of the task by a factor S, and the remainder of the
task is unaffected
JR.S00 11
Amdahl’s Law
ExTimenew = ExTimeold x (1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced
Speedupoverall =
ExTimeold
ExTimenew
1
=
(1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced
JR.S00 12
Amdahl’s Law
• Floating point instructions improved to run 2X;
but only 10% of actual instructions are FP
ExTimenew = ExTimeold x (0.9 + .1/2) = 0.95 x ExTimeold
Speedupoverall =
1
0.95
=
1.053
Law of diminishing return:
Focus on the common case!
JR.S00 13
Metrics of Performance
Application
Answers per month
Operations per second
Programming
Language
Compiler
ISA
(millions) of Instructions per second: MIPS
(millions) of (FP) operations per second: MFLOP/s
Datapath
Control
Function Units
Transistors Wires Pins
Megabytes per second
Cycles per second (clock rate)
JR.S00 14
Aspects of CPU Performance
CPU time
= Seconds
= Instructions x
Program
Program
CPI
Program
Compiler
X
(X)
Inst. Set.
X
X
Technology
x Seconds
Instruction
Inst Count
X
Organization
Cycles
X
Cycle
Clock Rate
X
X
JR.S00 15
Cycles Per Instruction
“Average Cycles per Instruction”
CPI = Cycles / Instruction Count
= (CPU Time * Clock Rate) / Instruction Count
n
CPU time = CycleTime *

i =1
CPI
i
* I
i
“Instruction Frequency”
n
CPI =

i =1
CPI i *
F
i
where F i =
I i
Instruction Count
Invest Resources where time is Spent!
JR.S00 16
Example: Calculating CPI
Base Machine (Reg / Reg)
Op
Freq CPIi CPIi*Fi
ALU
50%
1
.5
Load
20%
2
.4
Store
10%
2
.2
Branch
20%
2
.4
1.5
(% Time)
(33%)
(27%)
(13%)
(27%)
Typical Mix
JR.S00 17
Creating Benchmark Sets
•
•
•
•
Real programs
Kernels
Toy benchmarks
Synthetic benchmarks
– e.g. Whetstones and Dhrystones
JR.S00 18
SPEC: System Performance Evaluation
Cooperative
• First Round 1989
– 10 programs yielding a single number (“SPECmarks”)
• Second Round 1992
– SPECInt92 (6 integer programs) and SPECfp92 (14 floating point
programs)
» Compiler Flags unlimited. March 93 of DEC 4000 Model 610:
spice: unix.c:/def=(sysv,has_bcopy,”bcopy(a,b,c)=
memcpy(b,a,c)”
wave5: /ali=(all,dcom=nat)/ag=a/ur=4/ur=200
nasa7: /norecu/ag=a/ur=4/ur2=200/lc=blas
• Third Round 1995
– new set of programs: SPECint95 (8 integer programs) and
SPECfp95 (10 floating point)
– “benchmarks useful for 3 years”
– Single flag setting for all programs: SPECint_base95,
SPECfp_base95
JR.S00 19
How to Summarize Performance
• Arithmetic mean (weighted arithmetic mean)
tracks execution time: (Ti)/n or (Wi*Ti)
• Harmonic mean (weighted harmonic mean) of
rates (e.g., MFLOPS) tracks execution time:
n/ (1/Ri) or n/(Wi/Ri)
• Normalized execution time is handy for scaling
performance (e.g., X times faster than
SPARCstation 10)
– Arithmetic mean impacted by choice of reference machine
• Use the geometric mean for comparison:
(Ti)^1/n
– Independent of chosen machine
– but not good metric for total execution time
JR.S00 20
SPEC First Round
• One program: 99% of time in single line of code
• New front-end compiler could improve dramatically
800
700
500
400
300
200
100
tomcatv
fpppp
matrix300
eqntott
li
nasa7
doduc
spice
epresso
0
gcc
SPEC Perf
600
Benchmark
IBM Powerstation 550 for 2 different compilers
JR.S00 21
Impact of Means
on SPECmark89 for IBM 550
(without and with special compiler option)
Ratio to VAX:
Program
gcc
espresso
spice
doduc
nasa7
li
eqntott
matrix300
fpppp
tomcatv
Mean
Time:
Before After Before After
30
29
49
51
35
34
65
67
47
47
510 510
46
49
41
38
78 144
258 140
34
34
183 183
40
40
28
28
78 730
58
6
90
87
34
35
33 138
20
19
54
72
124 108
Geometric
Ratio
1.33
Ratio
1.16
Weighted Time:
Before After
8.91
9.22
7.64
7.86
5.69
5.69
5.81
5.45
3.43
1.86
7.86
7.86
6.68
6.68
3.43
0.37
2.97
3.07
2.01
1.94
54.42 49.99
Arithmetic
Weighted
Arith.
JR.S00 22
Ratio
1.09
Performance Evaluation
• “For better or worse, benchmarks shape a field”
• Good products created when have:
– Good benchmarks
– Good ways to summarize performance
• Given sales is a function in part of performance
relative to competition, investment in improving
product as reported by performance summary
• If benchmarks/summary inadequate, then choose
between improving product for real programs vs.
improving product to get more sales;
Sales almost always wins!
• Execution time is the measure of computer
performance!
JR.S00 23
Integrated Circuits Costs
IC cost 
Die cost 
Die cost  Testing cost  Packaging cost
Final test yield
Wafer cost
Dies per Wafer  Die yield
 (Wafer_dia m/2)2
  Wafer_diam
Dies per wafer 

 Test_Die
Die_Area
2  Die_Area




 Defect_Den sity  Die_area  
Die Yield  Wafer_yiel d  1  
 


 



Die Cost goes roughly with die area4
JR.S00 24
Real World Examples
Chip
Metal Line Wafer Defect Area Dies/ Yield Die Cost
layers width cost
/cm2 mm2 wafer
386DX
2 0.90 $900
1.0
43 360 71%
$4
486DX2
3 0.80 $1200
1.0
81 181 54%
$12
PowerPC 601 4 0.80 $1700
1.3 121 115 28%
$53
HP PA 7100 3 0.80 $1300
1.0 196
66 27%
$73
DEC Alpha
3 0.70 $1500
1.2 234
53 19%
$149
SuperSPARC 3 0.70 $1700
1.6 256
48 13%
$272
Pentium
3 0.80 $1500
1.5 296
40 9%
$417
– From "Estimating IC Manufacturing Costs,” by Linley Gwennap,
Microprocessor Report, August 2, 1993, p. 15
JR.S00 25
Cost/Performance
What is Relationship of Cost to Price?
• Recurring Costs
– Component Costs
– Direct Costs (add 25% to 40%) recurring costs: labor, purchasing, scrap,
warranty
• Non-Recurring Costs or Gross Margin (add 82% to
186%)
(R&D, equipment maintenance, rental, marketing, sales, financing
cost, pretax profits, taxes
• Average Discount to get List Price (add 33% to 66%): volume
discounts and/or retailer markup
List Price
Avg. Selling Price
Average
Discount
Gross
Margin
Direct Cost
Component
Cost
25% to 40%
34% to 39%
6% to 8%
15% to 33%
JR.S00 26
Chip Prices (August 1993)
• Assume purchase 10,000 units
Chip
386DX
Area Mfg. Price Multi- Comment
mm2
cost
43
$9
486DX2
81
PowerPC 601 121
plier
$31
$35 $245
$77 $280
3.4 Intense Competition
7.0 No Competition
3.6
DEC Alpha
234 $202 $1231
6.1 Recoup R&D?
Pentium
296 $473 $965
2.0 Early in shipments
JR.S00 27
Summary: Price vs. Cost
100%
80%
Average Discount
60%
Gross Margin
40%
Direct Costs
20%
Component Costs
0%
Mini
5
4
W/S
PC
4.7
3.5
3.8
Average Discount
2.5
3
Gross Margin
1.8
2
Direct Costs
1.5
1
Component Costs
0
Mini
W/S
PC
JR.S00 28
Power/Energy
100
Pentium Pro
(R)
Pentium(R)
10
386
386

Pentium(R)
MMX
486
486
1
?
Source: Intel
Max Power (Watts)
Pentium II (R)



   
 Lead processor power increases every generation
 Compactions provide higher performance at lower power
JR.S00 29
Energy/Power
• Power dissipation: rate at which energy is
taken from the supply (power source) and
transformed into heat
P = E/t
• Energy dissipation for a given instruction
depends upon type of instruction (and state
of the processor)
n
P = (1/CPU Time) *

i =1
E
i
* I
i
JR.S00 30
The Energy-Flexibility Gap
Energy Efficiency
MOPS/mW (or MIPS/mW)
1000
Dedicated
HW
100
10
Reconfigurable
Processor/Logic
ASIPs
DSPs
Pleiades
10-80 MOPS/mW
2 V DSP: 3 MOPS/mW
1
Embedded Processors
SA110
0.4 MIPS/mW
0.1
Flexibility (Coverage)
JR.S00 31
Summary, #1
• Designing to Last through Trends
Capacity
Logic
•
2x in 3 years
Speed
2x in 3 years
SPEC RATING:
2x in 1.5 years
DRAM
4x in 3 years
2x in 10 years
Disk
4x in 3 years
2x in 10 years
6yrs to graduate => 16X CPU speed, DRAM/Disk size
• Time to run the task
–
Execution time, response time, latency
• Tasks per day, hour, week, sec, ns, …
–
Throughput, bandwidth
• “X is n times faster than Y” means
ExTime(Y)
--------ExTime(X)
=
Performance(X)
-------------Performance(Y)
JR.S00 32
Summary, #2
• Amdahl’s Law:
Speedupoverall =
ExTimeold
ExTimenew
1
=
(1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced
• CPI Law:
CPU time
= Seconds
Program
= Instructions x
Program
Cycles
x Seconds
Instruction
Cycle
• Execution time is the REAL measure of computer
performance!
• Good products created when have:
– Good benchmarks, good ways to summarize performance
• Different set of metrics apply to embedded
systems
JR.S00 33
Review:
Instruction Sets, Pipelines, and Caches
JR.S00 34
Computer Architecture Is …
the attributes of a [computing] system as seen
by the programmer, i.e., the conceptual
structure and functional behavior, as distinct
from the organization of the data flows and
controls the logic design, and the physical
implementation.
Amdahl, Blaaw, and Brooks, 1964
JR.S00 35
Computer Architecture’s Changing
Definition
• 1950s to 1960s:
Computer Architecture Course = Computer Arithmetic
• 1970s to mid 1980s:
Computer Architecture Course = Instruction Set
Design, especially ISA appropriate for compilers
• 1990s:
Computer Architecture Course = Design of CPU,
memory system, I/O system, Multiprocessors
JR.S00 36
Computer Architecture is ...
Instruction Set Architecture
Organization
Hardware
JR.S00 37
Instruction Set Architecture (ISA)
software
instruction set
hardware
JR.S00 38
Interface Design
A good interface:
• Lasts through many implementations (portability,
compatability)
• Is used in many differeny ways (generality)
• Provides convenient functionality to higher levels
• Permits an efficient implementation at lower levels
use
use
use
Interface
imp 1
time
imp 2
imp 3
JR.S00 39
Evolution of Instruction Sets
Single Accumulator (EDSAC 1950)
Accumulator + Index Registers
(Manchester Mark I, IBM 700 series 1953)
Separation of Programming Model
from Implementation
High-level Language Based
(B5000 1963)
Concept of a Family
(IBM 360 1964)
General Purpose Register Machines
Complex Instruction Sets
(Vax, Intel 432 1977-80)
Load/Store Architecture
(CDC 6600, Cray 1 1963-76)
RISC
(Mips,Sparc,HP-PA,IBM RS6000,PowerPC . . .1987)
LIW/”EPIC”? (IA-64. . .1999)
JR.S00 40
Evolution of Instruction Sets
• Major advances in computer architecture are
typically associated with landmark instruction
set designs
– Ex: Stack vs GPR (System 360)
• Design decisions must take into account:
–
–
–
–
–
–
technology
machine organization
programming languages
compiler technology
operating systems
applications
• And they in turn influence these
JR.S00 41
A "Typical" RISC
•
•
•
•
32-bit fixed format instruction (3 formats I,R,J)
32 32-bit GPR (R0 contains zero, DP take pair)
3-address, reg-reg arithmetic instruction
Single address mode for load/store:
base + displacement
– no indirection
• Simple branch conditions (based on register values)
• Delayed branch
see: SPARC, MIPS, HP PA-Risc, DEC Alpha, IBM PowerPC,
CDC 6600, CDC 7600, Cray-1, Cray-2, Cray-3
JR.S00 42
Example: MIPS ( DLX)
Register-Register
31
26 25
Op
21 20
Rs1
16 15
Rs2
Rd
11 10
6 5
0
Opx
Register-Immediate
31
26 25
Op
21 20
Rs1
16 15
0
immediate
Rd
Branch
31
26 25
Op
Rs1
21 20
16 15
Rs2/Opx
0
immediate
Jump / Call
31
26 25
Op
0
target
JR.S00 43
Pipelining: Its Natural!
• Laundry Example
• Ann, Brian, Cathy, Dave
each have one load of clothes
to wash, dry, and fold
• Washer takes 30 minutes
A
B
C
D
• Dryer takes 40 minutes
• “Folder” takes 20 minutes
JR.S00 44
Sequential Laundry
6 PM
7
8
9
10
11
Midnight
Time
30 40 20 30 40 20 30 40 20 30 40 20
T
a
s
k
A
B
O
r
d
e
r
C
D
• Sequential laundry takes 6 hours for 4 loads
• If they learned pipelining, how long would laundry take?
JR.S00 45
Pipelined Laundry
Start work ASAP
6 PM
7
8
9
10
11
Midnight
Time
30 40
T
a
s
k
40
40
40 20
A
B
O
r
d
e
r
C
D
• Pipelined laundry takes 3.5 hours for 4 loads
JR.S00 46
Pipelining Lessons
6 PM
7
8
9
Time
T
a
s
k
O
r
d
e
r
30 40
A
B
C
D
40
40
40 20
• Pipelining doesn’t help
latency of single task, it
helps throughput of
entire workload
• Pipeline rate limited by
slowest pipeline stage
• Multiple tasks operating
simultaneously
• Potential speedup =
Number pipe stages
• Unbalanced lengths of
pipe stages reduces
speedup
• Time to “fill” pipeline and
time to “drain” it reduces
speedup
JR.S00 47
Computer Pipelines
• Execute billions of instructions, so
throughout is what matters
• DLX desirable features: all instructions same
length, registers located in same place in
instruction format, memory operands only in
loads or stores
JR.S00 48
5 Steps of DLX Datapath
Figure 3.1, Page 130
Instruction
Fetch
Instr. Decode
Reg. Fetch
Execute
Addr. Calc
Next SEQ PC
Adder
4
Zero?
RS1
L
M
D
MUX
Data
Memory
ALU
Imm
MUX MUX
RD
Reg File
Inst
Memory
Address
RS2
Write
Back
MUX
Next PC
Memory
Access
Sign
Extend
WB Data
JR.S00 49
5 Steps of DLX Datapath
Figure 3.4, Page 134
Execute
Addr. Calc
Instr. Decode
Reg. Fetch
Next SEQ PC
Next SEQ PC
Adder
4
Zero?
RS1
MUX
MEM/WB
Data
Memory
EX/MEM
ALU
MUX MUX
ID/EX
Imm
Reg File
IF/ID
Memory
Address
RS2
Write
Back
MUX
Next PC
Memory
Access
WB Data
Instruction
Fetch
Sign
Extend
RD
RD
RD
• Data stationary control
– local decode for each instruction phase / pipeline stage
JR.S00 50
Visualizing Pipelining
Figure 3.3, Page 133
Time (clock cycles)
Reg
DMem
Ifetch
Reg
DMem
Reg
ALU
DMem
Reg
ALU
O
r
d
e
r
Ifetch
ALU
I
n
s
t
r.
ALU
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
Ifetch
Ifetch
Reg
Reg
Reg
DMem
Reg
JR.S00 51
Its Not That Easy for Computers
• Limits to pipelining: Hazards prevent next
instruction from executing during its designated
clock cycle
– Structural hazards: HW cannot support this combination of
instructions - two dogs fighting for the same bone
– Data hazards: Instruction depends on result of prior
instruction still in the pipeline
– Control hazards: Caused by delay between the fetching of
instructions and decisions about changes in control flow
(branches and jumps).
JR.S00 52
One Memory Port/Structural Hazards
Figure 3.6, Page 142
Time (clock cycles)
Reg
DMem
Reg
DMem
Reg
DMem
Reg
ALU
Instr 4
Ifetch
ALU
Instr 3
DMem
ALU
O
r
d
e
r
Instr 2
Reg
ALU
I Load Ifetch
n
s
Instr 1
t
r.
ALU
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
Ifetch
Ifetch
Reg
Ifetch
Reg
Reg
Reg
DMem
Reg
JR.S00 53
One Memory Port/Structural Hazards
Figure 3.7, Page 143
Time (clock cycles)
Stall
Instr 3
DMem
Ifetch
Reg
DMem
Reg
ALU
Ifetch
Bubble
Reg
Reg
DMem
Bubble Bubble
Ifetch
Reg
Reg
Bubble
ALU
O
r
d
e
r
Instr 2
Reg
ALU
I Load Ifetch
n
s
Instr 1
t
r.
ALU
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
Bubble
DMem
Reg
JR.S00 54
Speed Up Equation for Pipelining
CPIpipelined  Ideal CPI  Average Stall cycles per Inst
Cycle Timeunpipelined
Ideal CPI  Pipeline depth
Speedup 

Ideal CPI  Pipeline stall CPI
Cycle Timepipelined
For simple RISC pipeline, CPI = 1:
Cycle Timeunpipelined
Pipeline depth
Speedup 

1  Pipeline stall CPI
Cycle Timepipelined
JR.S00 55
Example: Dual-port vs. Single-port
• Machine A: Dual ported memory (“Harvard Architecture”)
• Machine B: Single ported memory, but its pipelined
implementation has a 1.05 times faster clock rate
• Ideal CPI = 1 for both
• Loads are 40% of instructions executed
SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe)
= Pipeline Depth
SpeedUpB = Pipeline Depth/(1 + 0.4 x 1) x (clockunpipe/(clockunpipe / 1.05)
= (Pipeline Depth/1.4) x 1.05
= 0.75 x Pipeline Depth
SpeedUpA / SpeedUpB = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33
• Machine A is 1.33 times faster
JR.S00 56
Data Hazard on R1
Figure 3.9, page 147
Time (clock cycles)
and r6,r1,r7
or
r8,r1,r9
xor r10,r1,r11
Ifetch
DMem
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
ALU
sub r4,r1,r3
Reg
ALU
Ifetch
ALU
O
r
d
e
r
add r1,r2,r3
WB
ALU
I
n
s
t
r.
MEM
ALU
IF ID/RF EX
Reg
Reg
Reg
Reg
DMem
JR.S00 57
Reg
Three Generic Data Hazards
• Read After Write (RAW)
InstrJ tries to read operand before InstrI writes it
I: add r1,r2,r3
J: sub r4,r1,r3
• Caused by a “Dependence” (in compiler
nomenclature). This hazard results from an actual
need for communication.
JR.S00 58
Three Generic Data Hazards
• Write After Read (WAR)
InstrJ writes operand before InstrI reads it
I: sub r4,r1,r3
J: add r1,r2,r3
K: mul r6,r1,r7
• Called an “anti-dependence” by compiler writers.
This results from reuse of the name “r1”.
• Can’t happen in DLX 5 stage pipeline because:
– All instructions take 5 stages, and
– Reads are always in stage 2, and
– Writes are always in stage 5
JR.S00 59
Three Generic Data Hazards
• Write After Write (WAW)
InstrJ writes operand before InstrI writes it.
I: sub r1,r4,r3
J: add r1,r2,r3
K: mul r6,r1,r7
• Called an “output dependence” by compiler writers
This also results from the reuse of name “r1”.
• Can’t happen in DLX 5 stage pipeline because:
– All instructions take 5 stages, and
– Writes are always in stage 5
• Will see WAR and WAW in later more complicated
pipes
JR.S00 60
Forwarding to Avoid Data Hazard
Figure 3.10, Page 149
or
r8,r1,r9
xor r10,r1,r11
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
ALU
and r6,r1,r7
Ifetch
DMem
ALU
sub r4,r1,r3
Reg
ALU
O
r
d
e
r
add r1,r2,r3 Ifetch
ALU
I
n
s
t
r.
ALU
Time (clock cycles)
Reg
Reg
Reg
Reg
DMem
Reg
JR.S00 61
HW Change for Forwarding
Figure 3.20, Page 161
NextPC
mux
MEM/WR
EX/MEM
ALU
mux
ID/EX
Registers
mux
Immediate
Data
Memory
JR.S00 62
Data Hazard Even with Forwarding
Figure 3.12, Page 153
and r6,r1,r7
or
r8,r1,r9
DMem
Ifetch
Reg
DMem
Reg
Ifetch
Ifetch
Reg
Reg
Reg
DMem
ALU
O
r
d
e
r
sub r4,r1,r6
Reg
ALU
lw r1, 0(r2) Ifetch
ALU
I
n
s
t
r.
ALU
Time (clock cycles)
Reg
DMem
Reg
JR.S00 63
Data Hazard Even with Forwarding
Figure 3.13, Page 154
and r6,r1,r7
or r8,r1,r9
Reg
DMem
Ifetch
Reg
Bubble
Ifetch
Bubble
Reg
Bubble
Ifetch
Reg
DMem
Reg
Reg
DMem
ALU
sub r4,r1,r6
Ifetch
ALU
O
r
d
e
r
lw r1, 0(r2)
ALU
I
n
s
t
r.
ALU
Time (clock cycles)
Reg
DMem
JR.S00 64
Software Scheduling to Avoid Load
Hazards
Try producing fast code for
a = b + c;
d = e – f;
assuming a, b, c, d ,e, and f in memory.
Slow code:
LW
LW
ADD
SW
LW
LW
SUB
SW
Rb,b
Rc,c
Ra,Rb,Rc
a,Ra
Re,e
Rf,f
Rd,Re,Rf
d,Rd
Fast code:
LW
LW
LW
ADD
LW
SW
SUB
SW
Rb,b
Rc,c
Re,e
Ra,Rb,Rc
Rf,f
a,Ra
Rd,Re,Rf
d,Rd
JR.S00 65
22: add r8,r1,r9
36: xor r10,r1,r11
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
DMem
Ifetch
Reg
ALU
r6,r1,r7
Ifetch
DMem
ALU
18: or
Reg
ALU
14: and r2,r3,r5
Ifetch
ALU
10: beq r1,r3,36
ALU
Control Hazard on Branches
Three Stage Stall
Reg
Reg
Reg
Reg
DMem
JR.S00 66
Reg
Branch Stall Impact
• If CPI = 1, 30% branch,
Stall 3 cycles => new CPI = 1.9!
• Two part solution:
– Determine branch taken or not sooner, AND
– Compute taken branch address earlier
• DLX branch tests if register = 0 or  0
• DLX Solution:
– Move Zero test to ID/RF stage
– Adder to calculate new PC in ID/RF stage
– 1 clock cycle penalty for branch versus 3
JR.S00 67
Pipelined DLX Datapath
Figure 3.22, page 163
Instruction
Fetch
Instr. Decode
Reg. Fetch
Execute
Addr. Calc.
Memory
Access
Write
Back
This is the correct 1 cycle
latency implementation!
JR.S00 68
Four Branch Hazard Alternatives
#1: Stall until branch direction is clear
#2: Predict Branch Not Taken
–
–
–
–
–
Execute successor instructions in sequence
“Squash” instructions in pipeline if branch actually taken
Advantage of late pipeline state update
47% DLX branches not taken on average
PC+4 already calculated, so use it to get next instruction
#3: Predict Branch Taken
– 53% DLX branches taken on average
– But haven’t calculated branch target address in DLX
» DLX still incurs 1 cycle branch penalty
» Other machines: branch target known before outcome
JR.S00 69
Four Branch Hazard Alternatives
#4: Delayed Branch
– Define branch to take place AFTER a following instruction
branch instruction
sequential successor1
sequential successor2
........
sequential successorn
branch target if taken
Branch delay of length n
– 1 slot delay allows proper decision and branch target
address in 5 stage pipeline
– DLX uses this
JR.S00 70
Delayed Branch
• Where to get instructions to fill branch delay slot?
–
–
–
–
Before branch instruction
From the target address: only valuable when branch taken
From fall through: only valuable when branch not taken
Cancelling branches allow more slots to be filled
• Compiler effectiveness for single branch delay slot:
– Fills about 60% of branch delay slots
– About 80% of instructions executed in branch delay slots useful
in computation
– About 50% (60% x 80%) of slots usefully filled
• Delayed Branch downside: 7-8 stage pipelines,
multiple instructions issued per clock (superscalar)
JR.S00 71
Evaluating Branch Alternatives
Pipeline speedup =
Scheduling
Branch
scheme
penalty
Stall pipeline
3
Predict taken
1
Predict not taken
1
Delayed branch
0.5
Pipeline depth
1 +Branch frequency Branch penalty
CPI
1.42
1.14
1.09
1.07
speedup v.
unpipelined
3.5
4.4
4.5
4.6
speedup v.
stall
1.0
1.26
1.29
1.31
Conditional & Unconditional = 14%, 65% change PC
JR.S00 72
Summary :
Control and Pipelining
• Just overlap tasks; easy if tasks are independent
• Speed Up  Pipeline Depth; if ideal CPI is 1, then:
Cycle Timeunpipelined
Pipeline depth
Speedup 

1  Pipeline stall CPI
Cycle Timepipelined
• Hazards limit performance on computers:
– Structural: need more HW resources
– Data (RAW,WAR,WAW): need forwarding, compiler scheduling
– Control: delayed branch, prediction
JR.S00 73