Transcript t Gooi - UCSD VLSI CAD Laboratory

Design Cost Modeling and Data
Collection Infrastructure
Andrew B. Kahng and Stefanus Mantik*
UCSD CSE and ECE Departments
(*) Cadence Design Systems, Inc.
http://vlsicad.ucsd.edu/
ITRS Design Cost Model
 Engineer cost/year increases 5% / year ($181,568 in 1990)
 EDA tool cost/year (per engineer) increases 3.9% / year
 Productivity due to 8 major Design Technology
innovations
 RTL methodology
…
 Large-block reuse
 IC implementation suite
 Intelligent testbench
 Electronic System-level methodology
 Matched up against SOC-LP PDA content:
 SOC-LP PDA design cost = $20M in 2003
 Would have been $630M without EDA innovations
SOC Design Cost
Very Large Block Reuse
ES Level Methodology
Intelligent Testbench
IC Implementation tools
Large Block Reuse
Small Block Reuse
$10,000,000,000
Tall Thin Engineer
In house P&R
$100,000,000,000
Total Design Cost
$1,000,000,000
629,769,273
$100,000,000
20,152,617
RTL Methodology Only
With All Future Improvements
$10,000,000
1985
1990
1995
2000
2005
Year
2010
2015
2020
Outline
 Introduction and motivations
 METRICS system architecture
 Design quality metrics and tool quality metrics
 Applications of the METRICS system
 Issues and conclusions
Motivations
 How do we improve design productivity ?
 Is our design technology / capability better than last year?
 How do we formally capture best known methods, and how
do we identify them in the first place ?
 Does our design environment support continuous
improvement, exploratory what-if design, early predictions
of success / failure, ...?
 Currently, no standards or infrastructure for measuring
and recording the semiconductor design process
 Can
benefit project management
 accurate resource prediction at any point in design cycle
 accurate project post-mortems
 Can benefit tool R&D
 feedback on tool usage and parameters used
 improved benchmarking
Fundamental Gaps
 Data to be measured is not available
 Data
is only available through tool log files
 Metrics naming and semantics are not consistent among
different tools
 We do not always know what data should be measured
 Some
metrics are less obviously useful
 Other metrics are almost impossible to discern
Purpose of METRICS
 Standard infrastructure for the collection and the
storage of design process information
 Standard list of design metrics and process metrics
 Analyses and reports that are useful for design
process optimization
METRICS allows: Collect, Data-Mine,
Measure, Diagnose, then Improve
Outline
 Introduction and motivations
 METRICS system architecture
 Components
of METRICS System
 Flow tracking
 METRICS Standard
 Design quality metrics and tool quality metrics
 Applications of the METRICS system
 Issues and conclusions
METRICS System Architecture
Tool
Transmitter
Tool
Tool
Transmitter
Transmitter
wrapper
Java
Applets
API
XML
Inter/Intra-net
Web
Server
DB
DAC00
Reporting
Data
Mining
Metrics Data Warehouse
METRICS Server
Internet/Intranet
Input
Form
Receiver
Servlet
Reporting
Servlet
Data translator
Apache + Servlet
Receiver
Java Beans Decryptor
DB
JDBC
Dataminer
XML Parser
Reporting
External
Interface
Example Reports
nexus12 2%
nexus11 2%
nexus10 1%
nexus4 95%
% aborted per machine
synthesis ATPG
22%
20%
postSyntTA
13%
placedTA
physical
7%
18%
BA 8%
funcSim
7%
LVS 5%
% aborted per task
CPU_TIME = 12 + 0.027 NUM_CELLS
Correlation = 0.93
Flow Tracking
Task sequence: T1, T2, T1, T2, T3, T3, T3, T4, T2, T1, T2, T4
S
T1
T1
T2
T2
T3
T1
T2
T3
T2
T3
T4
T4
F
Run Current TASK_NO
FLOW_SEQUENCE
No
Task T1 T2 T3 T4
1
T1
1 - - 1
2
T2
1 1 - 1/1
3
T1
2 - - 2
4
T2
2 1 - 2/1
5
T3
2 1 1 2/1/1
6
T3
2 1 2 2/1/2
7
T3
2 1 3 2/1/3
8
T4
2 1 3 1
2/1/3/1
9
T2
2 2 - 2/2
10
T1
3 - - 3
11
T2
3 1 - 3/1
12
T4
3 1 - 1
3/1/0/1
Testbeds: Metricized P&R Flow
DEF
Capo Placer
Placed DEF
UCLA + Cadence flow
LEF
Legal DEF
QP
QP ECO
Placed DEF
Incr.
WRoute
Routed DEF
Congestion
Map
Incr WRoute
Cadence PKS flow
Final DEF
CongestionAnalysis
Synthesis & Tech Map
Pre-placement Opt
DEF
LEF
GCF,TLF
CTGen
M
E
T
R
I
C
S
Clocked DEF
Constraints
QP Opt
Optimized DEF
WRoute
Pearl
Routed DEF
QP
Post-placement Opt
Ambit
PKS
GRoute
WRoute
Cadence SLC flow
METRICS Standards
 Standard metrics naming across tools
name  same meaning, independent of tool
supplier
 generic metrics and tool-specific metrics
 no more ad hoc, incomparable log files
 same
 Standard schema for metrics database
 Standard middleware for database interface
Generic and Specific Tool Metrics
Generic Tool Metrics
tool_name
tool_version
tool_vendor
compiled_date
start_time
end_time
tool_user
host_name
host_id
cpu_type
os_name
os_version
cpu_time
string
string
string
mm/dd/yyyy
hh:mm:ss
hh:mm:ss
string
string
string
string
string
string
hh:mm:ss
Placement Tool Metrics
num_cells
num_nets
layout_size
row_utilization
wirelength
weighted_wl
integer
integer
double
double
double
double
Routing Tool Metrics
num_layers
integer
num_violations integer
num_vias
integer
wirelength
double
wrong-way_wl double
max_congestion double
Partial list of metrics now being collected in Oracle8i
Open Source Architecture
 METRICS components are industry standards
 e.g.,
Oracle 8i, Java servlets, XML, Apache web server,
PERL/TCL scripts, etc.
 Custom generated codes for wrappers and APIs are
publicly available
 collaboration
in development of wrappers and APIs
 porting to different operating systems
 Codes are available at:
http://www.gigascale.org/metrics
Outline
 Introduction and motivations
 METRICS system architecture
 Design quality metrics and tool quality metrics
 Applications of the METRICS system
 Issues and conclusions
Tool Quality Metric: Behavior in the
Presence of Input Noise [ISQED02]
 Goal: tool predictability
 Ideal
scenario: can predict final solution quality even
before running the tool
 Requires understanding of tool behavior
 Heuristic nature of tool: predicting results is difficult
 Lower bound on prediction accuracy: inherent tool
noise
 Input noise  "insignificant" variations in input data
(sorting, scaling, naming, ...) that can nevertheless
affect solution quality
 Goal: understand how tools behave in presence of
noise, and possibly exploit inherent tool noise
Monotone Behavior
 Monotonicity
solutions w.r.t. inputs
Quality
Quality
 monotone
Parameter
Parameter
Monotonicity Studies
 OptimizationLevel: 1(fast/worst) … 10(slow/best)
3
10
2.5
0
-10
1
2
3
4
5
6
7
8
9
2
-20
1.5
-30
1
-40
0.5
QP CPU
Total CPU
-50
QP WL
WR WL
0
-0.5
-60
1
2
3
4
5
6
7
8
Opt Level
1
2
3
4
5
6
7
8
9
QP WL
2.50
0.97
-0.20
-0.11
1.43
0.58
1.29
0.64
1.70
QP CPU
-59.7
-51.6
-40.4
-39.3
-31.5
-31.3
-17.3
-11.9
-6.73
WR WL
2.95
1.52
-0.29
0.07
1.59
0.92
0.89
0.94
1.52
Total CPU
4.19
-6.77
-16.2
-15.2
-7.23
-10.6
-6.99
-3.75
-0.51
9
 Note: OptimizationLevel is the tool's own knob for "effort"; it may or may
not be well-conceived with respect to the underlying heuristics (bottom
line is that the tool behavior is "non-monotone" from user viewpoint)
Noise Studies: Random Seeds
 200 runs with different random seeds
 ½-percent
spread in solution quality due to random seed
35
-0.05%
30
20
15
10
5
% Quality Loss
0.
12
0.
08
0.
04
0
-0
.0
4
-0
.0
8
-0
.1
2
-0
.1
6
-0
.2
0
-0
.2
4
# Run
25
Noise: Random Ordering & Naming
 Data sorting  no effect on reordering
 Five naming perturbation

random cell names without hierarchy (CR)



random net names without hierarchy (NR)
random cell names with hierarchy (CH)



E.g., AFDX|CTRL|AX239  CELL00134
E.g., AFDX|CTRL|AX129  ID012|ID79|ID216
random net names with hierarchy (NH)
random master cell names (MC)

E.g., NAND3X4  MCELL0123
Noise: Random Naming (contd.)
 Wide range of variations (±3%)
 Hierarchy matters
16
CR
NR
NR
CH
NH
NH
MC
14
Number of Runs
12
10
8
6
4
2
0
-4
-2
0
2
% Quality Loss
4
6
Noise: Hierarchy
 Swap hierarchy

AA|BB|C03  XX|YY|C03
XX|YY|Z12  AA|BB|Z12
10
Number of Runs

8
6
4
2
0
-1
1
3
5
7
% Quality Loss
9
11
13
Outline
 Introduction and motivations
 METRICS system architecture
 Design quality and tool quality
 Applications of the METRICS system
 Issues and conclusions
Categories of Collected Data
 Design instances and design parameters
 attributes
and metrics of the design instances
 e.g., number of gates, target clock frequency, number of
metal layers, etc.
 CAD tools and invocation options
 list
of tools and user options that are available
 e.g., tool version, optimism level, timing driven option,
etc.
 Design solutions and result qualities
 qualities
of the solutions obtained from given tools and
design instances
 e.g., number of timing violations, total tool runtime,
layout area, etc.
Three Basic Application Types
 Design instances and design parameters
 CAD tools and invocation options
 Design solutions and result qualities
 Given  and , estimate the expected quality of 
 e.g.,
runtime predictions, wirelength estimations, etc.
 Given  and , find the appropriate setting of 
 e.g.,
best value for a specific option, etc.
 Given  and , identify the subspace of  that is
“doable” for the tool
 e.g.,
category of designs that are suitable for the given
tools, etc.
Estimation of QP CPU and Wirelength
 Goal:
 Estimate
QPlace runtime for CPU budgeting and block
partition
 Estimate placement quality (total wirelength)
 Collect QPlace metrics from 2000+ regression logfiles
 Use data mining (Cubist 1.07) to classify and predict, e.g.:



Rule 1: [101 cases, mean 334.3, range 64 to 3881, est err 276.3] if
ROW_UTILIZATION <= 76.15 then CPU_TIME = -249 + 6.7 ROW_UTILIZATION +
55 NUM_ROUTING_LAYER - 14 NUM_LAYER
Rule 2: [168 cases, mean 365.7, range 20 to 5352, est err 281.6] if
NUM_ROUTING_LAYER <= 4 then CPU_TIME = -1153 + 192
NUM_ROUTING_LAYER + 12.9 ROW_UTILIZATION - 49 NUM_LAYER
Rule 3: [161 cases, mean 795.8, range 126 to 1509, est err 1069.4] if
NUM_ROUTING_LAYER > 4 and ROW_UTILIZATION > 76.15 then CPU_TIME = -33
+ 8.2 ROW_UTILIZATION + 55 NUM_ROUTING_LAYER - 14 NUM_LAYER
 Data mining limitation  sparseness of data
Cubist 1.07 Predictor for Total Wirelength
Optimization of Incremental Multilevel
FM Partitioning
 Motivation: Incremental Netlist Partitioning
 Scenario: Design changes (netlist ECOs) are made,
but we want the top-down placement result to remain
similar to previous result
Clustering
Refinement
Optimization of Incremental Multilevel
FM Partitioning
 Motivation: Incremental Netlist Partitioning
 Scenario: Design changes (netlist ECOs) are made,
but we want the top-down placement result to remain
similar to previous result
 Good approach [CaldwellKM00]: “V-cycling” based
multilevel Fiduccia-Mattheyses
 Our goal: What is the best tuning of the approach for a
given instance?



break up the ECO perturbation into multiple smaller
perturbations?
#starts of the partitioner?
within a specified CPU budget?
Optimization of Incremental Multilevel
FM Partitioning (contd.)
 Given: initial partitioning solution, CPU budget and
instance perturbation (I)
 Find: number of stages of incremental partitioning (i.e.,
how to break up I ) and number of starts
S
 Ti
T1
T2
T3
...
Tn
F
= incremental multilevel FM partitioning
 Self-loop  multistart
 n  number of breakups (I = 1 + 2 + 3 + ... + n)
Flow Optimization Results
 If (27401 < num edges  34826) and (143.09 < cpu time  165.28)
and (perturbation delta  0.1) then num_inc_stages = 4 and
num_starts = 3
 If (27401 < num edges  34826) and (85.27 < cpu time  143.09) and
(perturbation delta  0.1) then num_inc_stages = 2 and num_starts
=1
Design
Num Nets Cut
 ...
Name Optimized Regular
ibm01
215
217
ibm05
1685
1723
ibm02
249
269
ibm03
618
669
ibm06
363
371
ibm04
444
488
ibm08
1127
1219
ibm10
752
773
Up to 10% cutsize reduction with same CPU budget, using tuned
#starts, #stages, etc. in ML FM
Outline
 Introduction and motivations
 METRICS system architecture
 Design quality and tool quality
 Applications for METRICS system
 Issues and conclusions
METRICS Deployment and Adoption
 Security: proprietary and confidential information
cannot pass across company firewall  may be
difficult to develop metrics and predictors across
multiple companies
 Standardization: flow, terminology, data management
 Social: “big brother”, collection of social metrics
 Data cleanup: obsolete designs, old methodology, old
tools
 Data availability with standards: log files, API, or
somewhere in between?
 “Design Factories” are using METRICS
Conclusions
 METRICS System : automatic data collection and realtime reporting
 New design and process metrics with standard naming
 Analysis of EDA tool quality in presence of input noise
 Applications of METRICS: tool solution quality
estimator (e.g., placement) and instance-specific tool
parameter tuning (e.g., incremental partitioner)
 Ongoing works:
 Construct active feedback from METRICS to design
process for automated process improvement
 Expand the current metrics list to include enterprise
metrics (e.g., number of engineers, number of spec
revisions, etc.)
Thank You