Transcript Measuring and Improving Design Capability

Key Performance Indicators
Of Methodology Capabilities
Presented
by
Ronald Collett
Numetrics Management Systems
Santa Clara, CA
www.numetrics.com
Measuring Design Productivity is the Foundation
of Assessing Design Methodology Capabilities

Design productivity is a proxy for design process
quality
— High productivity means high design output per engineer
— High productivity is a reflection of:



Engineering skill and management skills,
Tools, flows, methodology, infrastructure
To be used as a proxy, productivity calculation
must contemplate that the product designed is
released to volume production
— Releasing to volume production implies that the product itself
offers the requisite functionality, performance, price,
reliability, form factor, etc. (i.e. the right value proposition)
Basic Productivity Definition
PRODUCTIVITY =
OUTPUT
LABOR INPUT
Measuring Manufacturing Productivity Is
Straightforward
MANUFACTURING
=
PRODUCTIVITY
VALUE-ADDED
LABOR INPUT
VALUE-ADDED = PRODUCT SELLING PRICE - COST OF MATERIALS
Measuring Design Productivity Is Much
More Difficult
DESIGN
=
PRODUCTIVITY
????????
Effort (Person-weeks)
Dissecting the Numerator of the Design Productivity
Metric--What’s the Best Measure of What a Design Team
Produces?
Overly simplistic and INACCURATE measure of what a
design team produces: Total transistors in the design
TOTAL TRANSISTOR COUNT
EFFORT (MAN-WEEKS)
There is Almost Zero Correlation between
Transistor Count and Project Effort
Relationship Between Raw Transistors and Project Effort*
3,500
R2 = 0.105
p = 0.008
Project Effort (Person Weeks)
3,000
= IC Design Project
2,500
2,000
1,500
1,000
500
0
0
2
4
6
8
10
Raw Transistors in Millions
12
14
16
Measuring Design Productivity
Units
P=
Design
Output
Design
Effort
Numetrics Complexity
Units (NCUs)
Person-Weeks
(Direct Measure
Of Staff & Schedule)
Influencing Factors
Transistor Count
Circuit Type
Reuse Levels
Timing
Density
Etc….
Factors that explain high
or low productivity:
- Engineering Skill Levels
- EDA Tools
- Design Flow
- Process Stability
- Customer Relationship
- Management Support
- Etc...
A Very Strong Correlation Exists Between
Numetrics Complexity Unit (NCU) Calculation
and Project Effort
3,500
Y = 179 + 544 * X
R2 = 0.520
p = 0.000
Project Effort (Person Weeks)
3,000
Relationship Between
NCUs and Project Effort*
2,500
2,000
1,500
1,000
500
0
0
0.5
1
1.5
2
2.5
3
Numetrics Complexity Units (NCUs) in Millions
3.5
Numetrics’ Normalization Methodology Yields an R-squared
Value of 0.52 (Project Effort vs. NCUs/Chip)
Accuracy of the Normalization Methodology
Actual Transistor™ Count
vs. Project Effort
Project Effort
(Person-weeks)
Numetrics Complexity Unit Count
vs. Project Effort
Project Effort
(Person-weeks)
2
R = 0.105
p = 0.008
= IC Design Projects
3,500
3,000
2,500
2,500
2,000
2,000
1,500
1,500
1,000
1,000
500
500
0
2
4
6
8
10
12
= IC Design Projects
3,500
3,000
0
R2 = 0.520
p = 0.000
14
Actual Transistors per chip (Millions)
16
0
0
1
2
NCUs per chip (Millions)
3
Numetrics’ Design Productivity Management System
(DPMS) Quantifies Design Productivity and, therefore,
Design Quality
High Productivity
Design Project
Low Productivity
Design Project
Comparing Design Capability
Without DPMS
High Productivity Project
Low Productivity Project
Comparing Design Capability
With DPMS
Other Factors Explain the Difference in Design
Effort Between Projects of Similar Complexity
EDA Tools/Flows/Methodology
Inherent Design Complexity
IC Design
Effort
39%
69%
Engineering Capability
External Factors
Leadership
Key Performance Indicators are a Prerequisite
for Determining Quality of Design Process
IC Design Productivity
2000
2500
$40
3000
$60
3500
1500
4000
1000
NCUs per
person week
500
0
8000
$20
5000
10000
$80
Dollars per
NCors
4500
IC Design Capacity™
$100
$0
IC Reuse Leverage
NCU= Numetrics Complexity Unit
12000
40%
60%
14000
6000
4000
2000
IC Development Cost
16000
NCUs per week
0
18000
20000
Performance
of a Particular Project
80%
20%
Percent
Effort
Saved per
IC Design
0%
Industry Average
100%
The Power of Measuring Design Process Quality by
Observing Three Key Performance Indicators
Simultaneously
ASSP Project Distribution by Design Productivity, Design Capacity & Development Cost*
Dev. Cost=$ per NCU
Design Productivity Industry Average
5% Trim Mean
100,000
Low-Cost Project
Design Capacity (Log Scale)
NCUs designed per Week)
(Dev Cost < $5.55)
Mid-Cost Project
($5.55 < Dev Cost < $13.40
High-Cost Project
10,000
(Dev Cost > $13.40)
Design Capacity Industry Average
5% Trim Mean
1,000
100
10
100
1,000
10,000
Design Productivity (Log Scale)
(NCUs designed per Person-Week)
100,000
Comparing the Quality of Two Different Design Flows
Average Productivity
HIGH
Design Capacity
Average Capacity
(NCUs designed
per week)
LOW
LOW
NCU= Numetrics Complexity Unit
OLD Design Flow
Design Productivity
(NCUs designed per Person-Week)
NEW Design Flow
HIGH
Two Steps are Needed to Compare
Different Chip Design Projects
1. Design complexity normalization is used to
Account for differences in reuse levels, circuit
types, process technology, timing, and other
circuit design characteristics.
2. Grouping similar projects by design application,
project scope, team goals, etc.
Combining Normalization with Grouping of Similar
Projects (in terms of design application, circuit content,
etc.) Provides for Best-in-Class Assessment
HIGH
Best-in-Class
Quadrant
Design Capacity
Average Capacity
(NCUs designed
per Week)
Analog & Mixed-Signal ICs
for Communications
Applications
Average Productivity
LOW
LOW
Design Productivity
(NCUs designed per Person-Week)
NCU= Numetrics Complexity Unit
Low Cost Project
Mid-Cost Project
High Cost Project
HIGH
Five Sets of Key Performance Metrics
Cycle Time Metrics
 Project Effort Metrics
 Project Cost Metrics
 Design Reuse Metrics
 Technology Metrics

Cycle Time Metrics

Design Capacity
— NCUs /Week (Numetrics Complexity Units designed per
Week)

Design Cycle Time
— Time from Project Start to Release to Volume Production

Project Schedule Slippage
— as a Percentage of Planned Schedule

First Prototype Turnaround Time
— 1st Tapeout to 1st Packaged Prototypes Received from Fab

Time Allocation by First Tapeout
— Time Consumed Prior to 1st Tapeout
— Time Consumed After 1st Tapeout
Example Project Staffing Profile
(People versus Time)
15
First Tapeout*
Industry Standard
Definition
10
Industry Standard
Definition
Phase 3
5
Phase 2
Phase 1
0
Project
Start
Milestone Milestone
1
2
Phase
4
Phase 5
Full-time Equivalent People
(FTE)
DPMS Yields a Profile of Project Effort and
Duration for Each Design Phase
Phase 6
Milestone
Milestone
3 Milestone 5
4
Project Duration = End Date - Start Date
Project Effort = ∑ (Phase Duration • FTE)
Project
End
Engineering Managers are using DPMS to
Analyze Cycle Time Improvements
Phase Duration Improvement
Phase 1
High Level
Design &
Partitioning
Phase 2
RTL
Design
Phase 3 Phase 4
Logic
Design
Test Insertion
APR & Timing
Verification
Phase 5
Phase 6
1st Proto
Fabrication.
System
Validation
107.8 Weeks
Projects
Started
in 1996
18.0
(17%)
20.3
(19%)
-28%
CAGR
15.4
(14%)
-33%
-49%
CAGR
CAGR
9.5
(9%)
10.0
(9%)
-35%
CAGR
-39%
-40%
CAGR
CAGR
24.5 Weeks
Projects
Started
in 1999
5.6
6.6
(26%) (23%)
6.3
(27%)
1.7
2.3
2.0
(7%)
(9%)
(8%)
TTM REDUCTION
-37% CAGR
34.5
(32%)
Cycle Time Metrics (cont’d)

Design Phase Improvements
(if a standard template is used)

Relative Capacity (for netlist-handoff ASIC only)
—Physical Design Cycle Time
—Number of Silicon Spins
—No. of Planned Spins
—Actual Metal-only Spins & Actual All-layer Spins
Summary and Conclusions

Measuring design productivity is the cornerstone
for measuring design methodology efficacy

Quantifying design complexity is a prerequisite to
measuring design productivity--requires a robust
normalization approach in order to compare
designs fairly

Numetrics measurement system is now being used
across the semiconductor and systems industry

Quality of design process has become tantamount
to quality of manufacturing process