Transcript On Representing Uncertainty in COCOMO® Parameter Values

On Representing Uncertainty In Some
COCOMO Model Family Parameters
October 27, 2004
John Gaffney
[email protected]
301-240-7038
Fellow, Software & Systems Resource Center,
Lockheed Martin
Background
• Uncertainties should be recognized in estimates of effort/cost,
schedule, and other measures of interest in software and systems
development.
• Managers and technical personnel need to make tradeoffs and
decisions under uncertainty through the effective assessment and
evaluation of risks.
• All too often, “the” value is presented, unaccompanied by any
statement of the degree of uncertainty in that value.
– Program managers and others involved in developing estimates for
proposals should be able to quantify the degree of uncertainty in the
estimates that they produce.
– Estimating cost, schedule, and other product or process variables as single
numbers fails to provide decision makers with information sufficient to
make good bidding and other decisions.
• Suggestion: Develop estimate of RISK for the value of the parameter
of interest (e.g., COSYSMO size drivers, development effort)
– Risk: A measure of the probability of an undesirable situation.
(c) Copyright, Lockheed Martin
Corporation, 2004
2
Overview
•
•
A method, prototype tools, and example applications are presented for
explicitly dealing with and quantifying uncertainty in the values of various
parameters used in COSYSMO and COCOMO II. They should be usable in
other members of the COCOMO family of estimation models as well.
The uncertainty is defined in terms of an empirical probability distribution for
the value of the parameter of interest and one minus its value or the
“risk.”Examples:
– Cost Risk= Probability that the target value for the cost (or effort) will be exceeded.
– Schedule Risk= Probability that the target value for the schedule (duration) will be
exceeded.
•
•
In general, if higher values (such as a prospective cost, or cost driver) are less
desirable, then the risk is the probability that the target value will be exceeded.
However, if higher values are more desirable (such as Mean-Time-BetweenFailures), then the risk is the probability that the actual value will be less than
the target value.
The Pearson-Tukey three-point approximation is used for estimating the
probability distribution for each parameter whose variation is represented. The
three points use values at the (estimated) 5%,50%, and 95% fractiles.
– Values obtained by expert opinion, experience data, or a combination of them; base
on an assessment of impact of “risk factors.”
(c) Copyright, Lockheed Martin
Corporation, 2004
3
COSYSMO Size Driver Prototype
• Excel-based prototype tool.
• Provides cumulative probability and risk distributions for
COSYSMO size,S.
• S=Σ(wi*si); i=1,2,3,4 . The wi are weights for “easy,”
“normal,” and “difficult.” The Si, sizes, are: # of system
requirements, # of system interfaces, # of operational
scenarios, and # of critical algorithms.
(c) Copyright, Lockheed Martin
Corporation, 2004
4
Example, Illustrative Application
of COSYSMO Prototype
(c) Copyright, Lockheed Martin
Corporation, 2004
5
13-Oct-04
PROTOTYPE COSYSMO SIZE DRIVER RISK ESTIMATOR TOOL
Place value in each box indicated in yellow.
DO NOT MAKE ENTRIES IN ANY OTHER CELLS.
Then, press cntrl+b to execute the tool and obtain the
size driver risk and cumulative probabilities
Risk Component
Number
Name
Most Likely Size=
Risk=
COSYSMO Size Driver Summary Statistics
Range Estimate Values
5% Low er
Most Likely
95% Upper
Weight: Enter E, N,
or D only
1
# of System Requirements
100
200
350
N
2
# of Interfaces
135
250
350
N
3
# of Critical Algorithms
10
15
17
N
4
# of Operational Scenarios
20
25
30
N
(c) Copyright, Lockheed Martin
Corporation, 2004
Minim um Size=
1179
Risk=
99.88%
Most Likely Size=
1882
Risk=
43.48%
Maxim um Size=
2559
Risk=
0.00%
20% Risk Size=
2063
30% Risk Size=
2003
50% Risk Size=
1880
6
Risk (= Prob. That Actual Size
Driver Will Exceed Value Indicated
on X-Axis)
Size Driver Value Risk
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10% Most Likely Size= 1882.1
5% Risk For Nom =
43.48%
0%
1000
1250
1500
1750
2000
2250
2500
Size Driver
(c) Copyright, Lockheed Martin
Corporation, 2004
7
Cumulative Probability of Size
Driver Value
Size Driver Cumulative Probability
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
1000
1250
1500
1750
2000
2250
2500
Size Driver
(c) Copyright, Lockheed Martin
Corporation, 2004
8
COCOMO II “Risk” Prototype, page 1 of 2
• An excel-based prototype tool was developed to demonstrate the
concept of identifying the uncertainty in COCOMO II cost drivers and
in the values of three other parameters of the equation that estimates
person months (PM).
• The tool estimates two principal components::
– The range for the product of 4 cost driver values
– The range of values of person months, based on the uncertainties in the
cost driver product (D),the size (S), the size exponent (E), and the
multiplying constant (A): PM=A*SE*D.
• The tool illustrates the possible range of uncertainty for 4 Early Design
Model cost drivers, and for D, their product, which would include
other drivers in an actual case:
•
– RCPX: Product Reliability and Complexity
– PERS: Personnel Capability
– PREX: Personnel Experience
– FCIL: Facilities
The coverage of the prototype could be expanded to explicitly handle more
than 4 cost drivers and of course, other drivers could be chosen and a more
nearly complete risk assessment made.
(c) Copyright, Lockheed Martin
Corporation, 2004
9
COCOMO II “Risk” Prototype, page 2 of 2
• Range of values provided for Person Months, PM:
PM=A*(SE)*D
where: PM=Person Months, S=Size (KSLOC); E=exponent;
D=Product of Cost Drivers.
– Three-point distribution estimates provided for:A,S,E, and D (obtained
from the excel tool that develops the distribution of the product of 4
drivers.
– The two tool sheets are linked together.
• The Pearson-Tukey three-point approximation for each parameter (and
for each of the four cost drivers) is used.
– The three points use values at the (estimated) 5%,50%, and 95% fractiles.
– Values obtained by expert opinion, experience data, or combination.
(c) Copyright, Lockheed Martin
Corporation, 2004
10
Example, Illustrative Application
of COCOMO II Prototype
(c) Copyright, Lockheed Martin
Corporation, 2004
11
PROTOTYPE COCOMO COST DRIVER RISK ESTIMATOR
4-Oct-04
Place one "X" in each row to select the low er, the m ost likely, and Upper values covering the range of your
uncertainty in the value of each of the four cost drivers.
Then, press cntrl+a to execute the tool and obtain the risk curve and cum ulative probabilities for the product of
the values of these cost drivers.
DO NOT MAKE ENTRIES IN ANY CELLS EXCEPT THOSE INDICATED IN YELLOW.
Driver
Number
1
Values
Name
RCPX
XL
VL
L
N
H
VH
XH
0.49
0.6
0.83
1
1.33
1.91
2.72
Low er Value
x
1
Most Likely Value
x
1
Upper Value
2
PERS
x
2.12
1.62
Low er Value
1.26
1.00
0.83
x
1
x
1.59
Low er Value
1.33
1.12
1.00
0.87
1
0.74
1
x
1
Upper Value
FCIL
Low er Value
Most Likely Value
0.62
x
Most Likely Value
4
0.50
1
Upper Value
PREX
0.63
x
Most Likely Value
3
1
X
1.43
1.3
1.1
1.00
0.87
0.73
x
1
0.62
1
x
Upper Value
1
x
(c) Copyright, Lockheed Martin
Corporation, 2004
1
12
4-Oct-04
PROTOTYPE COCOMO II EFFORT RISK ESTIMATOR
Place value in each box in dicated in yellow.
DO NOT MAKE ENTRIES IN ANY OTHER CELLS.
Then, press cntrl+b to execute the tool and obtain the
person month risk and cumulative probabilities
Risk Component
Number
Range Estimate Values
Name
5% Low er
Most Likely
95% Upper
1
A
2.40
2.94
3.60
2
S
135.0
200.0
300.0
3
E
1.05
1.12
1.20
4
D
0.4520
1.1571
1.9100
Person Months,PM=A*S E*D; where: D, cost driver product,
is obtained from the other tool sheet.
(c) Copyright, Lockheed Martin
Corporation, 2004
13
Risk (=Prob. That Product of Cost
Driver Values Will Exceed
Indicated,X-Axis, Figure)
COCOMO II Cost Driver Product Value Risk
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
Product of Cost Driver Values
(c) Copyright, Lockheed Martin
Corporation, 2004
14
Cumulative Probability of Product of
Cost Driver Values
COCOMO II Cost Driver Product Value Cumulative Probability
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
Product of Cost Driver Values
(c) Copyright, Lockheed Martin
Corporation, 2004
15
Risk (= Prob. That Actual Person
Months Will Exceed Indicated, X-Axis,
Figure)
Person Months Value Risk
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Person Months
(c) Copyright, Lockheed Martin
Corporation, 2004
16
Cumulative Probability of Person
Months
Person Months Cumulative Probability
100%
95%
90%
85%
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Person Months
(c) Copyright, Lockheed Martin
Corporation, 2004
17
Some References
1.
2.
3.
4.
In Chapter 8, How To Estimate Software Cost, in The Software
Measurement Guidebook, by J. Gaffney, R. Cruickshank et al.,
Thomson Computer Press, 1995.
In the section, How to Estimate Software Project Schedules, by J.
Gaffney, in the book, Software Engineering Project Management,
edited by Richard Thayer, IEEE Computer Press, 1997.
Software Risk Metrics and Tool, by J. Ulvila, J. Gaffney, and J.
Chinnis, Metrics 2002, IEEE 8th International Symposium on
Software Metrics.
A Methodology and Implementation For Software System Cost and
Risk Estimation, by J. Gaffney, J. Bridel, and D. McGovern, 8th
PSM Users’ Group Conference, July, 2004.
(c) Copyright, Lockheed Martin
Corporation, 2004
18