Transcript using evidence

Using Evidence Effectively
CS 4001
Mary Jean Harrold
1
Star Criteria
S
T
A
R
CS 4001
Mary Jean Harrold
2
Star Criteria
Sufficiency: Is there enough evidence?
Typicality: Are the chosen data representative and
typical?
Accuracy: Are the data accurate and up to date?
Relevance: Are the data relevant to the claim?
CS 4001
Mary Jean Harrold
3
Kinds of Evidence
1. From personal experience
a. Personal experience
b. Personal observation or field research
2. From interviews, questionnaires, surveys
3. From reading and research
4. Statistical Hypothetical examples, cases,
scenarios
5. Reasoned sequence of ideas
CS 4001
Mary Jean Harrold
4
Does this evidence meet the
STAR criteria?
Claim: Testing a software system before
deployment typically costs a company millions
of dollars
Evidence:
 It cost at least this much to test the Boeing 777
airplane before it was deployed
What kind of evidence is this?
CS 4001
Mary Jean Harrold
5
Does this evidence meet the
STAR criteria?
Claim: Software errors cost companies millions of
dollars per year
Evidence:
 June 2002 National Institute of Standards and
Technology (NIST) report found that software bugs
are so prevalent and so detrimental that they cost the
U.S. economy an estimated $59.5 billion annually or
about 0.6% of the GDP.
What kind of evidence is this?
CS 4001
Mary Jean Harrold
6
Does this evidence meet the
STAR criteria?
Claim: Test cases developed for large software
systems typically cover only about 30% of the
functionality of the software
Evidence:
 In the development of Java Architecture for Bytecode
Analysis (JABA) within my research group, the test
suite we developed covers approximately 30% of the
statements in the program.
What kind of evidence is this?
CS 4001
Mary Jean Harrold
7
Does this evidence meet the
STAR criteria?
Claim: One of the best ways to improve the
security of your software is to use good software
engineering practice and process
Evidence:
 This was stated by Prof. Eugene Spafford, Purdue
University in his Distinguished Lecture for Georgia
Tech Information Security Center in 2001.
What kind of evidence is this?
CS 4001
Mary Jean Harrold
8
Does this evidence meet the
STAR criteria?
Claim: On-line discussion of ethical issues
improves students’ willingness and ability to
participate in discussions (over class
discussions)
Evidence:
 CS4001d on-line chat conducted by Jim Hudson
What kind of evidence is this?
CS 4001
Mary Jean Harrold
9
Perspective, Angle of Vision
Need to understand writer’s perspective when
using evidence to support claim
 Two pieces of evidence
1. One of the best ways to improve the security of your
software is to use good software engineering
practice and process (E. Spafford…)
2. You cannot have a secure system without having a
secure operating system (Jonathan Spapiro,
researcher for Extremely Reliable Operating System
(EROS), in his GTISC talk, 2004)
CS 4001
Mary Jean Harrold
10