Transcript Statistics for science fairs

STATISTICS FOR
SCIENCE FAIRS
HOW TO MAKE SURE YOUR STUDENTS HAVE THE BEST CHANCE OF SUCCESS AT
SSEF OF FLORIDA
MY PERSONAL INVOLVEMENT
Judging Captain of Senior Animal Sciences
How we judge:
First, divide projects in orbit and focus groups
Use scores to group similar level projects in focus group
Discuss top scoring projects to determine overall category winner
Last year, much debate over specific project
Scores ranked from 1st – 10th depending on judge
Upon further discussion, deciding factor came down to use of statistics
(incorrect statistics for experimental design)
Likely would’ve placed higher, possibly 1st with correct statistics.
JUDGE’S PERSPECTIVE
Are there statistics?
Are they the right statistics?
Does the student understand the statistics?
Don’t need to know the underlying calculations being
used, just how they work and what they mean
COMMON PITFALLS
• Experimental design with small sample sizes
• Using averages to make conclusions about data results
without looking at the variability in the data
• Correlation vs. Causation
• Extrapolation of data
• Off-the-shelf programs (i.e. Excel) make statistics easy to
use, therefore easy to use incorrectly
• No statistics at all
ROLE OF TEACHERS/ADVISORS
Students are not often familiar with statistics – initial exposure
Preliminary discussion prior to starting project
Good statistical analysis is derived during experimental design
Make sure sample size is adequate
Understand what tests will help determine significance/validity
Reassess statistics as first collecting data
Modifications to design may be necessary
Find mentor to assist with statistics, if needed
EXPERIMENTAL DESIGN
Keys to designing project with statistics in mind:
• Accurately and clearly define variables and sample space
• Accurately define factors and levels of factors
• Identify the type of experiment, making sure to use appropriate
controls
• Making sure to perform enough replications
• Making sure to understand the likely distribution of data
• Awareness of types of exploratory and inferential analyses used
in your field of science. Look to journal articles.
VARIABLES
Quantitative variables – differ in magnitude, can be measured
Qualitative variables - categorical, observations differ in kind
(nominal)
Rank qualitative variables – ordinal variables (allows for
mathematical analysis)
Pre-planning allows to choose what kind of data you will collect,
what statistics you can use.
REDUCING NOISE IN RESULTS
Experimental Observations = combination
Signal – true effects of variable/outcome
Noise – random error introduced by experimental design
Increase signal-to-noise ratio (decrease noise)
•
•
•
•
•
•
Making repeated measurements of one item
Increasing sample size
Randomizing samples
Randomizing experiments
Repeating experiments
Including covariates (other variables that might impact results)
EXPLORATORY DATA ANALYSIS
Categorical Variables - bar graphs, pie charts, two-way tables
Quantitative Variables – stem plots, histograms, relative cumulative
frequency plots, time plots, scatterplots
Calculate and compare mean, median, standard deviation (report
value as a measure of center and measure of spread)
What is the distribution?
Are there outliers?
STATISTICAL INFERENCE
P-value – probability that the observed result is due to chance
The probability that from a randomized controlled experiment, the null
hypothesis is correct
Relationship between 2 quantitative variables
Scatter plot and regression
1. Plot independent variable on x-axis; discuss pattern
2. If linear, calculate correlation coefficient to measure strength and relationship
3. Use least squares regression to determine model of relationship between two
variables
4. Conduct t-test
5. If not linear, more advanced methods needed
STATISTICAL INFERENCE
Compare data from two different groups
Box plots and t-tests
1.
2.
3.
4.
5.
6.
Plot data using side-by-side box plot
Determine alternative hypothesis
Define level of significance – use two-sample t-test
Evaluate practical significance of difference between groups
Report the results as inferential analysis
When testing against set value (not between groups), use
one sample t-test
7. If comparing across more than 2 groups, consider ANOVA
STATISTICAL INFERENCE
Inference for categorical variables
Testing results against expected distribution
1. Start with two-way table. Calculate marginal
distributions and differences between marginal
distributions of experimental and control groups
2. Use bar graph or pie chart to show distributions
differences among groups
3. Use chi-square test for goodness of fit. Determine
test statistic and p-value
4. If chi-square finds significant results, examine to
find largest components
PRESENTING RESULTS
•
•
•
•
•
•
•
•
•
State statistical hypothesis along with your scientific hypothesis
Use flowchart to show experimental design
Show how replication, control and randomization are used
Show both exploratory data analysis and inferential analysis
Discuss meaning of graphs and measures
State level of significance in tests (p-value)
State conclusions of tests
State the statistical and practical significance of results
Tell whether null hypothesis was accepted or rejected
Statistics are meaningless if they are misrepresented or misunderstood!
CONFIDENCE INTERVALS
95% CI
Standard Deviation – measure of spread of data
How far away from the mean are data points?
Variance =
(𝑥−𝑥)2
(𝑛−1)
Standard deviation =
(𝑥−𝑥)2
(𝑛−1)
How do we express?
According to the current data, 68% of data falls within Average ± 1 standard
deviation
P-value of .05 = 95% confidence interval = 2 standard deviations
If this experiment were repeated on multiple samples, the calculated confidence
interval would encompass the true population of the parameter 95% of the time.
REGRESSION ANALYSIS
Estimates relationship between independent and
dependent variables
Calculates line of best fit between data points
Correlation measures how close line fits the data –
correlation coefficient (R2) – closest to 1 (positive
relationship) or -1 (negative relationship)
IS THERE A STATISTICAL DIFFERENCE?
Plant height
Control = 15 mL water
Treatment A = 30 mL water
Treatment B = 45 mL water
Does the amount of water
affect the
height of the plant?
On the surface, we might say
yes! The averages are higher
for treatment A and B. But is this
really the case?
N
Control
Treatment A Treatment B
1
5.5
6.0
5.5
2
5.5
6.0
6.0
3
6.0
6.5
6.5
4
6.0
6.5
6.5
5
6.0
6.5
6.5
6
6.0
7.0
7.0
7
6.0
7.0
7.0
8
6.5
7.0
7.0
9
6.5
7.5
7.5
10
6.5
7.5
7.5
Avg.
6.05
6.75
6.7
T-TEST
Determines whether the difference in the averages for two or more treatments is
mostly caused by the experimental treatment or whether the difference can be
explained by random variation
Requirements:
Two or more comparison groups (control and one treatment, or two or more
treatments).
A sample size of 10 or more for each experimental group.
Numerically measured data (no categories, even if labeled with numbers).
Easy to calculate T-Test
http://graphpad.com/quickcalcs/ttest1.cfm
Excel
Other statistics software
CONTROL AGAINST TREATMENT A
P-value: 0.05 is generally accepted
> 0.05, no difference between means
P-value of 0.0033 = .33% these results
are by chance
Watering plants with 30 mL instead of
15 mL will increase height under these
same conditions
CONTROL AGAINST TREATMENT B
P-value of 0.0117 = 1.17% these results
by chance
Watering plants with 45 mL instead of
15 mL will increase height under these
same conditions
But what about between 30 mL and
45 mL?
TREATMENT A AGAINST TREATMENT B
P-value of 0.8513 = 85.13% that
difference between treatment averages
are by chance
No significant difference between
watering with 30 mL or 45 mL
CHI-SQUARE TEST
2
Compare observed data with expected data according to a hypothesis 𝑥 =
(𝑂−𝐸)2
𝐸
Convert 𝑥 2 to a probability using degrees of freedom (n-1)
P-value = probability that null hypothesis (observed and expected are not different)
is correct
Simple example – flip a coin 100 times. Expected 50 heads, tails. Observed 41 heads, 59
2
𝑥 =
(41−50)2
50
(59−50)2
+
50
= 3.72
2 groups = 1 Degree of Freedom
7% chance that null hypothesis is correct
93% chance what we saw is different
than expected
BUT… > 0.05 so we support null hypothesis
ANOVA
Analysis of Variance test
Determine if there is a difference between means of
3 or more independent, unrelated groups
Tests the null hypothesis that no difference between
the groups exists.
Easy to perform analysis with software
If you determine there is a difference between the
groups, additional testing will be needed
Tukey HSD
Scheffe post hoc test
Games Howell
Dunnett’s C post hoc test
FINAL THOUGHTS
Proper use and understanding of statistics are often the deciding factor
between the top projects
Give your students the upper hand by discussing statistics at the start of
the project
Review data and industry standards to determine the correct statistical
analysis based on experimental design
Properly quantify statistical results in the context of project
If possible, review statistical procedures/results with students prior to
science fair by knowledgeable mentor
REFERENCES
Online statistics calculator - http://graphpad.com/quickcalcs/
Judge’s perspective - http://www.nsta.org/publications/news/story.aspx?id=53713
Statistics for science projects - https://slvsef.org/documents/teachers/SLVSEF_statistics_for_science_fair_students.pdf
Data analysis for science projects - http://www.sciencebuddies.org/science-fair-projects/top_research-project_dataanalysis.shtml
http://static.nsta.org/files/PB343Xweb.pdf
Biological Statistics - http://www.biostathandbook.com/
Engineering Statistics - http://www.itl.nist.gov/div898/handbook/
Contact Kim Unger – [email protected]