Transcript PPT slides
Third year project – review of basic statistical concepts Descriptive statistics Statistical significance Significance and effect size Interpreting a significant effect Interpreting a non-significant effect Descriptive statistics Choose descriptive statistics that are: Appropriate Relevant Revealing Previous research articles can be a useful guide Error bars If n is small, show data points not error bars You must show what n is in the Figure legend You must say what kind of error bar you are using Standard error based error bars are often used Confidence intervals are better Descriptive statistics almost never licence an inference Men (M = 32, SD = 6) v. women (M = 34, SD = 5) → no way to conclude from this alone that women (in the population) have a higher mean than men Exception: the direction of the difference may contradict a hypothesis If the hypothesis was that men have a higher mean than women – the data do not support that Inferential statistics T-test; ANOVA; Wilcoxon matched pairs Chi – squared Regression Correlation … and many more These tests assess the effects seen, comparing them to differences we'd expect 'anyway' (i.e. differences attributable merely to the kind of difference we'd expect from sampling) For example, is the difference between Men and Women greater than the difference you might get between two different samples of women. Statistical significance significance,p-value, alpha-level p < .05 “Fewer than five times out of a hundred, if you ran this study thousands of times, would you see a difference this great.” Exact and approximate p-values Some inferential statistics can give you an exact p-value Some only give an approximation Usually, with large samples, the approximation is very good Most inferential statistics rely on assumptions about the distribution of the data (1) Textbooks say the tests are 'robust' when assumptions are violated (2) But, really, we don't have a very clear picture (3) The assumptions often are violated (4) It's up to you to check the assumptions (ANOVA etc. don't) Significance and effect size We are interested in effects Significance – rarity [rarity of observing this if there were no real effect] Size – is it a big difference Effect size and significance are separate Can be significant with small, tiny, effect size, especially if the sample is huge A large effect can look non-significant, especially with a small sample Apart from sample size, reliability of measures and other sources of error variance can make it hard to detect an effect Power – the probability of detecting an effect if it really exists Report effect size -d - Partial eta squared - r (and r2) - R-squared (and adjusted R-squared) And compare effect size with previous research... Interpreting a significant effect If p < .05 (conventionally, significant) It is conventional to conclude that the null hypothesis [e.g. no difference between men and women] can be rejected Bear in mind, however, that up to five times in a hundred, we would get an effect like this if there was no real effect Allow for multiplicity Reporting p-values I recommend: Report exact p-value if p is <.10 Report p < .001 if p <.001 Report p > .20 if p > .20 Interpreting a non-significant effect - p > .20 don't quibble - p > .10 don't quibble, unless there is a substantial reason to - p < .10 mmm... Bear in mind that you may have low power In exploratory research, a more liberal approach is often taken