Transcript non
Experimental Design The “gold” standard?? • What kind of research is considered the “gold standard” by the Institute of Education Sciences? A. B. C. D. • Descriptive Causal-Comparative Correlational Experimental Why? • Why is most educational research comprised of non-experimental research designs?? A. Experiments are expensive. B. Maturation is a problem. C. Ethical and logistical considerations. D. Researchers are lazy. • How is experimental research different from non-experimental research designs? A. Experiments use random sampling. B. Experiments use more than one observer. C. Experiments control the independent variable. D. They are essentially the same. • What kind of research can claim causality? A.Descriptive B.Causal-Comparative C.Correlational D.Experimental It’s time to play… Name that RESEARCH!!!! Experimental Validity • Internal validity – The extent to which the independent variable, and not other extraneous variables , produced the observed effect on the dependent variable • External validity – The extent to which the results are generalizable Internal Validity • Threats that reduce the level of confidence in any causal conclusions • Key Question: Is this a plausible threat to the internal validity of the study? Threats to Internal Validity • History – Extraneous events have an effect on the subjects’ performance on the dependent variable – The crash of the stock market, 9-11, the invasion of Iraq, etc. • Selection – Groups that are initially not equal due to differences in the subjects in those groups – Positive and negative attitudes, high and low achievers, etc. Threats to Internal Validity • Maturation – Changes experienced within the subject over time • Pretesting – The effect of having taken a pretest • Instrumentation – Poor technical quality (i.e. validity, reliability) or changes in instrumentation Threats to Internal Validity • Subject attrition – Differential loss of subjects from groups • Statistical regression – The natural movement of extreme scores toward the mean • Diffusion of treatment – The treatment is given to the control group • Experimenter effects – Different characteristics or expectations of those implementing the treatments across groups Threats to Internal Validity • Subject effects – The effects of being aware that one is involved in a study – Four types • • • • Hawthorne effect John Henry effect Resentful demoralization Novelty effect Internal Validity • Key Point: Ultimately, validity is a matter of judgment. Ask if it is reasonable that possible threats are likely to affect the results. External Validity • The extent to which results can be generalized from a sample to a particular population. • Question – Why would really good internal validity often result in poor external validity? Experimental Designs • Single Group Pre-test Post-test – Mary, Colleen • Non-Equivalent Groups – Kelsey, Nick • Quasi-Experimental Design – Amanda, Emily, Kathleen, Heather • Randomized Post-test only – Tasha, Caitlin • Randomized Pre-test Post-test – Tim, Seth, Jamie, Jen • Time Series Design – Ally, Dana Your Task • Based on the topic of your proposal, design an experimental study using the design you were assigned. – Write a research question and hypothesis. – Sketch out the methods. • Identify strengths and weaknesses of each design. Evaluating Experimental Designs • Criteria for evaluating experimental research – The primary purpose is to test causal hypotheses – There should be direct manipulation of the independent variable – There should be clear identification of the specific research design Evaluating Experimental Designs • Criteria for evaluating experimental research – The design should provide maximum control of extraneous variables – Treatments are substantively different from one another – The number of subjects is dependent on or equal to the number of treatment replications