Transcript document
Gifts in the treasure chest of Methodology: A personal view Rolf Steyer Friedrich Schiller University Jena Institute of Psychology Department of Methodology and Evaluation Research Germany Items in the treasure chest • • • • • • • • Latent Class Models Mixture Distribution Models Structural Equation Models Generalized Linear Models Loglinear Models Multilevel Models CART (Classification and Regression Trees) ... • LISREL and the other SEM programs • ConQuest and other IRT programs • ... • EM-Algorithm • Newton-Raphson • ... Outline • Measurement • Causality • Statistics Outline • Measurement • Causality • Statistics I have deliberately choosen this order, because: • (Causal) modeling does not make sense if we don´t have reasonable measurements • Statistcal analysis does not make sense if we don´t have reasonable measurements and a causal model, if we are looking for causal effects (which we do most of the time) Measurement The fundament of every science - Fundamental measurement theory - IRT, uni- and multidimensional - SEM modeling including models for ordinal variables - Multidimensional Scaling Measurement Measurement - is much more than assigning numbers to observations - defines the concepts to which empirical research really refers to - explicates the relationship between observations and theoretical concepts (constructs) Measurement Measurement - is not reading numbers from a meter stick - it rather means introducing the concept of length - and spells out the rules of assigning numbers to observations representing the length of the objects considered Measurement (cont´d) Measurement defines our theoretical concepts by - selecting the observables or items - specifying a mathematical measurement model relating the observables to the theoretical concepts Of course both these points need substantive theory and ideas. They are part of the substantive theory. Measurement (cont´d) The measurement model determines - the logical nature of our theoretical variables (metric, ordinal, or nominal concepts) - the scale level of the theoretical variables Measurement (cont´d) Latent variable models are flexible enough to model complex reality Measurement (cont´d) 1.00 SEELE 0.56 Trait 1 1.00 Depr1 1.00 A_DEPR1 0.10 A_DEPR2 0.11 C_DEPR1 0.03 C_DEPR2 0.09 D_DEPR1 0.04 0.74 D_DEPR2 0.12 0.70 E_DEPR1 0.03 0.70 E_DEPR2 0.11 0.85 0.08 1.00 -0.26 Trait2-1 -0.06 Trait 2 1.00 0.10 1.00 Depr2 1.00 0.85 1.00 1.00 Depr3 1.00 Trait3-2 1.00 1.00 0.85 Trait 3 0.97 Depr4 1.00 0.85 Chi-Square=35.77, df=20, P-value=0.01637, RMSEA=0.020 MetFac Measurement (cont´d) IRT-Models - introduce metric variables on the basis of qualitative variables - allow adaptive testing - are recently going multivariate ... Measurement (cont´d) Where are we going? - IRT-Models and Structural Equation Models will merge - Adaquate and sophisticated latent variable modeling will increase Causality - we know much more than „correlation is not causality“ or „noncorrelation is not noncausality“ - what we really want are individual causal effects, what we get are expectations and their difference, which are not the average of the individual causal effects Causality (cont´d) - If we have a client, we have at least two alternative treatments for him - we are able to decide which treatment to choose only when we have a hypothesis about the individual effect of the treatment compared to its alternative - when there is no knowledge about the individual effect we need at least some knowledge about the average effect E(Y X = x2, U = u) E(Y X = x1, U = u) P(U = u) Observational units Causality (cont´d) Individual treatment assignment probabilities P(X = x1 U = u) Example Ia Example Ib Example Ic u1 1/2 85 91 1/4 1/2 1/3 u2 1/2 105 109 3/4 1/2 1/3 Note. According2 to the theorem of the total probability, the unconditional probability for treatment assignment is P(X = x1) = i1 P(X =x1 U = ui) P(U = ui) = 1/2 for Examples Ia and Ib and P(X = x1) = 1/3 for Example Ic. u E(Y X = x , U = u) P(U = u X = x ) = 85 1/4 + 105 3/4 = 100, 1 1 E(Y X = x ) = u E(Y X = x , U = u) P(U = u X = x ) = 91 3/4 + 109 1/4 = 95.5. E(Y X = x1) = 2 2 2 Causality (cont´d) we know - sufficient conditions for causal unbiasedness - necessary conditions for unconfoundedness - how to analyze causal effects in nonorthogonal ANOVA - how to test for unconfoundedness Causality (cont´d) Where are we going? We will learn more about - nonexperimental design and analysis - systems of regression equations (causality in SEMs) - generalize to distributions - applying it to sophisticated data Statistics - statistical tests important, but are we really happy with knowing the probability of the test statistic being this or more extreme under the assumption of H0 ? - Shall we be going Bayesian and ask for the probabilities of hypotheses? - How long will it take for the resampling procedures (bootstrapping etc.) to be an easy tool in our standard software? Conclusions - There is much more in our treasure chest than an individual could even learn and apply - Hence we need to organize, distribute and teach our knowledge in a more efficient way - Intensifying cooperation in Europe may be one way to achieve these goals Where to find more For example, this power point file and more useful things, such as papers, infos on workshops etc. may be found at: http://www.uni-jena.de/svw/metheval/ Or mail to: [email protected]