Transcript Slide 1
ASSESSMENT OF MORAL REASONING AT TWO FAITH BASED INSTITUTIONS David W. Kale, Ph.D. Director of Assessment, Mount Vernon Nazarene University [email protected] Joel Frederickson, Ph.D. Chair, Psychology Dept., Bethel University [email protected] ASSESSMENT OF MORAL REASONING BETHEL UNIVERSITY Moral Thinking: Using the DIT-2 as a Measure of Cognitive Development What does the DIT-2 measure? A revised measure of moral reasoning based on Kohlberg’s theory of moral development. Focus is on schemas NOT stages. Five scenarios (original DIT had six). Strong correlations between DIT-1 and DIT-2 (r=.79). Example Dilemma from original DIT: Heinz & the drug Schema Scores Personal Interest Schema Score: The proportion of items selected that appeal to stage 2 & 3 thinking. Stage 2 focuses on the direct advantages to the actor and on the fairness of simple exchanges of favor for favor. Stage 3 focuses on the good or evil intentions of the parties; concern for maintaining friendships and approval. Schema Scores Maintaining Norms Schema Score. The proportion of items that appeal to stage 4 thinking. Focus on maintaining existing legal system, existing roles and organizational structure. Schema Scores Postconventional Schema Score (P score) Focus on organizing society by appealing to consensus-producing procedures (majority vote), insisting on due process, and safeguarding basic rights. Organizing social arrangements & relationships in terms of intuitively appealing ideals. Example Famine Scenario: Read through this scenario and the accompanying “issues” related to the dilemma. Place the “issues” into the categories they are measuring – Personal Interests – Maintaining Norms – Postconventional Thinking – Meaningless/pretentious items New Index (N2) N2 score is a new index and is considered superior to the P score (post conventional reasoning score). There are two parts to this score: the degree to which respondents support post conventional responses (P score), plus the degree to which personal interest items receive lower ratings. Essentially, the N2 is a score that reflects the degree to which respondents will reject bad arguments (personal interest arguments). Validity & Reliability of DIT Differentiation of various age/education groups: 30% to 50% of the variance in DIT scores is attributable to level of education. Education and Moral Thinking As education level goes up, degree of postconventional thinking increases. 60 50 40 30 20 10 0 JC Fr So Jr Sr MS PD Ph.D Validity & Reliability of DIT Differentiation of various age/education groups: 30% to 50% of the variance in DIT scores is attributable to level of education. Longitudinal gains: Reviews of a dozen studies of Freshman to Senior college students show effect sizes of .80, one of the most dramatic effects of college. Validity & Reliability of DIT Sensitive to moral education interventions Reliability is adequate (Test-retest and Cronbach alphas in upper .70s, lower .80s). Use of the DIT at Bethel for Assessment Look at cross-sectional and longitudinal differences from Freshman to Senior year. We have also analyzed these Freshman to Senior changes by department. Example: Business department typically had low growth. They made some changes to the curriculum (and faculty) and saw better growth in moral reasoning. Use of the DIT at Bethel for Assessment Usually we see typical growth from Freshman to Senior year (.70-.80 effect sizes). A small cohort in our degree completion Business Management program saw similar growth. Just beginning to get longitudinal data for our MBA program. Use of the DIT at Bethel for Assessment Colleague uses pre/post DIT in his course “Being Just in an Unjust World” (essentially a course in Moral Thinking). Typically sees a big change for a semester (about .70 effect size). This is the amount of change you would typically see in 4 years! Pre/post DIT N2 Scores by Type of Course 60 50 40 Being Just course 30 Comparison courses 20 10 0 Pre Post Use of the DIT at Bethel for Assessment A small subset of students who had taken this course (N=23) took the DIT again four months later. Students not only maintained growth, but showed a marginally significant increase. ASSESSMENT OF MORAL REASONING AT MOUNT VERNON NAZARENE UNIVERSITY COMBINING THE QUANTITATIVE WITH THE QUALITATIVE QUALITATIVE ASSESSMENT The Potter Box Definition of situation Loyalties Values Principles Ralph B. Potter, “The Logic of Moral Argument” in Towards a Discipline of Social Ethics, ed. Paul Deats (Boston Univ. Press, 1972). DEFINITION OF THE SITUATION The letter is the engineer’s personal and professional correspondence. The letter is privileged communication between the engineer and his attorney. What other aspects of the situation would you consider important to consider? VALUES Privacy Truth What other values do you see? What is your highest value? PRINCIPLES Always tell the truth PRINCIPLES Tell the truth Protect the privacy of others LOYALTIES To whom is moral duty owed? - The 17 year old - The engineer - Who else has a stake in this situation that needs to be protected? To whom do you have the highest loyalty? MORAL AND ETHICAL THEORIES Aristotle’s Golden Mean Biblical concept of love Utilitarianism Rawls’ Theory of Justice Kant’s Categorical Imperative BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT After dealing with several case studies, I ask students to answer two questions anonymously. 1. In what ways have you improved in your moral and ethical reasoning ability? 2. In what ways would you like to continue to improve? BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT I identified eight dimensions of moral reasoning in their answers. 1. Using clearly worded arguments when I state my position; 2. Making sure I have good support for my position (facts, reasoning, statistics) 3. Taking the views of others into consideration when I make my decision. 4. Supporting my arguments with biblical truth. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT 5. Having a clear process for moving all the way to a decision; 6. Making my decisions based on reason rather than letting my emotions get the upper hand; 7. Taking my time rather than rushing into a decision. 8. Staying well informed on the issues of the day. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT I then have students vote on the top five. 1. Using clearly worded arguments when I state my position. 2. Making sure I have good support for my position (facts, reasoning, statistics). 3. Taking the views of others into consideration. 4. Supporting my position with biblical truth. 5. Taking my time rather than rushing into a decision. BUILDING A RUBRIC FOR STUDENT SELF ASSESSMENT Building a rubric in this fashion is an assessment strategy in itself. If there is an aspect of moral reasoning that does not turn up in their list that I think is important, that tells me either that they have not learned it or do not think it is nearly as important as I do. QUANTITATIVE ASSESSMENT USING THE DIT-2 AT MVNU On the traditional side, we administer the DIT-2 to our incoming first year students, to our Juniors on Testing Day and to our seniors at the end of the Senior Colloquium course. We also administer the DIT-2 to our nontraditional students at the beginning and at the end of their degree completion programs. N2 scores Trad. Juniors 26.51 Trad. Seniors 35.28 Trad. Juniors 26.51 Trad. Juniors 26.51 Trad. Seniors 35.28 Juniors nat’l. Av 32.65 Seniors nat’l. Av 36.85 Trad. Seniors 35.28 Non-trad. Junior 23.06 Non. Trad. Sen. 26.55 p = <.01 p = .079 p = <.001 p = .148 p = .0002 Trad. And Non-Trad. Students Post conventional Reason. Scores 40 35 30 25 Trad Non Trad 20 15 10 5 0 Junior Senior SUMMARY There appears to be strong evidence from the use of the Defining Issues Test at two faith based institutions that a senior level Christian ethics course significantly improves students’ moral reasoning ability. Using both quantitative and qualitiative measures provides richer data as to exactly what aspects of students’ reasoning has improved.