Transcript Admissions Workshop - PCAT: Proposed Changes for the

Admissions Workshop
PCAT: Proposed Changes for the Future
Andrea L. Wall, BSPharm., R.Ph.
Paul W. Jungnickel, Ph.D., R.Ph.
Renee M. DeHart, Pharm.D.
AACP Governance
Admissions
PCAT Advisory Committee
PharmCAS Advisory Panel
Gayle Brazeau, Univ of New Engl
Jennifer Adams, AACP
Kathy Besinque, Univ Southern Cal
Renee DeHart, Samford University
Mary Euler, Univ of Charleston
Eric Hanson, Roseman University
Paul Jungnickel, Auburn University
Rodney Larson, Husson University
Paul Ranelli, Univ of Minnesota
Theodore Tong, Univ of Arizona
Jennifer Clutter, West Virginia Univ
Jennifer Adams, AACP
Tom TenHoeve, Univ of Illinois
Jeremy Altschafl, Univ Wisconsin
Renae Chestnut, Drake University
Joel Gonzales, Univ California-SF
Peter Haeg, University of Minnesota
Jennifer Hess, University of Buffalo
Mike Kelly, University of Iowa
Jonathan Parker, Samford Univ
Cyndi Porter, Calif Northstate Univ
Andrea Wall, Univ of Cincinnati
Admissions Landscape
• Pre-Pharm Curriculum
• Holistic Admissions
• Other Health Professions
AACP Interim meeting
February 2013
Special meeting of the PCAT Advisory
Committee included members of PCAT
Advisory Committee, PharmCAS Advisory
Panel, Peggy Piascik, President-elect
AACP, Kathy Kelley, Marilyn Speedie,
Pearson staff
Argus Commission 2012
Recommendation of the Argus Commission 2012 report
Examination of the Role of Pre-Pharmacy
Education in Cultivating “Habits of the Mind” in
Pharmacists
Recommendation : use of assessment tools for affective
domain- critical thinking, problem solving,
inquisitiveness, professionalism
Cultivating ‘Habits of Mind’ in the Scholarly Pharmacy Clinician: Report of the 2011-12 Argus Commission AJPE 2012:76(6)
Examples of Assessment
tools
•
•
•
•
Personal Potential Index (PPI)
California Critical Thinking Skills Test
Situational Judgment Testing
Professionalism Assessment Tool
Cross-Validation of an Instrument for Measuring Professionalism Behaviors, Kelley et al AJPE 2011 75(9)
PCAT
Can PCAT be modified to assess the
affective domain?
Moving Forward: Proposed
Revisions to the PCAT
• Present current PCAT model and
proposals for 2 additional years
• Highlight changes under consideration
– Greater critical thinking and reasoning
– Items based on more extensive reading
– Decreased test duration
• Solicit discussion and feedback
Current PCAT Blueprint
Number of
Number of
Operational
Experimental
PCAT Subtest
Items
Items
Part 1: Writing
1 Prompt (operational)
Part 2: Verbal Ability
40
8
Analogies
25
5
Sentence Completion
15
3
Part 3: Biology
40
8
General Biology
20–22
4
Microbiology
7–8
1–2
Anatomy & Physiology
12–13
2–3
Part 4: Chemistry
40
8
General Chemistry
20–22
5–6
Organic Chemistry
12–13
2–3
Basic Biochemistry Processes
7–8
1–2
Rest Break
Part 5: Writing
1 Prompt (experimental)
Part 6: Reading Comprehension
Comprehension
Analysis
Evaluation
Part 7: Quantitative Ability
Basic Math
Algebra
Probability & Statistics
Pre-Calculus
Calculus
Total Test
40 / 5 passages
12–13
15
12–13
40
5–6
7–8
7–8
9–10
9–10
200 multiplechoice items + 1
writing prompt
8 / 1 passage
2–3
3–4
2–3
8
1–2
1–2
1–2
2–3
2–3
Time Allowed
30 min.
30 min.
30 min.
30 min.
15 min.
30 min.
50 min.
40 min.
40 multiple-choice
240 min. = 4 hrs.
items + 1 writing
+ Rest Break
prompt
Options for 2014-2015
• Goal is to move towards items that require higher order
thinking and reasoning
• Field-test items for Biology, Chemistry, and with more
extensive passages (4 items per passage)
• Field-test Quantitative items with problem or scenario
stated in stem text
• Field-test Reading Comprehension items with humanities
and Social Science content
• Remove Verbal Ability – need your feedback
– Would allow more time to test new item types in Biology,
Chemistry and Quantitative sections
Proposed PCAT Test Blueprint for 2014–15: Field-Testing New Item Types
PCAT Subtest
Part 1: Writing
Part 2: Verbal Ability
Analogies
Sentence Completion
Part 3: Biology
General Biology
Microbiology
Anatomy & Physiology
Part 4: Chemistry
General Chemistry
Organic Chemistry
Basic Biochemistry Processes
Part 5: Reading Comprehension
Comprehension
Analysis
Evaluation
Part 6: Quantitative Ability
Basic Math
Algebra
Probability & Statistics
Pre-Calculus
Calculus
Total Test
Number of
Number of Experimental
Operational Items
Items / Passages
1 Prompt (operational)
40
0
25
0
15
0
8 / 2 (experiment, scenario, or
40
other context in passage)
20–22
4
7–8
1–2
12–13
2–3
8 / 2 (experiment, scenario, or
40
other context in passage)
20–22
5–6
12–13
2–3
7–8
Rest Break
40 / 5 passages
12–13
15
12–13
40
5–6
7–8
7–8
9–10
9–10
200 multiple-choice
items + 1 writing
prompt
Time Allowed
30 min.
25 min.
35 min.
35 min.
1–2
15 min.
8 / 1 (humanities or social
sciences content)
2–3
3–4
2–3
8 (problem or scenario stated
in stem text)
1–2
1–2
1–2
1–2
1
32 multiple-choice items
50 min.
45 min.
220 min. = 3 hrs.
40 min. + Rest
Break
Looking Forward to
2015-2016
• Goal is to increase testing of critical thinking
• Biological and Chemical processes: 40-50 percent of
items with passages
• Critical Reading: 40-60 percent of items from
humanities and social sciences
• Quantitative Reasoning
– 40-50 percent of items with stem text set-up
– Less emphasis on calculus and pre-calculus
Proposed PCAT Test Blueprint for 2015–16: Introducing New Operational Items
Operational Experiment
Items /
al Items /
Time
PCAT Subtest
Passages
Passages
Allowed
Part 1: Writing
1 Prompt (operational)
30 min.
Part 2: Biological Processes
40 Items / 4–5 8 Items / 2
(40–50% of items with passages)
Passages
Passages
45 min.
General Biology
20–22
4
Microbiology
7–8
1–2
Anatomy & Physiology
12–13
2–3
Part 3: Chemical Processes
40 Items / 4–5 8 Items / 2
(40–50% of items with passages)
Passages
Passages
45 min.
General Chemistry
20–22
5–6
Organic Chemistry
12–13
2–3
Basic Biochemistry Processes
7–8
1–2
Rest Break
15 min.
Part 4: Critical Reading
8 Items / 1
(40–60% of items with humanities
40 Items / 5
Passage
or social science passages)
Passages
50 min.
Comprehension
12–13
2–3
Analysis
15
3–4
Evaluation
12–13
2–3
Part 5: Quantitative Reasoning
(40–50% of items with stem text
set-up)
40
8
45 min.
Basic Math
9–10
1–2
Algebra
9–10
1–2
Probability & Statistics
7–8
1–2
Pre-Calculus
7–8
1–2
Calculus
5–6
1
160 multiple32 multiple- 215 min. = 3
choice items
Total Test
choice
hrs. 35 min.
+ 1 writing
items
+ Rest Break
prompt
Situational Judgment Tests (SJTs)
• Test which presents realistic,
hypothetical scenarios
• SJTs not simply “off the shelf” tests, but
are designed to address specific role
requirements
• Have been used in functions such as
personnel selection
Desired SJT Assessments
Presented to Pearson
•
•
•
•
•
•
Empathy and altruism
Ethical reasoning, integrity
Interpersonal skills
Team player
Self awareness
Problem solving, creativity
Sample SJT
(Available at: http://www.assessmentday.co.uk/situationaljudgement-test/SJT-Questions.pdf)
SJT: Current Status
• Pearson’s Talent Team has been
further researching SJTs this spring
• Update to PCAT Advisory Committee
due at its meeting here in Chicago
• AAMC also looking at SJTs this summer
Questions for Today’s
Audience
Future Follow-up
• A call for feedback will be sent to
Student Services SIG: due August 1st
• Webinar(s) being planned:
Tuesday, 9/17/13 from 2 - 3 p.m. CT
Wednesday, 9/18/13 from 10 -11
a.m. CT.
Discussion and Questions