Transcript How to Improve a Course - Rosalind Franklin University

HOW TO BE A GOOD COURSE DIRECTOR
PART 2 HOW TO IMPROVE A COURSE
Master Teacher Guild
Mike Fennewald, PhD (SGPDS)
Daniel Bareither, PhD (SCPM)
Lecia Apantaku, MD (CMS)
Roberta Henderson, PT, PhD (CHP)
Fred Sierles, MD (CMS)
John Becker (SCPM)
December 15, 2011
1
WHO IS RESPONSIBLE FOR
IMPROVEMENT?
• Course Director:
– This individual has been selected by the department
and the college to coordinate the learning activities for
the course (or clerkship). They must understand that
the course is not THEIRS but rather they have the
responsibility for providing an excellent educational
experience for the students. They are responsible for
the selection and evaluation of the faculty that
participate in the course.
• Department Chairs:
– Responsible for selecting qualified faculty to serve as
course directors and for reviewing their performance.
2
WHO IS RESPONSIBLE FOR
IMPROVEMENT?
• Deans:
– Responsible for making sure the departments are providing
excellent educational experiences, that there is assessment
of the curriculum, and that the students are achieving the
learning outcomes.
• Assessment Committee:
– Responsible for coordinating the methods of assessment
used to collect data on the curriculum, the analysis of the
data, and identifying any problems and making
recommendations to the Curriculum Committee and Dean.
• Curriculum Committee:
– Responsible for management of the curriculum and
taking action regarding recommendations from the
Dean and Assessment Committee.
3
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Course/Clerkship Evaluations student input regarding:
– Organization of the course/clerkship
– Learning objectives
– The learning outcomes that the
course/clerkship helps to develop
– Method of evaluation
– Use of learning management system
(D2L)
– Integration of basic science and
clinical science information
– Faculty availability
– Faculty contribution to their
understanding of the course material
– Constructive comments
•
Note: Process for review and
feedback from evaluations
(assessment committee).
4
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Process:
– Development of the course evaluation (assessment and curriculum
committees)
– Determine method for students to complete course evaluation
– Evaluation results compiled by someone other than the course
director
– Evaluation results provided to the department chair
– Department chair reviews evaluation results with the course
director and recommends any actions to be taken
– Evaluation results and recommended actions forwarded to the
Assessment Committee and Dean
– Assessment Committee makes recommendations to the Curriculum
Committee and/or Dean
– If recommendation is to the Curriculum Committee then
they make a recommendation to the Dean.
•
Note: Correspondence from Dr. Marinelli regarding review of the
graduate curriculum and the Oversight Committee
5
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Peer Evaluations – faculty input
regarding:
– Course materials available to
students
– Established learning objectives
– Quality of visual materials
– Instructor – knowledge, interaction
with students, utilization of AV,
professional conduct
– Presentation – organized,
understandable, emphasized major
points, integrated basic science
and clinical information, delivered
at a good pace, timely
•
Note: Need a process for review and
feedback from evaluations
(department chair).
6
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Assessment Committee – identifies a variety of internal and
external methods to assess the achievement of the student
learning outcomes/competencies, and the achievement of the
programmatic outcomes, and curriculum/teaching:
– Internal assessment -
• ongoing performance in a course (individual exams)
• overall course performance (grade distribution) compared to other
courses occurring at the same time
• mean of the overall course performance compared to previous years
• performance on comprehensive exams
• course evaluations
• curriculum evaluations
• peer evaluations
– External assessment •
•
•
•
performance
performance
performance
performance
on board exams
in clerkships
in residency
on licensure exams
7
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Curriculum Committee:
– major mechanism by which the faculty develop and review the
curriculum
– coordinates the process for course evaluations
– coordinates the review of the syllabi, the methods of instruction,
the scheduling of courses, the size and sequencing of courses,
and receives and evaluates course proposals
– annual evaluation of the curriculum to look at the sequencing of
courses, the integration of basic science and clinical science
information, the development of critical thinking, and the
utilization of the learning management system
– develops and reviews academic policies
• Student Focus Groups:
– focus groups of students can be used to obtain additional
information on a particular issue identified in the
assessment results
8
WHAT INFORMATION CAN BE USED FOR
IMPROVEMENT?
• Curriculum Committee:
– major mechanism by which the faculty develop and review
the curriculum
– coordinates the process for course evaluations
– coordinates the review of the syllabi, the methods of
instruction, the scheduling of courses, the size and
sequencing of courses, and receives and evaluates course
proposals
– annual evaluation of the curriculum to look at the
sequencing of courses, the integration of basic science and
clinical science information, the development of critical
thinking, and the utilization of the learning management
system
– develops and reviews academic policies
9
FACULTY PERFORMANCE AND PROBLEMS
•
•
•
•
•
Prevent problems.
Monitor teaching closely.
Continue to orient, mentor and monitor.
Expect, praise and reward improvement.
If a few cycles fail to yield improvement,
reassign the faculty member.
10
Prevent Problems (Primary Prevention)
• Primary prevention. Prevent problems
before they occur.
– Consider teaching ability when hiring.
– Assign classes to best teachers
• Not necessarily most expert on the topic
– Orient and mentor new teachers.
• Prepare new teachers for first class.
11
Primary Prevention (cont.): Reward Good
Teaching
• Reward Good
Teaching
– Offer praise,
encouragement.
– Suggest—to your
chair—raises, travel
support, promotion,
tenure.
– Write
recommendation
letters.
– Nominate for
teaching awards.
– In considering
tenure, your
chair (with your
input) should
ask, “What will
he or she do if he
or she loses this
grant?”
12
Monitor Teaching Closely
• For new or “struggling” teachers
– Attend, help set up, give “tech
support” for
• First few lectures, some (or all)
subsequent lectures
• Early small group, occasional
subsequent small groups
13
Monitor Teaching Closely: Prompt Oral
Feedback
• Give prompt oral feedback.
– During a break
– Soon after class (same day, next couple of
days)
• How to give feedback.
– Begin with praise.
– Empathize with faculty member’s feelings,
perceptions and experiences in the class.
– Follow with constructive criticism.
– Summarize hopefully, end on a positive
note.
14
Monitor Teaching Closely: Review Written
Feedback
• Obtain, read, distribute and review with
faculty member
– Student evaluations of teaching, test item
preparation
– Student exam performance on faculty
member’s topic
• Department exams
• National exams
15
Monitor Teaching Closely: Keep Chair
Posted; Maintain Records
• Keep chair apprised of, and give chair access
to, above indicators.
• Obtain and maintain records of feedback
received by faculty member.
– Documentation crucial if faculty member
challenges decisions based on his or her
teaching.
16
Follow-up Beyond Early Interactions and
Feedback
• Continue to orient, mentor, monitor.
• Expect and praise each effort and success at
improvement.
• If sub-par teaching and failure to improve
continue after two or three cycles of
monitoring and feedback,
– Suggest, to chair, reassignment of faculty
member to other duties.
17
Course Director Vignettes
What Would You Do in Each Case?
18
Vignette 1
• The Course Director is attending a two hour lecture
presented by an early-career faculty member who is
bright, confident and flexible. During the first 25
minutes of the first 50 min., the lecturer gives
multiple colorful, vivid, relevant illustrations of
patients she has treated, regularly poses questions to
the class, and the class is attentive and energized.
Inexplicably, during the second 25 min. of the first 50
min., the lecturer provides no illustrations, poses no
questions, and many class members—as well as the
Course Director, nod off periodically to sleep. The
Course Director should…
19
Vignette 2
• A part-time associate professor of psychiatry with nine
years’ teaching experience gives a two hour, first-time, onepresentation per year lecture on a topic on which she is
expert. She presents the topic at the intellectual level of an
introductory college psychology class, a topic that is already
“known by heart” by most of the students. Her voice is
monotonous. She poses no questions to the class, takes no
questions, and gives only two illustrative examples of
patients she has seen. During the class, several students—
and the Course Director—“nod off.” Post-course, the
students give her low ratings, and their narrative comments
point out the weak points noted above. The Course Director
should…
20
Vignette 3
• In the anonymous student feedback following a
third year required psychiatry clerkship, two
students give an assistant psychiatry professor an
overall rating of “F,” and each of them mentions—
in the narrative—something like, ”Just awful. The
only thing he had us do was draw blood for his lab
experiments. He gave us no patient care
responsibilities and spent virtually no time
teaching us.” This fits with this faculty member’s
previous behavior. The Clerkship Director
should…
21
Vignette 4
• In the written anonymous student feedback
following a third year required psychiatry
clerkship, five of the 23 clerks spontaneously
mention, in their narrative comments, something
like: “Resident Dr. A gave us awesome off-thecuff mini-lectures, and is a terrific supervisor and
clinician.” Prior to his psychiatry residency, Dr. A
published 17 psychiatry papers in refereed
journals. The Clerkship Director should…
22
STUDENT PERFORMANCE
Using internal measures to adjust curriculum
Exams
Clinical Performance
End of course evaluations
Using external measures to adjust curriculum
National Board of Medical Examiners (NBME) subject
exams
Residency director questionnaires
Medical School Graduation Questionnaires
23
NBME SUBJECT EXAMINATION
CONTENT AREA ITEM ANALYSIS REPORT
24
NBME STEP I
25
Medical School
Graduation Questionnaire
2011 All Schools Summary Report
26
MEDICAL SCHOOL GRADUATION
QUESTIONNAIRE
27
MEDICAL SCHOOL GRADUATION
QUESTIONNAIRE
28
M1 AND M2 QUESTIONNAIRES
Please rate the professionalism with which you were treated in this course by the course
faculty.
Excellent, Good, Adequate, Unacceptable, Cannot evaluate
Please rate the professionalism with which other students were treated in this course by
the course faculty.
Please rate the professionalism with which you were treated by your fellow students in
this course.
Additional comments on professionalism in this course:
Lecturers and Notes/Labs:
Please rate Dr. _______’s lectures:
Please rate Dr. _______’s notes and related materials (e.g. audio, video, PowerPoint):
Constructive comments.
29
M1 AND M2 QUESTIONNAIRES
I attended about _______ of the lectures:
The D2L site for this course was:
Excellent
Good
Adequate
Unacceptable
Cannot evaluate
The clinical relevance of this course was:
The effectiveness of the exams to assess course objectives was:
Integration of course content with other courses was:
This course overall was:
30
M3 QUESTIONNAIRE
Question 1
Please rate how well the specific clerkship objectives and clerkship policies (e.g. grading
criteria, examination format) were identified early in the clerkship.
Superior Very Good Acceptable Deficient
Question 2
Please rate how well significant facts, principles, and basic science concepts in this
clinical area were explained.
Question 3
Please rate how well lectures and/or seminars related to the objectives.
Question 4
Constructive written comments on the lectures and/or seminars in this clerkship:
box)
(text
Question 5
Please rate how well chart, bedside or team rounds were conducted. Please rate how
patient care opportunities directly applied the facts, concepts and principles presented in
the didactic sessions.
31
RESIDENCY PROGRAM DIRECTOR
QUESTIONNAIRE
•
•
•
•
•
•
•
Can collect data from the following sources: patient history or
history from other relevant information sources (e.g. the family)
and physical exams.
Has basic science knowledge sufficient to interpret data gathered
from the patient interview and physical examination.
Knows the sources available for securing additional data through
laboratory procedures, and can interpret such data.
Can construct a differential diagnosis and a plan for its
confirmation or rejection.
Treats each patient as an integrated whole, rather than as a
disease entity.
Uses current literature and other educational resources to
understand and resolve diagnostic problems.
Can develop multidimensional treatment plans which include
diet, place of care, use of medications and therapeutic
procedures, and which take into account the patient’s
socioeconomic and psychological status.
32
RESIDENCY PROGRAM DIRECTOR
QUESTIONNAIRE
• Learns new concepts in medical literature.
• Critically evaluates new concepts and interprets new data
• Respects other health professionals and facilitates the development
of their skills.
• Knows concepts of primary prevention, epidemiology and healthcare
delivery and accounts for socioeconomic status.
• Participates effectively in peer review, especially in providing
constructive criticism of professional behavior.
• Able to choose a career track appropriate to his/her abilities and
performance.
• Has the basics of comprehensive health care delivery, regardless of
long-term specialty choice.
• Is self-directed in solving problems.
33
LEARNING ENVIRONMENT
Using internal and external data to
improve the learning environment
Space
Technology
34
CONTINUOUS QUALITY IMPROVEMENT
How do you measure success?
Set goals
Re-measure
35