Transcript Assessing verbal communication skills of medical students

Assessing verbal
communication skills
of medical students
J Voges
E Jordaan *
L Koen
DJH Niehaus
Department of Psychiatry, University of Stellenbosch and
Stikland Hospital
* Biostatistics Unit: Medical Research Council, Bellville
Positioning of the study

Large project:
 Correlation
of communication skills with
academic performance of medical students

Sub-studies:
 Facial
affect recognition
 Oral examination marks in psychiatry
 Non-verbal communication skills
 Verbal communication skills
Introduction
Communication is one of six required
competencies identified by the ACGME
 Effective communication associated with:

 Improved
patient and doctor satisfaction
 Treatment compliance
 Strong predictor of medical school success

Assessment of communication skills is
complex and difficult to implement
Aim
First phase:
 To evaluate the usability of the Liverpool
Communication Skills Assessment Scale for
assessing the communication skills of medical
students of the University of Stellenbosch
Second phase:
 To determine effectiveness of undergraduate
medical students’ communication skills using the
Liverpool Communication Skills Assessment
Scale
 To determine if there is a correlation between
communication skills and overall academic
performance
Measurement equivalence
Central issue in determining the
applicability of instrument cross-nationally
and cross-culturally
 Factors to consider:

 Content
equivalence
 Semantic equivalence
 Technical equivalence
 Criterion equivalence
 Conceptual equivalence (Flaherty et al, 1988)
Methods

Subjects:
 Medical
students completing late rotation
 5 min. semi-structured interview with patient
that was videotaped
 Permission granted by Faculty of Health
Sciences and Ethics committee of SU

Venue:
 5-week
Psychiatry rotation at Stikland hospital
Methods
 Assessment tool:






Raters:




Liverpool Communication Skills Assessment Scale (LCSAS)
Consists of 12-items measuring several aspects of
communication
Mixed method of using both a checklist and a rating approach
4-point ordinal rating scale ranging from Unacceptable to Good
Ease of use, acceptable reliability
2 independent raters, additional training
Third rater included, instruction given
Help sheet with additional descriptors to guide scoring
Primary statistical evaluation:

Inter-rater reliability


Marginal homogeneity (Chi-square statistic, p-value<0.01 as
significant)
Agreement (Cohen’s weighted Kappa index for ordinal data)
Results: Distribution of score by rater

Intra-class correlation coefficient = 0.8 (0.71-0.87)
Results: Distribution of score by item
Results: Distribution of score by item
Results: Distribution of score by item
Agreement between raters

Agreement for items that had marginal homogeneteity
(Cohen’s weighted Kappa index for ordinal data, 95%
confidence interval)
Items
Rater 1 vs. Rater 2
Rater 1 vs. Rater 3
Item 1: Greeting
0.71 (0.52-0.91)
0.54 (0.33-0.75)
0.57 (0.38-0.77)
Item 2:
Introduction
0.76 (0.63-0.70)
0.79 (0.66-0.92)
0.92 (0.83-1.00)
Item 4: Eye-contact 0.66 (0.41-0.91)
0.38 (0.10-0.65)
0.69 (0.47-0.91)
Item 8: Questions
0.37 (0.14-0.60)
0.55 (0.37-0.74)
0.62 (0.43-0.81)
Rater 2 vs. Rater3
Discussion





LCSAS evaluated for usability to measure
communication skills in medical students
Additional training
Inclusion of additional rater and help sheet
Total score: high level of correlation
Inter-rater reliability

Marginal homogeneity




4 of 12 items
Agreement
Additional training – greater agreement
Reliability of measure:


Continue with development and standardisation of assessment
scale for use in South Africa
Training
Limitations and future directions

Limitations:
 Small
sample
 Inter-rater reliability
Marginal homogeneity and agreement
 Training


Future directions:
 Re-evaluation
of the scoring categories to
promote understanding
 Language
 Gender
 Culture
Conclusion
Use of LCSAS in South African setting
 Correlation for total score
 Training necessary to improve agreement
for each item
 Further development necessary
 Use in education context of South Africa
by various health professionals

Selected references




Epstein, R.M. Campbell, T.L., Cohen-Cole, S.A., McWhinney,
I.R. & Smilkstein, G. (1993). Perspectives on patient-doctor
communication. Journal of Family Practice 37(4): 377–388.
Flaherty, J.A., Gaviria, F.M., Pathak, D., et al. (1988). Developing
instruments for cross-cultural psychiatry needs. Journal of
Nervous and Mental Disorders 176(5): 257-263.
Humphris, G.M. & Kaney, S. (2001). The Liverpool Brief
Assessment System for Communication Skills in the Making of
Doctors. Advances in Health Sciences Education 6: 69–80.
Parker, G. (1993). On our selection: predictors of medical school
success. Medical Journal of Australia 158(11): 747–751.
Project supported by funding from FINLO
Faculty of Health Sciences