147 - BICA: A Tool to Assess Student Performance in Conflict Resolution OSCEs

Objectives: Interprofessional collaborations are essential for providing quality and efficient care for patients; however, conflict can impair team functioning and result in ongoing issues with morale that negatively impact patient care. Conflict management training can increase trust and cohesion in teams. Previous research demonstrates the utility of using standardized professional (S-Pro) scenarios to evaluate learners’ performances in professional conflict resolution, but it is unclear how learner performance should best be evaluated and whether evaluations are consistent across evaluators from different professional backgrounds. This study aims to determine if ratings of student performance in Objective Structured Clinical Exam (OSCE) scenarios are consistent among evaluators when using a standardized rating tool.

Design: S-Pro OSCEs were designed to teach conflict resolution to 4^th year medical students. Student performance was rated by an interprofessional panel. Evaluators were trained with lectures, OSCE descriptions, video recordings, and group evaluations. The Brody Interpersonal Conflicts Assessment (BICA) tool was used to assess performance in conflict resolution. The conflict management style within the tool is broken down into nonverbal and verbal communication. Scores were analyzed with intraclass correlation coefficient (ICC), a two-way mixed model with rater fixed effects.

Results: Twenty-five medical students participated in the OSCE scenarios conducted by five different standardized professionals. Each OSCE was graded by six different evaluators. For verbal communication, the consistency of agreement of mean scores was high at 0.753. However, the ICC for mean scores in nonverbal communication was considerably lower at 0.441. The overall performance (combined verbal and nonverbal) is high with the ICC average at .702.

Conclusions: With training, evaluators where consistent in their assessments of participants, particularly in the verbal communication and overall performance domains. These results indicate that the BICA tool can be used as an objective method to assess students’ conflict resolution skills.