(O-F7) Augmented Reality for Empathy Training: Stepping into the Patient’s Shoes

Objectives: The patient-physician relationship and satisfaction are highly reliant on effective physician communication.1, 2 We proposed that by utilizing the Microsoft HoloLens augmented reality to record and watch medical student standardized patient (SP) encounters students would be more empathetic and imagine what it is like to be “in the patient’s shoes.”

Background: Although virtual reality has been widely adopted for training practical skills, virtual and augmented reality are minimally utilized to improve MSs’ empathy skills.3 MS have limited options of reviewing their SP encounters and typically can only view footage from one camera angle. This limited point of view (POV) does not give a representation of eye contact, body language, or communication directly from the patient’s POV. We initiated a pilot project utilizing Microsoft HoloLens augmented reality to give MSs an opportunity to view themselves from the patient’s POV. The figurative “putting yourself in the patient’s shoes” became lliteral with digital education.

Methods: To analyze the efficacy of augmented reality video, we designed a pilot study that evaluated students’ self perception of their performance immediately before and after watchback sessions. 20 MS1s and MS2s were randomly assigned into two groups, those which would review their video footage captured on the HoloLens, and those which would review their video footage captured from a standard camera. Students completed a 5 minute SP encounter, with all SPs wearing the HoloLens during the encounter. Students then completed the Consultation and Relational Empathy (CARE) validated survey, and additional questions evaluating body language, eye contact, and facial expressions. Students assigned to the HoloLens group then had the opportunity to wear the device and experience themselves delivering their 5 minute patient interview in an immersive augmented reality experience, from the patient’s POV. The other group of students watched the third-person POV of their encounter. After completing the watchback sessions, students repeated the CARE survey and additional questions, and provided feedback on the experience. Changes in CARE survey scores, evaluation of body language, eye contact, and facial expressions. Each group then reviewed the alternative video and completed a final survey on their overall thoughts comparing HoloLens to the third-person POV camera.

Results: Each participant was able to view their encounter from the augmented reality and third-person perspective and completed a final survey--84% (16/19) marked the HoloLens footage as “more informative” versus the third-person camera. Many of the students’ reviews of the experience included descriptions of evaluating body language and facial expression with patients, and seeing their mannerisms from a new perspective. 15/19 participants noted in their free response that the HoloLens was better than the third-party perspective for facial expressions, eye contact, and seeing from the patient’s perspective. The third-person perspective was better for overall body language.

One student stated that “It was great to see myself from the patient’s perspective (HoloLens) and see the importance of body language and facial expressions”.

Other notable quotes regarding the HoloLens included “HoloLens helped me empathize better with the patient.” “The HoloLens footage gave me a more detailed look into my facial expressions and how I was translating empathy through small acts of non-verbal communication (eyebrow raise, eye contact)."

All 20 students stated that they felt the experience was valuable to their clinical practice, that they would participate in a study like this again and would recommend the session to a colleague.

Conclusions: This pilot study provided strong beneficial evidence to using augmented reality in medical communication training. The overwhelmingly positive reviews suggest that using augmented reality video feedback during SP encounters is an important supplement to traditional education and allows MSs to experience what it is like to be in a patient’s shoes.

References (Optional): [1] Buller MK, Buller DB. Physicians' communication style and patient satisfaction. J Health Soc Behav. 1987;28(4):375-388.
[2] Hannawa AF. "Explicitly implicit": examining the importance of physician nonverbal involvement during error disclosures. Swiss Med Wkly. 2012;142:w13576. Published 2012 May 9. doi:10.4414/smw.2012.13576
[3] Kyaw BM, Posadzki P, Paddock S, Car J, Campbell J, Tudor Car L. Effectiveness of Digital Education on Communication Skills Among Medical Students: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019;21(8):e12967. Published 2019 Aug 27. doi:10.2196/12967

Funding (Optional): None