(P25) Stop! Collaborate and Evaluate: How a Stop Light Timer Impacts Evaluation Completion Rates in Emergency Medicine Didactic Sessions

Objectives: The purpose of this study was to evaluate the impact of a stop light timer on the quantity and length of evaluations completed by Emergency Medicine team members attending an academic residency's didactic sessions.

Background: Emergency medicine residents require a deep understanding of medical concepts and the ability to apply them in high-pressure situations. Didactic sessions are crucial in providing the necessary knowledge and skills, and their evaluation is essential to assess teaching effectiveness, identify areas for improvement, and enhance the learning experience. Evaluating content, delivery, and learner engagement helps educators adjust their teaching methods and improve education quality. Effective evaluation benefits both learners and educators, so emergency medicine residency programs should prioritize it to ensure residents receive the best education and are prepared for real emergencies.

Methods: In an effort to ensure adequate time for evaluation and cognitive rest between lectures, emergency medicine residents, faculty, nurses, and technicians participated in an intervention involving the implementation of a stoplight timer during didactic sessions. The timer ensured an adequate break before the next session by showing a yellow light after 43 minutes, a red light at 50 minutes, followed by a verbal cutoff at 53 minutes. The intervention aimed to address the common issue of didactic sessions going over their time limit. Attendees used Airtable, a data collection platform, to record evaluations of lectures before and after the 13-week intervention, allowing measurement of the average length of response. Data analysis involved a t-test to compare the average number of evaluations per session and the length of responses before and after the intervention, to determine if there was a significant difference. This intervention was implemented to improve evaluation participation and quality, and data analysis aimed to determine if the stop light timer was effective in achieving this goal.

Results: The results of the study suggest that the implementation of a stoplight timer during didactic sessions has had an impact on evaluation completion rates and the content of the evaluations. Specifically, the number of evaluations completed per session decreased from 16.8 to 14.1 (p=0.02), indicating that fewer attendees completed evaluations after the intervention. The word count per evaluation also decreased from 13.1 to 11.5, although the p-value for this finding was only borderline significant at p=0.06. The type of attendees and the content of the didactic lectures were not recorded or analyzed. Occasionally didactic days had fewer or additional sessions, and in total there were 43 sessions that were evaluated during the first 13 weeks and 42 total sessions during the second 13 weeks.

Conclusions: Implementation of a stoplight during didactic sessions was not associated with increased evaluation rates or length of evaluations. There are numerous factors that could contribute to the general downward trend in evaluations over the course of the academic year despite the introduction of a stoplight to provide increased evaluation time between didactic sessions. Important limitations include the following: The population studied had a significant capacity to change as only the residents were required to be present at didactics. The time of year may have influenced participation and response count. Providing a stoplight timer may have increased cognitive offload and socialization too much, detracting from completing evaluations. Further analysis on the impact of learner satisfaction is warranted.

References (Optional):

Funding (Optional): No monies have funded this research