(O-D1) Simulation-based Assessment for the Emergency Medicine Milestones

Objectives: The purpose of this study is to identify the ACGME milestones that are the most difficult to assess using traditional methodology and the most suitable milestones to be assessed using simulation.

Background: The Accreditation Council for Graduate Medical Education (ACGME) has recently revised the educational milestones for all accredited residencies programs. The Emergency Medicine (EM) Milestones 2.0 contains an updated specialty -specific, competency -based behavioral anchors for the assessment of residents. Most programs use their current assessment methods to fulfill data points for these milestones subcompetencies rather than devise new tools. This has resulted in subcompetencies that are difficult to assess using traditional methods. Simulation based medical education (SBME) measures outcomes based on observational rating, while providing opportunities for formative and summative feedback which can be used as an alternative solution.

Methods: This is a survey-based study that was targeted toward emergency medicine residency programs with simulation fellowship affiliation. The web-based survey contained 12 key questions, which focused on demographics of the program, the educational role of the respondents, frequency and type of simulation used in the program, the most difficult to assess education milestones using traditional assessment methods and most suitable milestones for using simulation – based assessment. The survey was conducted using SurveyMonkey and was sent weekly for 6 weeks to the Program director, associate and assistant program director, and simulation fellowship director that were listed on the program’s website. Descriptive statistics were used to analyze the data for demographic data as well as the total number of votes for each of the 22 EM milestones subcompetencies for each question. The outcome variables for each subcompetency included the number of votes for “most difficult to assess using traditional methodologies” and “best assessed using simulation.” These were counted from both simulation experts and program directors, for a total of 5 non-ranked votes per category.

Results: Thirty eight of 115 respondents completed the survey (33% response rate). The milestone that was ranked most difficult to assess using traditional methodologies was Systems- based practice: Quality Improvement. The milestone identified by most respondents as most suitable for assessment using simulation was Patient care: Emergency Stabilization. There was no overlap between the two categories of milestone subcompetencies.

Conclusions: System-based practice and reflective practice and commitment to personal growth are difficult to assess using traditional methods. Non-traditional assessment methods as well as innovative use of simulation may be helpful in assessing these subcompetencies.

References (Optional):

Funding (Optional): This research received biostatician support via the University of Maryland School of Medicine. We acknowledge the support of the University of Maryland, Baltimore, Institute for Clinical & Translational Research (ICTR) and the National Center for Advancing Translational Sciences (NCATS) Clinical Translational Science Award (CTSA) grant number 1UL1TR003098.