Administration of the American Board of Anesthesiology’s virtual APPLIED Examination: successes, challenges, and lessons learned.

Publication date: Jul 11, 2024

In response to the COVID-19 pandemic, the American Board of Anesthesiology transitioned from in-person to virtual administration of its APPLIED Examination, assessing more than 3000 candidates for certification purposes remotely in 2021. Four hundred examiners were involved in delivering and scoring Standardized Oral Examinations (SOEs) and Objective Structured Clinical Examinations (OSCEs). More than 80% of candidates started their exams on time and stayed connected throughout the exam without any problems. Only 74 (2. 5%) SOE and 45 (1. 5%) OSCE candidates required rescheduling due to technical difficulties. Of those who experienced “significant issues”, concerns with OSCE technical stations (interpretation of monitors and interpretation of echocardiograms) were reported most frequently (6% of candidates). In contrast, 23% of examiners “sometimes” lost connectivity during their multiple exam sessions, on a continuum from minor inconvenience to inability to continue. 84% of SOE candidates and 89% of OSCE candidates described “smooth” interactions with examiners and standardized patients/standardized clinicians, respectively. However, only 71% of SOE candidates and 75% of OSCE candidates considered themselves to be able to demonstrate their knowledge and skills without obstacles. When compared with their in-person experiences, approximately 40% of SOE examiners considered virtual evaluation to be more difficult than in-person evaluation and believed the remote format negatively affected their development as an examiner. The virtual format was considered to be less secure by 56% and 40% of SOE and OSCE examiners, respectively. The retirement of exam materials used virtually due to concern for compromise had implications for subsequent exam development. The return to in-person exams in 2022 was prompted by multiple factors, especially concerns regarding standardization and security. The technology is not yet perfect, especially for testing in-person communication skills and displaying dynamic exam materials. Nevertheless, the American Board of Anesthesiology’s experience demonstrated the feasibility of conducting large-scale, high-stakes oral and performance exams in a virtual format and highlighted the adaptability and dedication of candidates, examiners, and administering board staff.

Open Access PDF

Concepts Keywords
American Virtual APPLIED Examination
Conducting Virtual candidate experiences
Covid Virtual examiner experiences
Exams
Pandemic

Semantics

Type Source Name
disease MESH COVID-19 pandemic
disease VO time
disease MESH retirement
drug DRUGBANK Etoperidone
disease IDO process
disease IDO site
drug DRUGBANK Coenzyme M
disease VO volume
disease VO organization
drug DRUGBANK Methionine
disease VO effective
drug DRUGBANK Cysteamine
drug DRUGBANK Tropicamide
drug DRUGBANK Pentaerythritol tetranitrate
disease MESH aids
drug DRUGBANK Trestolone
disease MESH Emergency
disease IDO country
disease VO vaccination
disease IDO contact tracing
drug DRUGBANK Spinosad
disease IDO quality
disease VO USA

Original Article

(Visited 1 times, 1 visits today)