Extreme Risk Protection Orders: An Opportunity to Improve Gun Violence Prevention Training States are increasingly enacting extreme risk protection order (ERPO) laws, also known as “red flag” or gun violence restraining order laws, as one part of a multidisciplinary approach to address the national gun violence epidemic. Passed into law in more than 10 states and under consideration by legislatures in approximately 30 others, ERPO laws create a legal process to temporarily remove firearms from people who may pose a risk to themselves or others. By enabling family or household members, law enforcement, and, in some cases, health care professionals to petition courts when they are concerned about a potential crisis, these laws can potentially prevent firearm-related violence and save lives. Most states with ERPO laws do not give health care professionals a direct role in filing petitions; still physicians may serve as a resource for patients or their families by counseling on firearm safety and raising awareness of this legal pathway. In this way, the success of ERPO laws depends, in part, on the ability of physicians to accurately assess risk. However, physicians are often not proficient in making these types of risk assessments, due largely to insufficient training, particularly in the context of firearm-related violence. The authors review the literature on physician skill in violence-related risk assessment, medical education in gun violence prevention, and the capacity for training to improve such risk assessments. The authors then make recommendations for integrating focused gun violence prevention training into undergraduate, graduate, and continuing medical education, reviewing notable examples. The authors have informed the journal that they agree that both S. Gondi and A.G. Pomerantz completed the intellectual and other work typical of the first author. Acknowledgments: The authors wish to thank Edward Hundert, MD, dean of medical education at Harvard Medical School, for providing information about risk assessment-related curricula. Funding/Support: Work supported by the Carney Family Foundation (C.A. Sacks). Other disclosures: None reported. Ethical approval: Reported as not applicable. Correspondence should be addressed to Suhas Gondi, 25 Shattuck St., Boston, MA 02215; email: suhas_gondi@hms.harvard.edu; Twitter: @suhas_gondi. © 2019 by the Association of American Medical Colleges

“Progress in Medicine Is Slower To Happen”: Qualitative Insights Into How Trans and Gender Nonconforming Medical Students Navigate Cisnormative Medical Cultures at Canadian Training Programs Purpose: Trans and gender nonconforming (TGNC) people face significant health disparities compared to their cisgender (non-trans) counterparts. Physician-level factors play a role in these disparities, and increasing the participation of individuals from sexuality and gender minority (SGM) communities in medical training has been proposed as one way of addressing this issue, however very little is known about the experiences of current TGNC medical students. This study aimed to understand the experiences of TGNC medical students in Canada. Method: Between April 2017 and April 2018, seven TGNC participants either currently enrolled in or recently graduated from a Canadian medical school completed audio-recorded semi-structured interviews. Interviewers asked about experiences with admissions; academic, clinical, and social environments; and interactions with administration. The authors analyzed interviews using a constructivist grounded theory approach. Results: The authors developed five overarching themes: navigating cisnormative medical culture; balancing authenticity, professionalism, and safety; negotiating privilege and power differentials; advocating for patients and curricular change; and seeking mentorship in improving access and quality of care to TGNC patients. This article focuses on the first theme, with associated subthemes of culture and context, interactions with classmates, curriculum, policy and administration, and gendered spaces. Conclusions: The results of this study delineate heterogeneous experiences of medical cultures with a shared underlying pattern of erasure of TGNC people as both patients and clinicians. Findings were largely consistent with previously published recommendations for improving academic medical institutional climates for SGM people, though the need for access to appropriate gendered spaces beyond washrooms was highlighted. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A727. Acknowledgments: The authors wish to thank the participants who shared their time, energy, and stories, as well as Dr. Meredith Vanstone (McMaster University), Dr. Natasha Johnson (McMaster University), Dr. Claire Bodkin (McMaster University), and Dr. Kira Abelsohn (University of Toronto) for their helpful edits and feedback on the manuscript. Funding/Support: 2017 Ontario Medical Student Association/Associated Medical Services (OMSA/AMS) Research Grant in Compassionate Care. Other disclosures: None reported. Ethical approval: This study was reviewed and approved by the Hamilton Integrated Research Ethics Board Student Research Committee on April 24, 2017, project reference number 3138. Previous presentations: Material from this study was previously presented at the Rainbow Health Ontario Conference, March 23, 2018, Sudbury, Ontario; McMaster Medical Student Research Day, April 25th 2018, Hamilton, Ontario; and the Ontario Medical Student Association Leadership Summit, May 5, 2018, Toronto, Ontario, Canada. Correspondence should be addressed to Kat Butler, Michael G. DeGroote School of Medicine, McMaster University, 1280 Main Street E, Hamilton, ON; email: kat.butler@medportal.ca; Twitter: @heykatbutler. © 2019 by the Association of American Medical Colleges

Closing the Gap Between Preclinical and Clinical Training: Impact of a Transition-to-Clerkship Course on Medical Students’ Clerkship Performance Problem: Medical students typically perform worse on clinical clerkships that take place early in their training compared to those that occur later. Some institutions have developed transition-to-clerkship courses (TTCCs) to improve students’ preparedness for the clinical phase of the curriculum. Yet, the impact of TTCCs on students’ performance has not been evaluated. Approach: The authors developed and implemented a TTCC at Virginia Commonwealth University School of Medicine and measured its impact on students’ clerkship performance. During the 2014-2015 academic year, they introduced a 2-week intersession TTCC. The goal was to improve students’ readiness for clerkships by fostering the knowledge, skills, and attitudes required to care for patients throughout a hospitalization. The TTCC included panel discussions, skills development sessions, case-based workshops, and a 4-station standardized patient simulation. The authors assessed the feasibility of designing and implementing the TTCC and students’ reactions and clerkship performance. Outcomes: The total direct costs were $3,500. Students reacted favorably and reported improved comfort on entering clerkships. Summative performance evaluations across clerkships were higher for those students who received the TTCC with simulation compared to those students who received the standard clerkship orientation (P < .001 - .04, Cohen’s d range = 0.23 - 0.62). This finding was particularly apparent in those clerkships that occurred earlier in the academic year. Next Steps: Future plans include evaluating the impact of the TTCC on student well-being and incorporating elements of the TTCC into the preclinical curriculum. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A729. Acknowledgements: The authors would like to thank Sally Santen, MD, PhD, for critically reviewing an earlier version of this article. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This work was deemed exempt from ethical review by the institutional review board at Virginia Commonwealth University. Previous presentations: This work was presented as an oral abstract at the Association of American Medical Colleges Learn Serve Lead annual meeting in Austin, Texas, on November 5, 2018. Correspondence should be addressed to Michael S. Ryan, Virginia Commonwealth University School of Medicine, 1201 E. Marshall St., Suite 4-200, Box 980565, Richmond, VA 23298-0565; telephone: (804) 828-4589; email: michael.ryan1@vcuhealth.org; Twitter: @MichaelSRyanMD. © 2019 by the Association of American Medical Colleges

Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate? Purpose: Medical educators have developed no standard way to assess the operative performance of surgical residents. Most residency programs use end-of-rotation (EOR) evaluations for this purpose. Recently some programs have implemented workplace-based “micro-assessment” tools that faculty use to immediately rate observed operative performance. The authors sought to determine (1) the degree to which EOR evaluations correspond to workplace-based micro-assessments and (2) which factors most influence EOR and directly observed workplace-based performance ratings and how the influence of those factors differs for each assessment method. Method: In 2017, the authors retrospectively analyzed EOR evaluations and immediate postoperative assessment ratings of surgical trainees from a university-based training program from the 2015-2016 academic year. A Bayesian multivariate mixed model was constructed to predict operative performance ratings for each type of assessment. Results: Ratings of operative performance from EOR evaluations vs. workplace-based micro-assessment ratings had a Pearson correlation of 0.55. Postgraduate year (PGY) of training was the most important predictor of operative performance ratings on EOR evaluations: model estimates ranged from 0.62 to 1.75 and increased with PGY. For workplace-based assessment, operative autonomy rating was the most important predictor of operative performance (coefficient = 0.74). Conclusions: EOR evaluations are perhaps most useful in assessing the ability of a resident to become a surgeon compared to other trainees in the same PGY of training. Workplace-based micro-assessments may be better for assessing a trainee’s ability to perform specific procedures autonomously, therefore, perhaps providing more insight into a trainee’s true readiness for operative independence. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A728. Dedication: Dr. Reed Williams unexpectedly passed away during this study. In this project, as in so many others, Reed’s contributions were substantial. He leaves a hole in the authors’ hearts, and the authors greatly miss his humility, kindness, and generosity. Acknowledgements: The authors would like to thank the Surgical Education Research Fellowship of the Association of Surgical Education, as well as the many institutional members of the Procedural Learning and Safety Collaborative (PLSC) for their support of this project. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: The institutional review board of Yale School of Medicine has granted exemption for this study. Correspondence should be addressed to Samantha L. Ahle, Department of Surgery, Yale University School of Medicine, 330 Cedar Street, FMB 107, New Haven, CT 06519; telephone: (203) 785-7890; email: samantha.ahle@yale.edu; Twitter: @YaleSurgery © 2019 by the Association of American Medical Colleges

Building the Bridge to Quality: An Urgent Call to Integrate Quality Improvement and Patient Safety Education with Clinical Care Current models of quality improvement and patient safety (QIPS) education are not fully integrated with clinical care delivery, representing a major impediment toward achieving widespread QIPS competency among health professions learners and practitioners. The Royal College of Physicians and Surgeons of Canada organized a 2-day consensus conference in Niagara Falls, Ontario, Canada, called Building the Bridge to Quality, in September 2016. Its goal was to convene an international group of educational and health system leaders, educators, front-line clinicians, learners, and patients to engage in a consensus-building process and generate a list of actionable strategies that individuals and organizations can use to better integrate QIPS education with clinical care. Four strategic directions emerged: Prioritize the integration of QIPS education and clinical care; build structures and implement processes to integrate QIPS education and clinical care; build capacity for QIPS education at multiple levels; and align educational and patient outcomes to improve quality and patient safety. Individuals and organizations can refer to the specific tactics associated with the 4 strategic directions to create a roadmap of targeted actions most relevant to their organizational starting point. To achieve widespread change, collaborative efforts and alignment of intrinsic and extrinsic motivators are needed on an international scale to shift the culture of educational and clinical environments and build bridges that connect training programs and clinical environments, align educational and health system priorities, and improve both learning and care, with the ultimate goal of achieving improved outcomes and experiences for patients, their families, and communities. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A730. Acknowledgments: The authors would like to thank the 37 members of the Building the Bridge to Quality Program Advisory Board. We would also like to acknowledge Ms. Ginette Bourgeois of the Royal College of Physicians and Surgeons of Canada for the administrative support provided to the Building the Bridge to Quality initiative. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Correspondence should be addressed to Brian M. Wong, Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Room H466, Toronto, Ontario, Canada M4N 3M5; telephone: (416) 480-6100 ext. 83709; email: BrianM.Wong@sunnybrook.ca. © 2019 by the Association of American Medical Colleges

Realizing One’s Own Subjectivity: Assessors’ Perceptions of the Influence of Training on Their Conduct of Workplace-Based Assessments Purpose: Although assessor training is essential for defensible assessments of physician performance, research on the effectiveness of training programs for promoting assessor consistency has produced mixed results. This study explored assessors’ perceptions of the influence of training and assessment tools on their conduct of workplace-based assessments of physicians. Method: In 2017, the authors used a constructivist grounded theory approach to interview 13 physician assessors about their perceptions of the effects of training and tool development on their conduct of assessments. Results: Participants reported that training led them to realize how variable assessor judgement can be, prompting them to change their scoring and feedback behaviors to enhance consistency. Nonetheless, many participants noted they had not substantially changed their numerical scoring. However, most thought training would lead to increased standardization and consistency among assessors, highlighting a “standardization paradox” in which participants perceived a programmatic shift toward standardization but observed minimal changes in their own ratings. An “engagement effect” was also found: Participants involved in tool development and training cited more learnings than participants involved only in training. Conclusions: Findings suggest that training may help assessors develop awareness of their own subjectivity when judging performance. This learning may prompt behaviors that support rigorous and consistent scoring but may not lead to perceptible changes in assessors’ numeric ratings. Results also suggest that participating in tool development may help assessors align their expectations of physicians with scoring criteria. Overall, results support the continued study of assessor training programs as a means of enhancing assessor consistency. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A733. Acknowledgments: The authors wish to acknowledge William Tays for his integral role in redeveloping the CPSO’s Peer Assessment Program and conducting assessor training sessions. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: The University of Toronto’s research ethics review board granted ethical approval. Previous presentations: This work was originally published, in a different form, as a master of science thesis at the University of Toronto, completed in April 2018. This work was also presented at the University of Toronto’s Department of Medicine Annual Day in June 2018. Correspondence should be addressed to Kathryn Hodwitz, Research and Evaluation Department, College of Physicians and Surgeons of Ontario, 80 College Street, Toronto, Ontario, Canada, M5G 2E2; telephone: 416-967-2600 x522; email: khodwitz@cpso.on.ca; hodwitz@gmail.com. © 2019 by the Association of American Medical Colleges

Career Impact of the Chief Resident in Quality and Safety Training Program: An Alumni Evaluation Purpose: Most evaluations of quality improvement and patient safety (QI/PS) training programs provide inadequate data on their impact on alumni careers and QI/PS involvement. To address this gap, the authors investigated continued participation in and barriers to QI/PS work, employment, and satisfaction with training among alumni of the Department of Veterans Affairs (VA) Chief Resident in Quality and Safety (CRQS) program. Method: A cross-sectional, web-based survey was administered in January 2018 to all 238 CRQS program alumni (program years 2009-2017, 54 program sites). Results: A total of 145 alumni (61%) completed the survey, of whom 40% were employed at the VA. Participants reported various professional roles including academic appointments, QI/PS-specific positions, and hospital leadership positions. Most respondents reported involvement in QI/PS activities within the past year including conducting QI or PS projects and teaching QI or PS. Alumni dedicated a median 15% of their work time to QI/PS. Almost all alumni reported experiencing barriers to QI/PS involvement, most frequently lack of time given clinical responsibilities. Most were satisfied with the training, and almost all reported CRQS participation helped their professional career advancement. Conclusions: The continued involvement in QI/PS reported by alumni suggests training programs such as the CRQS program may be successful in building a workforce of leaders equipped to conduct and teach QI/PS. Dedicated time for QI/PS efforts is an important barrier. Future research should address possible career options and assess the larger, overall effect training physicians in QI/PS has on health systems and patient care. Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A732. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This project was reviewed and approved by the Research and Development Committee, White River Junction VA Medical Center (August 4, 2018). Correspondence should be addressed to Maya Aboumrad, Veterans Affairs Medical Center, 215 North Main Street, White River Junction, VT 05009; Maya.Aboumrad@va.gov. Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a “work of the United States Government” for which copyright protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government. © 2019 by the Association of American Medical Colleges

The Inevitable Interim: Transitional Leaders in Academic Medicine This Invited Commentary calls attention to the growing phenomenon of interim leadership, temporary leadership during which a permanent leader is sought, in academic medicine and the gap in intentional, formal leadership training of future health care leaders. The authors consider interim leadership from the perspectives of the organization, the appointing authority, and the interim leader. The authors highlight a fundamental challenge facing interim leaders— providing stability during periods of significant change. They also stress the strategic importance of interim periods and the need for succession planning within academic medicine organizations. Drawing on personal experiences and existing literature, the authors offer a four-stage framework for considering interim leadership at academic medicine organizations and strategies for success at each stage: (1) expectations and exploration, (2) adjusting expectations, (3) accommodation, and (4) phasing out. This Invited Commentary is intended to serve as a resource for interim leaders, those responsible for appointing them, and the organizations they are called upon to lead. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Correspondence should be addressed to Chris Merritt, MD, MPH, MHPE Section of Pediatric Emergency Medicine, Department of Emergency Medicine, Alpert Medical School of Brown University/Hasbro Children’s Hospital, 593 Eddy St, Claverick 264, Providence, RI 02903; telephone: (401) 451-8012; Twitter: @chris__merritt; email: cmerritt@brown.edu. © 2019 by the Association of American Medical Colleges

Technological Pedagogical and Content Knowledge Among Medical Educators: What Is Our Readiness to Teach With Technology? Purpose: This study aimed to empirically assess medical educator knowledge of pedagogy and technology to inform the direction of medical school faculty development efforts. Method: The Technological Pedagogical and Content Knowledge framework (TPACK) survey is a validated instrument for understanding educators’ knowledge of content (CK), pedagogy (PK) and technology (TK) in teaching. A modified version of the TPACK was administered to medical educators (N = 76) at two public institutions, University of California, Irvine (UC Irvine) and University of Colorado (CU). Results: An independent-samples t-test compared TK to PK and CK within each institution. The means of TK (UC Irvine: 3.4; CU 3.4) and both PK for a didactic session (UC Irvine: 3.9; CU: 4.4) and PK for a clinical setting (UC Irvine: 4.0; CU: 4.4) were compared using a t-test and found to be statistically different, P < .01. Similarly, the means of TK and CK (UC Irvine: 4.5; CU: 4.7) were found to be statistically different, P < .01. A Wilcoxon Rank Sum test indicated that the CU PK for a didactic session (Mean: 4.4) was greater than the UC Irvine PK for a didactic session (Mean: 3.9), P < .01. Similarly, the CU PK for a clinical setting (Mean: 4.4) was greater than the UC Irvine PK for a clinical setting (Mean: 4.0), P < .01. Conclusions: There is a clear need for faculty development programs for medical educators to focus on how to teach with technology if medical schools continue to adopt technology within their curricula. Acknowledgements: The authors wish to thank Monica McNulty, Senior Professional Research Assistant/Data Analyst at the University of Colorado School of Medicine for her contributions to the data analysis. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This study was approved by the institutional review boards of University of California, Irvine and the University of Colorado as exempt. Correspondence should be addressed to Julie Youm, University of California, Irvine School of Medicine, 836 Health Sciences Road, Irvine, CA 92697; email: jyoum@uci.edu. © 2019 by the Association of American Medical Colleges

Learning Conversations: An Analysis of Their Theoretical Roots and Their Manifestations of Feedback and Debriefing in Medical Education Feedback and debriefing are experience-informed dialogues upon which experiential models of learning often depend. Efforts to understand each have largely been independent of each other, thus splitting them into potentially problematic factions. Given their shared purpose of improving future performance, the authors asked whether efforts to understand these dialogues are, for theoretical and pragmatic reasons, best advanced by keeping these concepts unique, or whether some unifying conceptual framework could better support educational contributions and advancements in medical education. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable. Correspondence should be addressed to Walter Tavares, Wilson Centre, 200 Elizabeth St., 1ES-565, Toronto, Ontario, Canada M5G 2C4; telephone: 416-340-3646; email: walter.tavares@utoronto.ca; Twitter: @WalterTava. © 2019 by the Association of American Medical Colleges