Flexible Endoscopy Curriculum, Fundamentals of Endoscopic Surgery, and Robotic Training

---

Topic 1: Flexible Endoscopy Curriculum / Fundamentals of Endoscopic Surgery Examination

Introduction

There is a considerable level of confusion among many Program Directors with respect to the American Board of Surgery Flexible Endoscopy Curriculum (FEC) and the SAGES Fundamentals of Endoscopic Surgery Program (FES).[1, 2]  The FEC was announced by the ABS in 2014, and fully implemented beginning with the 2018 graduating class.  It provides Program Directors with a stepwise curricular outline to help ensure that graduating general surgery residents possess the knowledge and skill to be a surgical endoscopist.  Within the FEC, a surgical endoscopist is defined as someone able to competently perform diagnostic upper and lower endoscopy, perform basic tissue sampling procedures, manage peri-procedural bleeding, and place percutaneous endoscopic gastrostomy (PEG) tubes.   There are 5 levels of the curriculum designed to be implemented sequentially within the overall residency curriculum.  Within those 5 levels, there are essentially 4 measureable requirements:

1. A concentrated rotation in GI endoscopy (required resource).  This rotation(s) can be embedded in other GI or general surgery rotations provided there is enough volume to satisfy requirement 3 below and be lead by an endoscopist who will be accountable for the quality of the experience.  This gives PD’s lots of flexibility to fit this within their curriculum. 
2. Achieving a Global Assessment of Gastrointestinal Endoscopic Skills (GAGES) score of 18/20 or higher for both upper and lower endoscopy. (FEC Level 4) Work on the validity evidence for GAGES has demonstrated that a score of 18 or greater was predictive of better endoscopic performance.   This score is also predictive of passing the FES skills exam.
3. Achieving the Residency Review Committee (RRC) Surgery required procedural volume for both upper and lower endoscopy.  (FEC Level 5)  At the time of this writing, the defined category requirement for endoscopy includes 35 EGDs/PEGs and 50 colonoscopies.  Of note however, the passing score for the FES skills exam was based on a mean endoscopy experience of 100 procedures, so meeting the bare minimums without additional training will result in increased risk of failure.
4. Passing the FES Exam. (FEC Level 5)  The FES exam has 2 parts. A multiple choice test designed to measure knowledge about the indications, setup, troubleshooting, and peri-procedural patient management of endoscopic patients, and a skills test focusing on the core tasks of GI endoscopy, navigation, loop reduction and management, working in retroflexion, mucosal inspection, and coordinated use of basic tools.  Pass/fail scores for both parts of the exam were set using best practices in test development methodology.  Scores on the skills exam are based on both time and performance factors, so there is no “maximum time” or “passing time” for any given skill.  Currently the skills are scored in compensatory fashion (i.e. excellent performance on one task can make up for poor performance on another).  Both the multiple choice and skills tests must be passed to fulfill the FEC requirement. 

Requirements 1 & 3 are fairly easy to achieve with simple rotation curricular planning.  Requirements 2 & 4 can be more challenging and will be addressed in more depth below.

Evidence/Data

GAGES:  The use of GAGES scores as a measure of endoscopic performance is well established in the literature.   GAGES scores are only designed to measure performance during the early portion of the learning curve, with scores of upper endoscopy plateauing at approximately 50 cases and colonoscopy around 75 cases.[3]  Thus using GAGES to discriminate performance of those with more extensive experience is unlikely to be successful.   The mean GAGES score is >= 18 for “experienced endoscopist” as defined by both RRC and American Society of Gastrointestinal Endoscopists (ASGE) minimum case volume criteria (>50 and >140 respectively).[3, 4] Lastly, additional work showed that a GAGES score >15 was highly predictive of passing the FES skills exam.[5]  Given this evidence, it is reasonable to assume that a resident who has achieved a high clinical volume of GI endoscopy, and has consistently achieved a GAGES score >15-18 as assessed by an experienced faculty endoscopist, will pass the FES skills exam without the need for simulation based training.  

FES multiple choice test: The FES program provides an e-Learning platform to help trainees prepare for the multiple-choice portion of the FES exam.  This content is accessible at https://www.fesdidactic.org/.  Additionally, Appendices A & B of the FEC outline modules available through the Surgical Council on Resident Education (SCORE) curriculum as well as other resources and text books to help trainees prepare for this test.  With adequate reading and preparation, pass rates for the multiple choice exam are very high. [6]

FES skills test:  The FES skills exam has proven to be more difficult, with first time failure rates for seemingly prepared test takers as high as 40%.[7, 8]  Four curricula using the best practice of simulation based mastery learning or proficiency based training, have been published and assessed for their ability to prepare trainees for the FES skills test.  These curricula could also be used as a pathway for remediation for those who fail the skills exam on the first attempt.   The published curricula are based on 3 available simulation platforms, the Surgical Training for Endoscopic Proficiency (STEP) which consists of several “homemade” task trainers and the Kyoto colonoscopy model[9], the Simbionix GI Mentor II, which is the platform used for FES skills testing, although the FES tasks are not available for direct practice, and the Endoscopy Training System (ETS) from Limbs and Things.[10, 11]   The curricula will be addressed below, organized by platform:

Surgical Training for Endoscopic Proficiency (STEP):  The STEP curriculum was intended to be a low cost alternative to help trainees prepare for the FES skills test.  It consists of one training task made from a shoe box and used to develop skill in navigation and retroflexion, a training task based on the children’s game “Operation” used to develop navigation and tool targeting, and the Kyoto trainer to address loop reduction and mucosal inspection.   The group from UT Southwestern has published several papers on the implementation of this curriculum with published first time pass rates in different cohorts ranging from 75% – 87%.[12, 13]  However, these results were only achieved by substituting some of the colonoscopy modules on the GI Mentor to for the Kyoto trainer as the subjects found the Kyoto trainer too difficult to use.  Given the wide range of reported first time pass rates, >20 trainees would need to complete this curriculum to prevent one failure (number needed to train). With the addition of the need for a GI Mentor simulator, this approach loses most of its “low cost” appeal as both the simulator and an actual endoscopy tower would be needed. (estimated cost >$100,000)

GI Mentor II:  Two published curricula have used primarily the Simbionix GI Mentor II as their simulation training platform.  The first is from Johns Hopkins, and uses Tasks from the GI Mentor software along with a few case modules that were chosen by a small group of experienced endoscopists.[14]  Subjects were instructed to train to expert derived goals on each as well as one table top model described in the paper.  While pre and post training scores or pass rates were not reported, all trainees passed the FES skills test on their first attempt.  The average training time for each trainee was about 4 hours.  The effect size of the training intervention could not be determined without pre and post test scores.  The cost of the curriculum includes the cost of the GI Mentor II simulator, the basic training and case modules, maintenance contract, and the cost of purchasing or creating the table-top model and having a real endoscopic tower available (>$100,000 estimate).

The second curriculum comes from Massachusetts General Hospital, and also uses a mix of GI Mentor tasks and case module, including bleeding modules.[15] Some of the modules selected are the same as the Hopkins group, but most are not.  The modules were chosen based on their ability to discriminate between experienced and novice endoscopists on prior analysis.   The pass rate of a group at MGH prior to implementation of the curriculum was 62%, and after the curriculum, the pass rate was 100%.  The mean and standard deviation of the post training group resulted in a z-score compared to the FES skills test passing score of 2.1.  This means that those completing this curriculum score on average 2.1 standard deviations above the minimum passing score.  Thus, the estimated failure rate of those completing this curriculum would be ~2%, and approximately 6 trainees would need to complete the curriculum to prevent one failure.  The cost of the curriculum includes the cost of the GI Mentor II simulator, the basic training and case modules, bleeding modules, and maintenance contract (>$100,000 estimate).

Endoscopy Training System:  The ETS was developed out of the simulation program at Walter Reed as a hybrid option to bridge the gap between “make it yourself” physical simulators and full VR simulators.  From a curriculum development standpoint, training goals for each task were determined by multidisciplinary expert performance.[16] The curriculum implementation study looking at training for FES was a collaboration between USU/Walter Reed and UT Southwestern.[17]  The pre-training pass rate of a group of PGY 1’s and 2’s was 18%.  After training, the pass rate was 100%. The mean and standard deviation of the post training group resulted in a z-score compared to the FES skills test passing score of 1.9.  This means that those completing this curriculum score on average 1.9 standard deviations above the minimum passing score.  Thus, the estimated failure rate of those completing this curriculum would be ~3%, and approximately 6 trainees would need to complete the curriculum to prevent one failure.  The cost of the curriculum includes the cost of the ETS simulator, and the cost of an endoscopy tower (>$25,000 estimate (simulator only~$9k)).

Anecdotes

Remediation: Very little has been written about remediation for FES failure, but given that first time failure rates for general surgery residents remain in the 10% – 20% range, remediation is clearly an issue many trainees and program directors must address.  At Walter Reed, we were asked to assist in the remediation of a chief resident in the spring of the PGY 5 year who had failed the FES skills test 3 times.  Given the time of the year, there was not a great deal of time to add meaningful clinical experience as graduation was quickly approaching.  This individual came to our institution for 5 days and completed our ETS based curriculum.  On the afternoon of the 5^th day, the FES skills test was administered.  Figure 1 shows the performance improvement demonstrated by this individual in a very short time period.  Measured skill went from about one standard deviation below the average for the experience cohort to a score that is above average for even a highly experienced cohort (>300 endoscopic procedures).[18] This type of improvement would be expected when implementing either the ETS curriculum or the GI Mentor curriculum designed by the MGH group as both achieve similar post training results.  

References

1. Surgery ABo (2014) Flexible Endoscopy Curriculum for General Surgery Residents.

2. Surgery ABo (2014) ABS Establishes New Requirement for Endoscopic Training and Assessment.

3. Vassiliou MC, Kaneva PA, Poulose BK, Dunkin BJ, Marks JM, Sadik R, Sroka G, Anvari M, Thaler K, Adrales GL, Hazey JW, Lightdale JR, Velanovich V, Swanstrom LL, Mellinger JD, Fried GM (2010) How should we establish the clinical case numbers required to achieve proficiency in flexible endoscopy? Am J Surg 199:121-125

4. Vassiliou MC, Kaneva PA, Poulose BK, Dunkin BJ, Marks JM, Sadik R, Sroka G, Anvari M, Thaler K, Adrales GL, Hazey JW, Lightdale JR, Velanovich V, Swanstrom LL, Mellinger JD, Fried GM (2010) Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 24:1834-1841

5. Mueller CL, Kaneva P, Fried GM, Feldman LS, Vassiliou MC (2014) Colonoscopy performance correlates with scores on the FES manual skills test. Surg Endosc 

6. Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR, Jr., Marks JM (2014) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc 28:631-638

7. Gardner A, Ujiki M, Dunkin BJ (2017) Passing the Fundamentals of Endoscopic Surgery (FES) Exam: Linking Specialty Choice and Attitudes About Endoscopic Surgery to Success. Surg Endosc 

8. Gardner AK, Scott DJ, Willis RE, Van Sickle K, Truitt MS, Uecker J, Brown KM, Marks JM, Dunkin BJ (2017) Is current surgery resident and GI fellow training adequate to pass FES? Surg Endosc 31:352-358

9. Wilcox V, Jr., Trus T, Salas N, Martinez J, Dunkin BJ (2014) A proficiency-based skills training curriculum for the SAGES surgical training for endoscopic proficiency (STEP) program. J Surg Educ 71:282-288

10. Ritter EM, Cox TC, Trinca KD, Pearl JP (2013) Simulated Colonoscopy Objective Performance Evaluation (SCOPE): a non-computer-based tool for assessment of endoscopic skills. Surg Endosc 27:4073-4080

11. Trinca KD, Cox TC, Pearl JP, Ritter EM (2014) Validity evidence for the Simulated Colonoscopy Objective Performance Evaluation scoring system. Am J Surg 207:218-225

12. Guzzetta AA, Gardner A, Willis RE, Wilcox V, Dunkin BJ, Hogg D, Scott L, Hennessy SA, Scott DJ (2017) Proficiency-Gased Preparation Significantly Improves FES Certification Performance. SAGES Annual Meeting, Houston, Texas

13. Weis JJ, Scott DJ, Busato L, Hennessy SA (2020) FES exam outcomes in year two of a proficiency-based endoscopic skills curriculum. Surg Endosc 34:961-966

14. Gearhart S, Marohn M, Ngamruengphong S, Adrales G, Owodunni O, Duncan K, Petrusa E, Lipsett P (2018) Development of a train-to-proficiency curriculum for the technical skills component of the fundamentals of endoscopic surgery exam. Surg Endosc 32:3070-3075

15. Hashimoto DA, Petrusa E, Phitayakorn R, Valle C, Casey B, Gee D (2018) A proficiency-based virtual reality endoscopy curriculum improves performance on the fundamentals of endoscopic surgery examination. Surg Endosc 32:1397-1404

16. Franklin BR, Placek SB, Gardner AK, Korndorffer JR, Jr., Wagner MD, Pearl JP, Ritter EM (2017) Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the Endoscopy Training System. Am J Surg 

17. Ritter EM, Taylor ZA, Wolf KR, Franklin BR, Placek S, Korndorffer JR, Gardner A (2017) Simulation Based Mastery Learning for Endoscopy Using the Endoscopy Training System: A Strategy to Improve Endoscopic Skills and Prepare for the Fundamentals of Endoscopic Surgery (FES) Manual Skills Exam. SAGES Annual Meeting, Houston, Texas

18. Lineberry M, Matthew Ritter E (2017) Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination. Surg Endosc 31:5219-5227

---

Topic 2: Robotic Surgery

Introduction

Robotic-assisted general surgery procedures continue to increase in frequency across the country [1]. However, there is currently no standardized robotic surgery training curriculum and competency evaluation for general surgery residents. In a recent survey of ACGME general surgery resident programs, 67% of the programs surveyed had a formal training curriculum, but only 55% offered formal certification for their residents [2]. However, programs vary in the requirements, credentialing and case tracking used for robotic surgery training [3]. Additionally, a recent national survey found that 60% of residents were provided no formal training before their first robotic case [4]. As a result, residency programs are left to independently design a robotic curriculum to prepare their general surgery residents for this advancing technology.

Evidence/Data

The Fundamentals of Robotic Surgery (FRS) is the only standardized robotic surgical skills education, training and assessment program available [5]. It is a multi-specialty proficiency-based curriculum of basic technical skills in robotic surgery. A validation study involving 15 international centers was approved in 2015, and the results of this trial are still pending. The FRS curriculum consists of four modules:

1. Introduction to surgical robotic systems
2. Didactic instructions for robotic surgery systems
3. Psychomotor skills curriculum 
4. Team training and communication skills

This curriculum is currently focused on training established surgeons, and its applicability to resident education has yet to be determined.

The majority of residency programs have not adopted the FRS curriculum, and instead have their own curriculum that has been tailored to fit the needs of their program. Formal robotic surgery curricula consist of online modules and hands-on skills labs to learn the basics of the technology. The technical skills are trained on a robotic simulator, either the daVinci skills simulator (dVSS) or Robotic Surgery Simulator (RoSS) [6]. 

The daVinci skills simulator (dVSS) is a backpack that sits on the daVinci surgeon console. The simulation technology from Mimic Technologies, Inc. (Seattle, WA) provides trainees with basic and advanced skills simulation in a 3D platform. These exercises focus on camera manipulation, energy, dexterity, range of motion, fourth arm control, and suturing. Scores are tracked and proficiency reports are provided for trainees and trainers to track progress and receive feedback on specific areas for improvement. The challenges of the dVSS are primarily related to access. The simulator backpack must be connected to a robotic console, which is typically in the operating room. Therefore, there are limitations for trainees to access the simulator during the day and when the robotic console is being used. Alternatively, institutions can purchase an additional console solely for robotic simulation.

Robotic Surgery Simulator (RoSS) is an alternative to the dVSS that has been available since 2009. The RoSS is a portable, stand-alone virtual reality simulator that teaches the technical skills for operating on the daVinci surgical robot. Techniques focus on orientation, motor, basic and intermediate surgical skills. A Robotic Skills Assessment (RSA) score is generated to assist them in evaluating their performance. One of the main advantages of the RoSS is that simulator is portable, and therefore able to be taken to a location convenient for the trainee. Additionally, since it is not used for operating, there is no interference with operative cases. However, it is an additional purchase required by the training program.

Anecdotes

The primary resource for robotic surgery training curriculum design within a residency program is gained from other institutions. Presently, the majority of institutions utilize the daVinci pathway for resident and fellow training as a guide, and fit this into the context of their own program [7]. This curriculum is divided into four modules:

1. Introduction to da Vinci
2. daVinci skills drills training
3. Intra-operative and advanced skills application
4. DaVinci case participation

At the completion of the modules, residents and fellows have the opportunity to obtain a training equivalency certificate.

At the Medical College of Wisconsin, the robotic surgery training curriculum, implemented in 2017, has the following outline:

Overview

The MCW Robotic Surgery Curriculum will be completed in four phases, which will be distributed by PGY level

1. Phase 1 – Introduction to robotic surgery technology (PGY-1 residents)
   1. Online certificate completion for residents and fellows
   2. Review of two full-length procedure videos
   3. System overview and in-service training
   4. Completion of basic skills simulation exercises
2. Phase 2 – Robotic surgery advanced skills training (PGY-2 residents)
   1. Two case observations
   2. Technical skills practicum
   3. Completion of advanced skills simulation exercises
3. Phase 3 – Robotic surgery advanced skills application (PGY 1-5 residents)
   1. Bedside assistant cases
4. Phase 4 – Robotic surgery case participation (PGY 3-5 residents)
   1. Console surgeon cases

References

1. Wormer BA, et al. The first nationwide evaluation of robotic general surgery: a regionalized, small but safe start. Surgical Endoscopy 2014;28:767-776
2. Tom CM, et al. A survey of robotic surgery training curricula in general surgery residency programs: How close are we to a standardized curriculum? American Journal of Surgery 2019;217(2):256-260
3. Winder JS, et al. Implementing a robotics curriculum at an academic general surgery training program: our initial experience. Journal of Robotic Surgery 2016;10:209-213
4. Farivar BS, Flannagan M, Leitman M. General surgery residents’ perception of robotic-assisted procedures during surgical training. Journal of Surgical Education 2015;72(2):235-242
5. Fundamentals of Robotic Surgery. http://frsurgery.org/. Date of Access: Jan 13 2020
6. Bric JD, Lumbard DC, Frelich MJ, Gould JC. Current state of virtual reality simulation in robotic training: a review. Surgical Endoscopy 2016;30(6):2169-2178
7. DaVinci Residency and Fellowship Training Program Implementation Guide. https://www.intuitive.com/en-us/-/media/Project/Intuitive-surgical/files/pdf/davinci-resident-fellows-training-guide-1024650.pdf?la=en&hash=A755B62C01C010BA6C1A4F00EF7F2734