Abstract
Background: In Canada, endoscopy is primarily performed by gastroenterologists and surgeons, and some studies report that colonoscopies performed by nongastroenterologists have more complications and higher rates of future colorectal cancer. Our objective was to determine whether rural-based nongastroenterologist endoscopists are achieving quality benchmarks in colonoscopy.
Methods: This quality improvement initiative prospectively evaluated 6 key performance indicators (KPIs) (cecal intubations, polyp detection [males and females; for first-time colonoscopies on patients aged ≥ 50 yr], bowel preparations, patient comfort and withdrawal times) on consecutive colonoscopies performed by participating Alberta North Zone endoscopists. The study period was June 2018 to March 2020. Overall and individual endoscopist’s KPIs were compared with standard benchmarks. Additional performance indicators included mean number of polyps per colonoscopy and an exploration of study-defined sedation-related level of consciousness.
Results: Data were collected on 6212 colonoscopies performed by 16 endoscopists (9 surgeons, 5 family physicians and 2 internists) in 6 hospitals. All 6 KPI benchmarks were achieved when results were pooled over all endoscopists in the study. Overall, cecal intubation occurred in 6006 of 6209 (96.7%, 95% confidence interval 94.5%–99.0%) cases. Polyp detection was 65.9% (592/898) and 49.8% (348/699) for male and female patients, respectively, aged 50 years or older. Variability in individual endoscopist results existed, especially for the mean number of polyps per 100 colonoscopies and sedation-related level of consciousness.
Interpretation: Overall, Alberta North Zone endoscopists are performing high-quality colonoscopies, collectively achieving all 6 KPIs. To understand endoscopic performance and encourage individual and group reflection on endoscopic practices, Canadian endoscopists are encouraged to participate in similar colonoscopy quality initiative studies.
In Canada, more than 1 million endoscopic procedures are performed annually,1 primarily by gastroenterologists and surgeons,1,2 but also by some internists and family physicians.1,3,4 Nongastroenterologists perform most of the endoscopic evaluations in rural and smaller urban Canadian communities.1,2,5
Variability exists between individual endoscopists’ colonoscopy performance, which may affect future rates of colorectal cancer6,7 or adverse events.8 Some Canadian studies report that polyp detection may be lower9 and future cancer rates may be higher when colonoscopies are performed by nongastroenterologists.10–12 Other studies show that nongastroenterologists perform colonoscopies that exceed quality standards.3,4
We performed the Alberta North Zone Endoscopy Quality study to determine whether a diverse group of rural-based nongastroenterologist endoscopists within a large health region are achieving key performance indicator (KPI) benchmarks in colonoscopy, including cecal intubation, polyp detection, bowel preparation, patient comfort and withdrawal times.
Methods
This quality improvement initiative was a prospective cohort study exploring KPIs on colonoscopies performed by voluntarily participating Alberta North Zone colonoscopists. Data were collected on consecutive colonoscopies performed from study commencement (June 2018 — with asynchronous participant and community starts) until March 2020, when non-urgent endoscopies were temporarily halted owing to the COVID-19 pandemic. Interim results have been presented previously,13 and our study is reported in accordance with an adapted Standards for Quality Improvement Reporting Excellence checklist for quality initiatives.14
Setting
Alberta is a Canadian province of 4.4 million people,15 divided into 5 health zones. The North Zone is the largest Alberta health region geographically,16 roughly the size of Sweden and with a population of 484 000 people.15 Through 13 hospital-based endoscopy units, 25 endoscopists perform more than 9000 colonoscopies annually (Dr. Kelly Burak, Physician Learning Program, University of Calgary: unpublished data, 2019). Eleven of the hospitals serve populations of about 10 000 or less, and Grande Prairie and Fort McMurray have populations greater than 60 000.15 All of the North Zone communities are considered rural. All participating hospitals, except 1, are affiliated with Alberta medical schools.
All North Zone endoscopists who were actively performing colonoscopies were invited to voluntarily participate in the study. We enrolled interested endoscopists and their communities after a study onboarding meeting and local operational approval from their hospital administration.
Data sources
All participating sites received data entry training with endoscopists and their staff before commencing the study. Study personnel provided local in-house support on the first day of data collection. Data were entered in real time during the colonoscopy by the endoscopy room nurses, in collaboration with the endoscopist, into a cloud-based REDCap database hosted by the Women and Children’s Health Research Institute at the University of Alberta.17
Data were collected and synthesized through a program developed by the study team. Missing or potentially out-of-range results were flagged for the participating endoscopist to review. Individual endoscopist, community and overall report cards with benchmarks and peer group comparisons were autogenerated. Endoscopists were provided with live, quarterly and annual reports and were encouraged to actively reflect on their results. Annual feedback sessions occurred (at local sites or via webinars), where community and overall results were presented to the participating endoscopists and their teams.
Outcomes
Primary KPIs
Key performance indicators and corresponding quality benchmarks were derived from existing literature18–21 and current Global Rating Scale–Canada guidelines,22 and chosen by the study co–principal investigators (M.R.K. and D.C.M.M.). When there were different measurements of the same outcome (e.g., bowel preparations), for efficiency and ease of reporting, we chose a simplified definition of the outcome. If differing benchmarks for the same outcome existed, we used the most stringent target for comparison. For example, one society recommends that inadequate bowel preparations should occur in 15% or fewer of procedures,18 whereas others recommend inadequate bowel preparations should occur in 10% or fewer of colonoscopies.19–21 For our study, we used 10% or fewer as the benchmark target (Appendices 1 and 2, available at www.cmajopen.ca/content/11/4/E654/suppl/DC1). The 6 KPIs for the study were cecal intubation, polyp detection (male and female), bowel preparation, patient comfort and withdrawal times (when no lesions were detected).
Proportion of successful cecal intubations
The proportion of successful cecal intubations was defined as the number of procedures in which landmark-confirmed cecal intubation occurred, divided by the number of colonoscopies attempted. No adjustments were made for poor bowel preparations or other potential causes of incomplete colonoscopies. Typically, cecal intubation benchmarks are 95% for screening colonoscopies and 90% for colonoscopies performed for symptom investigation.18 As average-risk screening colonoscopies are rarely performed in Alberta,4 and participating endoscopists performed both diagnostic and screening colonoscopies, 90% was the benchmark for comparison.20,21
Proportion with at least 1 polyp at first-time colonoscopy (polyp detection rate)
The proportion of male and female patients aged 50 years and older undergoing a colonoscopy for the first time with at least 1 detected polyp was calculated. Polyp detection rates can be extrapolated to estimate adenoma detection rates.20 A guideline from the European Society of Gastrointestinal Endoscopy recommends that to achieve an adenoma detection rate of 25%, one should have a polyp detection rate of 40% or greater.20 As current benchmarks for adenoma detection rates are 30% in males and 20% in females,18 study benchmarks for polyp detection rates were 45% and 35%, respectively. Historically, the benchmark for adenoma detection rates was initially derived from studies of average-risk screening colonoscopies. 18 However, as average-risk screening colonoscopies are infrequently performed in Alberta,4 we determined polyp detection rates from all colonoscopies in the study.
Bowel preparation, patient comfort and withdrawal times
Bowel preparation results were recorded as excellent, adequate or inadequate, and our benchmark was that 10% or fewer of patients should have inadequate preparations. For patient comfort we used the Modified Gloucester Scale19 and equated “moderate” or “severe” discomfort from the Modified Gloucester Scale to equal a Nurse-Assessed Patient Comfort Score (NAPCOMS) of 6 or greater.23 As guidelines recommend that fewer than 10% of patients should have a NAPCOMS of 6 or greater,24 we set the benchmark at less than 10% having moderate or severe discomfort. Finally, for withdrawal times (when no lesions were detected), we used 6 minutes as our standard benchmark.25
Mean number of polyps per 100 colonoscopies
We defined the number of polyps per 100 colonoscopies as the sum of all polyps identified, divided by the number of colonoscopies performed, multiplied by 100. Although this is likely one of the most important colonoscopy-related KPIs, no benchmark currently exists. We used all colonoscopies performed in the study as the denominator.
Patient level of consciousness
Sedation-related cardiopulmonary events is a known potential complication of endoscopic procedures.18,26 Although the optimal sedation level for a colonoscopy is not defined and no benchmark target exists, endoscopy leaders are promoting less-sedated endoscopy.27 Using this principle of minimally sedated endoscopy, the study leads (M.R.K. and D.C.M.M.) proposed a study-defined conservative benchmark where fewer than 33% of patients should be unresponsive or only respond when stimulated during the procedure.
We also collected data on procedural indications and findings: who performed sedation (endoscopist or anesthesiologist) and sedation agents used and procedure times. For definitions of study performance indicators and benchmarks, see Appendices 1 and 2.
Statistical analysis
Outcomes were reported both in the aggregate and by anonymized endoscopist. Binary outcomes were reported as percentages with confidence intervals (CIs) for overall and the most important individual KPIs. Continuous variables were reported as means with ranges or standard deviations, where appropriate.
Given that variability in endoscopist outcomes is likely due to both patient-related factors (age, sex, indications, first time or surveillance colonoscopy) and endoscopist performance characteristics,28 we performed a cluster-level analysis to help explore this influence further. We considered each endoscopist a “cluster” and, for each main outcome, estimated an intraclass correlation coefficient using methods previously described.29 From each intraclass correlation coefficient, a design effect was computed and then used to compute the effective sample size and adjusted CI.
Ethics approval
The University of Alberta Health Research Ethics Board approved the study. All endoscopists consented to participate in the study. Patient consent was waived as patient treatment did not change; we simply collected data pertaining to the procedure.
Results
The study commenced June 2018, with 9 endoscopists from 4 hospitals, and increased to a total of 16 endoscopists (9 surgeons, 5 family physicians and 2 internists) from 6 hospitals (in Grande Prairie, High Level, Hinton, McLennan, Peace River and Whitecourt) at the end of the study. For each participating community, all endoscopists eventually participated in the study.
The 16 endoscopists varied in their endoscopy experience, with about half having performed endoscopy for more than 10 years (Table 1). Participating sites ranged from a solo family physician endoscopist site to a larger endoscopy unit with 7 general surgeons and 2 internists. Two sites were satellite endoscopy units, serviced by visiting endoscopists already enrolled at their primary site.
Characteristics of study endoscopists
Procedures were performed using Olympus 180 or 190 series or Pentax 90i series, without routine use of a scope guide. Most units had a pediatric colonoscope, half had carbon dioxide for insufflation and all aimed for 2 nurses in their endoscopy suites. Individual endoscopists participated in the study for an average of 16 (range 2–21) months and performed a mean of 388.3 (range 40–937) colonoscopies in the study (Appendix 3, available at www.cmajopen.ca/content/11/4/E654/suppl/DC1).
Patients
Demographic characteristics were recorded before endoscopy for 7613 patients. After excluding no-shows and cancellations (n = 548 [7.2%]) and procedures for which staffing shortage or Wi-Fi issues prohibited data collection, we had 6212 procedures with sufficient data for analysis (Figure 1). The mean age of patients was 56.9 years, 3071 (49.4%) were female and 2568 (41.3%) underwent their first colonoscopy in the study (Table 2). Overall, 2345 (37.7%) of the colonoscopies were performed for the investigation of symptoms and 1757 (28.3%) for colorectal cancer screening. The 3 most common specific indications were polyp surveillance (1154, 18.6%), positive fecal immunochemical test (FIT) (1058, 17.0%) and rectal bleeding (961, 15.5%). Sedation was performed by the endoscopist in 4236 (68.2%) cases, and 31.8% (predominantly from 3 sites) had anesthesiologists provide sedation. Midazolam, fentanyl and propofol were used in 6079 (97.9%), 4855 (78.2%) and 1952 (31.4%) cases, respectively, and in 54 cases (< 1%), no sedation was used.
Record flow.
Characteristics of study patients
Key performance indicators: overall
All 6 KPI benchmarks were achieved when results were pooled over all endoscopists in the study.
Cecal intubation
Cecal intubation was confirmed in 6006 of 6209 (96.7%, 95% confidence interval [CI] 94.5%–99.0%) colonoscopies (Table 3). The terminal ileum was intubated in 1626 (27.1%) of the cases.
Individual and overall results — 6 key performance indicators
In cases where the terminal ileum was not intubated or the cecum was not altered by previous surgery, at least 2 of the cecal landmarks (trifolds, appendiceal orifice or ileocecal valve) were identified in 4240 of 4287 (98.9%) cases. Poor bowel preparation was the most common reason (33.0%) for an incomplete colonoscopy.
Polyp detection
Males
A total of 898 males aged 50 years and older had their first colonoscopy in the study, and 592 (65.9%, 95% CI 56.4%–75.4%) had at least 1 polyp (Table 3).
Females
A total of 699 females aged 50 years and older had their first colonoscopy in the study, and 348 (49.8%, 95% CI 42.5%–57.1%) had at least 1 polyp (Table 3).
Bowel preparation, patient comfort and withdrawal times
Inadequate bowel preparations occurred in 4.6% (288/6209) of cases, and patient discomfort was moderate or severe in 5.8% (357/6208) of cases. Withdrawal time (when no lesions were detected) averaged 7.3 minutes, and procedure time averaged 19.7 minutes (Table 3).
Sedation level of consciousness and number of polyps detected
For sedation level of consciousness, 54.9% (3405/6206) of patients were unresponsive or only responded when stimulated (Table 4). In approximately 1% of cases, no sedation was used. There were 7542 polyps detected in the 6212 colonoscopies for an average of 121.4 polyps per 100 colonoscopies (Table 4).
Key performance indicator benchmarks achieved by endoscopists
The most common finding was a polyp or polyps that appeared adenomatous (37.1%, 2299/6212). Colorectal cancer was reported in 87 (1.4%) procedures.
Data integrity
From a total of 93 180 data points (15 data fields per procedure × 6212 procedures), there were 136 incomplete or out-of-range values for completion and accuracy rate of 99.9%.
Colonoscopies for positive FITs
A total of 1058 colonoscopies were performed for a FIT-positive patient (Appendix 4, available at www.cmajopen.ca/content/11/4/E654/suppl/DC1). Patients with a positive FIT result were older (mean age 61.8 yr), more often male (62.8%) and more likely to be having their first colonoscopy in the study (66.4%). Among males with a positive FIT result, 498 of 664 had a polyp (75.0%), and among females with a positive FIT result, 232 of 394 had a polyp (58.9%). Compared with the entire cohort, FIT-positive patients had a normal colonoscopy approximately 40% less often (22.3% v. 36.8%) and had an adenomatous-appearing polyp or colorectal cancer approximately 50% more often (56.5% v. 37.1% and 2.1% v. 1.4%, respectively).
Benchmarks achieved by endoscopists
Overall, the 16 endoscopists achieved an average of 5.4 of the 6 key benchmarks (cecal intubation, bowel preparation, polyp detection [males and females], patient comfort and withdrawal times) (Table 4). Eleven endoscopists achieved all 6 benchmarks, and 4 achieved the 6 benchmarks plus the study-defined level-of-consciousness benchmark.
All individual endoscopists achieved the bowel preparation benchmark. Fourteen endoscopists achieved cecal intubation rates of 90% or greater (Table 4). The 2 endoscopists who did not achieve the cecal intubation benchmark performed fewer than 70 procedures in the study and had 95% CIs that included the 90% benchmark. For polyp detection rates, 13 endoscopists achieved rates of 45% in males and 35% in females.
Variability in endoscopist results
There was an approximate 20-fold range (from 5.4% to 99.2%) between endoscopists in the proportion of patients being unresponsive or responding only with stimuli (Table 4). A fivefold difference also existed between endoscopists’ mean number of polyps per 100 colonoscopies (from 42.7 to 218.1). As each endoscopist’s patient cohort differed by age, sex, indications and first-time colonoscopy, these findings are likely a result of both endoscopist performance and patient-related factors.
To attempt to explain how much of the variability might be due to the individual endoscopist, we performed a cluster-level analysis, comparing each endoscopist to the collective group of endoscopists (Appendix 5, available at www.cmajopen.ca/content/11/4/E654/suppl/DC1). Our estimated intraclass coefficients were low for all outcomes, which leads us to conclude that most of the outcome variability was due to patient-related factors, rather than the endoscopist.
Interpretation
In this prospective study of the quality of colonoscopies performed by a diverse group of Alberta endoscopists (surgeons, family physicians and internists), we found that all 6 KPI benchmarks were achieved when results were pooled over all endoscopists in the study. Individual endoscopists achieved, on average, 5.4 of the 6 KPIs (cecal intubation, bowel preparation, polyp detection [male and female], patient comfort and withdrawal times), with 11 endoscopists achieving all 6 key benchmarks. Our findings are similar to those of other studies of colonoscopy quality, in which institutional or jurisdictional KPIs were achieved, but significant variability existed between endoscopists.30–36 For example, whereas overall cecal intubation rates were greater than 96%, 2 participants did not achieve the benchmark of 90%. Both endoscopists performed fewer than 70 procedures in the study and had 95% CIs that included the 90% benchmark. Conclusions pertaining to performance should be made with caution when a small number of procedures are analyzed.
The mean number of polyps per colonoscopy is likely one of the most meaningful outcomes in colonoscopy, incorporating patient (age, gender, procedure indications, first time or surveillance), system (surveillance intervals, bowel preparations) and endoscopist performance variables. We found a fivefold difference between the lowest and highest polyp-detecting endoscopists. This variability is likely explained by the differences in patient demographic characteristics, procedure indications and surveillance intervals, and other quality metrics (e.g., bowel preparation), but also may reflect endoscopist performance.28 Owing to the nature of the study design (evaluating consecutive colonoscopies performed on patients for a variety of reasons), each endoscopist’s patient cohort was unique. Our low intraclass correlation coefficient results would lead one to conclude that most of the variability in findings was due to patient-related factors. However, until we have precise modelling that enables an accurate prediction of mean number of polyps per scope or polyp detection rates based on patient factors (age, gender, indication, first or subsequent colonoscopy, and family history of colorectal cancer) as well as procedure (bowel preparation), we can surmise that variability in outcomes (e.g., mean adenomas per colonoscopy) is due to both patient- and endoscopist-related factors.
The only benchmark not obtained collectively by the group was the study-defined patient level of consciousness. Whereas 7 endoscopists achieved this benchmark, overall, approximately 55% of patients were relatively unresponsive during their procedure. Endoscopy level of sedation is likely dependent on the experience of the endoscopist or anesthetist, as well as patient expectations.31 Current recommendations are to perform colonoscopies with both minimal sedation and patient discomfort.27 We conservatively decided that fewer than 33% of patients should be unresponsive or only responsive when stimulated, and found that collectively the endoscopists did not achieve this recommendation. We encourage future studies to further explore this outcome to help determine whether teams should evaluate their current sedation practices to attempt to achieve our recommendation or whether a more appropriate benchmark should be defined.
A systematic review on the quality of colonoscopies performed in rural communities included 11 studies evaluating the quality of 8703 colonoscopies.37 Most included studies involved a single endoscopist or single centre, had small numbers of procedures analyzed and involved retrospective data collection. Our study prospectively collected data from 6 communities, on more than twice the number of colonoscopies than the largest included study, and included surgeons, family physicians and internists. A recent study evaluated the outcomes of 1865 screening colonoscopies performed by surgeons, gastroenterologists and family physicians in 8 hospitals in rural Iowa.38 Collectively, the endoscopists achieved colonoscopy quality benchmarks, but outcomes varied by practitioner (including a more than fivefold range in adenoma detection rates). Our study adds to this body of research in analyzing many colonoscopies performed in several communities for a variety of indications.
Our study also highlights findings from 1058 FIT-positive colonoscopies. Compared with the other indications in our study, we found that patients with positive FIT results were 50% more likely to have either an adenomatous-appearing polyp or colorectal cancer. Our results will aid discussions between patients and their family physicians as well as endoscopists and endoscopy programs when triaging patients with positive FIT results.
We demonstrated that busy endoscopy units were able to incorporate an endoscopy quality initiative into existing workflow. All Canadian endoscopy programs are similarly encouraged to participate in the Global Rating Scale,22 a quality-improvement program that emphasizes the patient experience and clinical outcomes, including the measurement of KPIs in colonoscopy.22 Using similar data collection tools, all units should be measuring and receiving reports pertaining to their personal and programmatic KPIs. These results can be used as a starting point for any individual- or unit-based quality-improvement initiatives and will also help further refine benchmarks in colonoscopy KPIs. Future similar studies may provide insight into the value of participating in an endoscopy quality study in which personal feedback is provided and individual or systematic improvements are subsequently implemented.39
Limitations
Pathological verification remains the gold standard for determining the incidence of adenomas and cancers. However, because of the substantial additional resources required, we did not require pathological verification of lesions found at endoscopy. As polyp detection correlates to adenoma detection, 20,40–42 we used published adenoma-to-polyp quotients to determine our benchmarks for polyp detection rates. In future studies, we could consider determining endoscopist-specific adenoma-to-polyp quotient before starting the study. In addition, we did not adjudicate and verify potential adverse events, and therefore, potential complications were not reported. Some of our outcomes (e.g., bowel preparation, patient comfort or level of consciousness), while specifically defined, are relatively subjective. Variability in reporting these outcomes could exist between different endoscopy teams. It is uncertain whether having the endoscopy team nurse (in collaboration with the participating endoscopists) inputting the data eliminated all potential reporting bias. The Hawthorne effect (in which behaviour changes when one is being observed) may have inflated performance, as all participants were aware of the study. Although we attempted to explain the variability in outcomes by comparing each endoscopist with the larger group, the exact contribution of patient-related or endoscopist-related factors to outcomes is unknown. Finally, this study relied on the voluntary participation of endoscopists and their teams, and had 16 of the 25 North Zone endoscopists participate. We do not know what reasons precluded the other 9 North Zone endoscopists from participating in the study and whether these endoscopists and sites would have had similar results to those found in the study.
Conclusion
We found that participating Alberta North Zone endoscopists (surgeons, family physicians and general internists) collectively achieved 6 colonoscopy KPIs. Similar to other studies of endoscopy quality, we found variability in results of individual endoscopists, likely due to patient, system and endoscopist factors. Finally, we demonstrated the ability of endoscopy units (from rural single endoscopist units to busy regional referral centres) to implement an endoscopy quality study without substantially affecting patient care or unit efficiencies. To better understand individual and group endoscopy performance, we encourage all Canadian endoscopists to participate in a similar colonoscopy quality study.
Acknowledgements
The authors thank the participating endoscopy teams (Grande Prairie, High Level, Hinton, McLennan, Peace River and Whitecourt) and their local administrations for their support of the study. The authors also thank Kyle Androschuk for his help in programming and report card generation and Ben Vandemeer for statistical support. Finally, the authors acknowledge Dr. Allison Hobbs and Nicole Olivier, who were actively involved in the study, but owing to other commitments, were unable to fulfill authorship requirements.
Footnotes
Competing interests: Michael Kolber founded EMPRSS, a University of Alberta spin-off company that supported the data collection, synthesis and reporting of results to the participants and their teams. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Dereck Mok and Michael Kolber contributed to study conception. Peter Miles, Marcus Shaw and Hilgard Goosen provided early study feedback, local/community or regional study promotion and/or data acquisition or interpretation. All authors contributed to drafting and reviewing the manuscript. All authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: The study was funded by the Alberta Health Services Quality Innovation Fund and supported by the Alberta Rural Health Professionals Action Plan, the University of Alberta Department of Family Medicine and indirectly through Enhancing Alberta Primary Care Research Networks.
Data sharing: Collective (overall) results could be made available to others. Individual endoscopist results remain anonymized, as per agreement between the study team and participating endoscopists.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/11/4/E654/suppl/DC1.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
- © 2023 CMA Impact Inc. or its licensors
In this issue
Article tools
