Abstract
Background: Most children who need emergency care visit general emergency departments and urgent care centres; the weighted pediatric readiness score (WPRS) is currently used to evaluate emergency departments’ readiness for pediatric patients. The aim of this study was to determine whether a higher WPRS was associated with decreased mortality and improved health care outcomes and utilization.
Methods: We conducted a systematic review of cohort and cross-sectional studies on emergency departments that care for children (age ≤ 21 yr). We searched MEDLINE (Ovid), Embase (Ovid), the Cochrane Library (Wiley), CINAHL (EBSCO), Global Health (Ovid) and Scopus from inception until July 29, 2022. Articles identified were screened for inclusion by 2 independent reviewers. The primary outcome was mortality, and the secondary outcomes were health care outcomes and utilization. We used the Newcastle–Ottawa Scale to assess for quality and bias of the included studies. The I2 statistic was calculated to quantify study heterogeneity.
Results: We identified 1789 articles. Eight articles were included in the final analysis. Three studies showed an inverse association between highest WPRS quartile and pediatric mortality (pooled odds ratio [OR] 0.45, 95% confidence interval [CI] 0.26 to 0.78; I2 = 89%, low certainty of evidence) in random-effects meta-analysis. Likewise, 1 study not included in the meta-analysis also reported an inverse association with a 1-point increase in WPRS (OR 0.93, 95% CI 0.88 to 0.98). One study reported that the highest WPRS quartile was associated with shorter length of stay in hospital (β −0.36 days, 95% CI −0.61 to −0.10). Three studies concluded that the highest WPRS quartile was associated with fewer interfacility transfers. The certainty of evidence is low for mortality and moderate for the studied health care outcomes and utilization.
Interpretation: The data suggest a potential inverse association between the WPRS of emergency departments and mortality risk in children. More studies are needed to refute or confirm these findings.
Protocol registration: PROSPERO-CRD42020191149.
Children account for approximately 20% of the total emergency department visits in Canada1 and the United States2 each year. A portion of these visits are to specialized pediatric emergency departments; however, the majority of the visits are to general emergency departments or urgent care centres, which have a wide range of experience with pediatric patient care.3 In the US, approximately 90% of pediatric visits are to general emergency departments,4 and in Canada, approximately 85% of pediatric visits are to general emergency departments.5 Therefore, it is important to ensure that all emergency departments are optimizing pediatric patient outcomes and safety independent of their pediatric patient volumes, location or presentations.
There is a growing literature evaluating emergency department readiness to provide optimal medical care to acutely ill and injured pediatric patients. The weighted pediatric readiness score (WPRS) was developed as part of the National Pediatric Readiness Project6 to assess the level of readiness of emergency departments to care for pediatric patients. The 100-point scale includes weighted items in the categories of pediatric-specific infrastructure, administration and coordination, personnel, pediatric-specific policies, equipment and resources.7 The goal of the WPRS is to identify areas of improvement for emergency departments to maximize readiness to care for pediatric patients.3,6–8 The National Pediatric Readiness Project assessment of emergency departments across the US in 2013 identified that the median WPRS was 68.9, suggesting that many emergency departments are missing key components of pediatric readiness.7
Recent literature also suggests that higher pediatric readiness scores are associated with better pediatric patient outcomes, including a recent study by Ames and colleagues, which found that high pediatric readiness scores are associated with a fourfold decreased risk of mortality.9 Mortality is a key measure of quality of emergency department care, as well as an important measure of outcomes of critically ill children presenting to the emergency department. Therefore, the primary objective of this study was to conduct a systematic review and meta-analysis to determine whether a higher pediatric readiness score results in a decreased mortality rate for children presenting to emergency departments and urgent care centres. The secondary objective was to determine whether higher pediatric readiness scores result in improved health care outcomes and health care utilization.
Methods
The systematic review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 statement,10 and the protocol was registered with International Prospective Register of Systematic Reviews (PROSPERO) in June 2020 (registration no. CRD42020191149). The primary objective of the study was to determine whether higher WPRS is associated with lower rates of mortality in children presenting to emergency departments and urgent care centres, by synthesizing current research and conducting meta-analysis where possible. The secondary objective of this review was to determine whether there is a difference in health care utilization between centres with high and low pediatric readiness scores.
Study selection and search strategy
We included studies if they met all the population, intervention, comparison, outcomes and study (PICOS) design criteria: population — conducted in an acute care facility that cares for children, including emergency departments or urgent care centres; intervention — used the pediatric readiness score or WPRS; comparator — compared low versus high or use versus nonuse of WPRS; outcome — the primary outcome was mortality, and secondary outcomes were health care outcomes (length of stay in an emergency department, revisits or readmissions to the emergency department), health care utilization (proximity to an emergency department, e.g., access to an emergency department within a 30-min drive); study — observational studies, including cohort and cross-sectional studies, or controlled-clinical study. We included studies published in the English language.
Studies were excluded if the outcome was not relevant, if the study did not include pediatric patients (defined as age ≤ 21 yr)11 or if the setting was outside of an acute care facility.
An experienced health sciences librarian (M.-L.L.) designed and executed the search strategy, using a combination of subject terms and keywords that were later translated for each database. A slightly modified version of a validated filter was used to focus the search on a pediatric population.12 Searches were performed in MEDLINE (Ovid), Embase (Ovid), the Cochrane Library (Wiley), CINAHL (EBSCO), Global Health (Ovid), and Scopus from inception until May 25, 2020, and then an updated search was performed on June 16, 2021, and July 29, 2022. The MEDLINE search was peer reviewed by an independent health sciences librarian as per the Peer Review of Electronic Search Strategies guidelines. 13 Our search strategy is available in Appendix 1, available at www.cmajopen.ca/content/11/5/E956/suppl/DC1. Identified studies were deduplicated in EndNote (version X9).
The articles identified in the literature search were first screened by title and abstracts for inclusion in the systematic review by independent reviewers (J.A.H., A.C., C.T., M.S., M.R. and L.W.). The independent reviewers (J.A.H., A.C., C.T., M.S., M.R. and L.W.) then reviewed the full text for inclusion in the final analysis. Disagreements during screening were resolved by discussion between reviewers or in consultation with a third reviewer (A.A.).
Data extraction
The data from the included studies were extracted by independent reviewers (J.A.H., C.T. and L.W.). Reviewers used a customized data extraction tool to identify key characteristics of the articles, including information on study design, objectives, population, intervention, outcomes and conclusion details. The tool was used to pilot-test 5 studies, after which it was adopted for the entire included studies. Two reviewers independently completed the screening and data extraction in duplicate. A third reviewer (A.A.) examined the data to ensure accuracy and identify any errors. We contacted Balmaks and colleagues for more information to rescale their data on mortality.
Risk-of-bias assessment
Independent reviewers (J.A.H., C.T. and L.W.) assessed the included articles for quality and bias using the Newcastle–Ottawa Scale (NOS),14 a validated critical appraisal checklists for nonrandomized observational studies. A modified version of the NOS for cohort studies was used to assess the cross-sectional studies. We replaced the word “cohort” with “sample” in the selection domain. We omitted questions 2 and 3 regarding follow-up and added a question assessing the statistical tests performed in the outcome domain. The NOS rates articles on a star system to evaluate the selection of study groups, comparability of groups, and ascertainment of exposure or outcome of interest.14 Two reviewers independently completed the risk-of-bias assessment, and disagreements were resolved by a third reviewer.
GRADE assessment
We used the GRADEPro software15 to assess the certainty of evidence for the included studies.
Data analysis and synthesis
Data were collected and managed using Excel and Covidence. Individual article characteristics were summarized and presented in tabular form. We used Review Manager 5.4. to perform the statistical analysis to generate the forest plot that showed the point estimates with 95% confidence intervals (CIs) of the studies included in the meta-analysis.
The association between WPRS and mortality was examined in random-effects models, estimated using the DerSimonian and Laird method,16 with the fixed-effects model included for sensitivity analysis. The pooled estimate of odds ratios (ORs) with 95% CIs was computed and demonstrated graphically with a diamond in the forest plot. The I2 statistic was calculated to quantify study heterogeneity. We performed a sensitivity analysis to adjust for the risk of possible overlapping studies on the pooled estimate. We narratively synthesized the secondary outcomes, according to the synthesis without meta-analysis (SWiM) guidelines,17 where the measures of effects were different.
Ethics approval
We did not require ethics approval for this systematic review, because publicly available data were used.
Results
The search and study screening were conducted initially in May 2020, with an update performed in June 2021 and July 2022 (Figure 1). The initial systematic search of the databases identified 1263 articles, and the updated searches identified 186 articles in June 2021 and 340 articles in July 2022. After duplicate articles were excluded, titles and abstracts reviewed, and full-text articles screened, 8 studies were included in the final analysis. See Appendix 2 (available at www.cmajopen.ca/content/11/5/E956/suppl/DC1) for a list of full-text articles screened and reasons for exclusion.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagrams of articles identified on initial screening, updated in June 2021 and July 2022 and included in the final analysis.
Characteristics of included studies
Characteristics of studies included in the analysis are presented in Table 1. Of the 8 included studies, 7 were conducted in the US9,18–23 and 1 in Latvia.24 All 8 studies were completed in emergency departments.
Study characteristics
Outcomes
Primary outcome: mortality
Four studies included the primary outcome of interest, pediatric mortality.9,19,22,24 Table 2 describes the study outcomes, including the adjusted ORs that were used for analysis. The study by Ames and colleagues9 compared the odds of mortality between emergency departments in the highest (fourth) versus lowest (first) WPRS quartiles. The study demonstrated that critically ill children have a significantly lower risk of mortality if they present to an emergency department with a high WPRS (OR 0.25, 95% CI 0.18 to 0.37).9 Secondary analysis showed no significant association between emergency department pediatric readiness scores and mortality for children with cardiac arrest (OR 0.23, 95% CI 0.02 to 2.16) or sepsis (OR 0.59, 95% CI 0.05 to 7.31). However, they did identify a significant decrease in mortality risk for children with traumatic brain injury presenting to emergency departments with high WPRS (OR 0.21, 95% CI 0.06 to 0.78).9
Primary outcome: mortality
The study in Latvia by Balmaks and colleagues24 demonstrated that higher WPRS was associated with lower 6-month mortality (OR 0.93, 95% CI 0.88 to 0.98).24 The authors were contacted in order to rescale the OR to be comparable with the other primary outcome studies; however, the authors could not provide us with the necessary data to be included in the meta-analysis.
The 2021 study by Newgard and colleagues19 evaluated the association between emergency department pediatric readiness and in-hospital mortality among injured children presenting to trauma centres. This study also demonstrated that injured children who were treated in trauma centres with high WPRS had lower risk of mortality when comparing the highest and lowest WPRS quartiles (OR 0.58, 95%CI 0.45 to 0.75). The 2022 study by Newgard and colleagues22 evaluated the association between emergency department pediatric readiness and mortality including in-hospital and 1-year survival among children presenting to trauma centres. This study found that children presenting with injuries to trauma centres with the highest quartile of pediatric readiness had significantly lower risk of death in hospital (adjusted OR 0.61, 95% CI 0.42 to 0.89) and by 1 year (adjusted OR 0.60, 95% CI 0.44 to 0.81).22
Random-effects meta-analysis was conducted to examine the association between WPRS and mortality including the studies by Ames and colleagues,9 Newgard and colleagues,19 and Newgard and colleagues22 (Figure 2). The pooled estimate of OR of the 3 included studies9,19,22 was 0.45 (95% CI 0.26 to 0.78; I2 = 89%, low certainty of evidence), indicating high heterogeneity between the 3 studies. We performed a sensitivity analysis with either of the Newgard and colleagues studies19,22 and found pooled estimates OR of 0.39 (95% CI 0.16 to 0.93) and OR of 0.38 (95% CI 0.17 to 0.87), respectively.
Random-effects meta-analysis of the association between in-hospital mortality and weighted pediatric readiness score. Note: CI = confidence interval, IV = inverse variance, SE = standard error.
Secondary outcome: health care outcomes and utilization
Five studies included the secondary outcome of interest18,20,21,23,24 (Table 3). The study by Balmaks and colleagues24 showed that a 1-point increase in WPRS was associated with shorter ICU length of stay (β −0.06 days, 95% CI −0.10 to −0.01) and shorter hospital length of stay (β −0.36 days, 95% CI −0.61 to −0.10).
Secondary outcome: health care outcomes and utilization
The studies by Ray and colleagues18 and Newgard and colleagues23 quantified children’s geographic access to emergency departments in the US with high WPRS. Ray and colleagues18 identified that 93.7% of children have access to an emergency department within a 30-minute drive.18 They found that 33.7% of children in the US have access to an emergency department with WPRS of 100, and 55.3% have access to an emergency department with a WPRS score in the 90th percentile or greater (WPRS score ≥ 94.3).18 Newgard and colleagues23 examined how often children were transported by emergency medical serves to emergency departments with high pediatric readiness, and number of children within 30 minutes of an emergency department with high fourth quartile WPRS score. Newgard and colleagues23 found that 50.85% of high-risk children were transported to a WPRS with highest quartile score. However, 23.26% of high-risk children were transferred to emergency departments with lower WPRS but were within 30 minutes of an emergency department with high WPRS, which they hypothesize would have had a measurable effect on patient survival.
Both studies by Lieng and colleagues20,21 investigated the association between WPRS and transfers between facilities. Lieng and colleagues20 found that a 10-point increase in WPRS is associated with lower odds of potentially avoidable transfers in injured (OR 0.92, 95% CI 0.86 to 0.98) and noninjured (OR 0.94, 95% CI 0.88 to 1.00) children. Additionally, Lieng and colleagues21 concluded that children presenting to small rural hospital emergency departments with higher WPRS (score > 70) are less likely to be transferred to another facility (OR 0.55, 95% CI 0.33 to 0.93).
The findings of our assessment on the certainty of evidence for the included studies are presented in Table 4 and Table 5. There is low certainty of evidence for mortality and moderate certainty of evidence for the studied health care outcomes and utilization.
Summary of findings, high WPRS compared with low WPRS in mortality
Summary of findings, high WPRS compared with low WPRS in health care outcomes and utilization
Risk of bias across studies
The NOS14 was used to evaluate the included studies. The results of the assessment are presented in Table 6. All 8 studies were rated as having a low risk of bias in the areas of representativeness of exposed sample, selection of the nonexposed sample, ascertainment of exposure, comparability, and follow-up and statistical tests (where applicable). All articles were classified as high risk for selection bias as they could not demonstration that the outcome of interest was not present at the start of the study.
Risk of bias: Newcastle–Ottawa Scale quality-assessment summary
Interpretation
The results of our systematic review highlight that a critically ill or injured child who presents to an emergency department with a high pediatric readiness score has a lower risk of mortality than a child presenting to an emergency department with low pediatric readiness score. Our study also identified that higher pediatric readiness scores as assessed by WPRS are associated with shorter length of stay in hospital and lower rates of interfacility transfer, which can have an impact on patient outcomes.
The 4 articles included in the systematic review that assessed pediatric mortality in relation to WPRS each found that higher WPRS significantly lowers the risk of mortality. 9,19,22,24 All studies evaluated a diverse range of centres and pediatric volumes, suggesting that these findings are relevant for a wide range of emergency departments. When 3 of 4 studies9,19,22 were combined in the random-effects meta-analysis, a higher WPRS was significantly associated with lower risk of mortality. These findings have important implications for pediatric emergency medicine, as they support all hospitals advocating for improved access to pediatric-specific resuscitation equipment, medication dosing, interfacility transfer guidelines, emergency department policies and care coordinators.6 As higher WPRS and readiness to care for critically ill and injured children has a direct impact on risk of mortality, emphasis should be placed on preparing all emergency departments for children.
The systematic review also highlighted several important health care utilization outcomes associated with pediatric readiness. Length of stay in the intensive care unit and hospital is a common quality indicator for patient care, as well as an important factor when considering hospital resource allocation. 25 Length of stay is a multifactorial measure; however, if by increasing WPRS, hospital length of stay decreases, this has positive impact on the patient, patient outcomes, and hospital costs and resource use.
Ray and colleagues18 identified that 93.7% of children in the US live within a 30-minute drive of any emergency department; however, only 33.7% of children live within a 30-minute drive of an emergency department with a WPRS of 100. Additionally, Newgard and colleagues23 identified that approximately 50% of high-risk children were transported to emergency departments with the highest level of pediatric readiness, and that 23% of children were transported to emergency departments with lower WPRS but were within 30 minutes of an emergency department with high WPRS. Improving emergency department readiness to care for children in all types of centres could lead to improved mortality rates for children.26–28 Furthermore, if children are presenting to hospitals with low readiness scores, they are more likely to require interfacility transportation, as concluded by Leing and colleagues.20,21 Although sometimes necessary, interfacility patient transfers can involve increased risk of psychological distress, delay in accessing care, repetition of care, communication issues, increased mortality and increased costs.7,29
Future research is needed to continue to explore the association between pediatric readiness scores and mortality, as well as the role of WPRS in emergency departments in rural or remote communities, and whether location affects WPRS and mortality. To advocate for implementation of WPRS and pediatric readiness in all emergency departments, barriers to implementation and strategies for improvement should be explored. More rigorous studies of WPRS, such as randomized control trials, would be beneficial to identify evidence-based strategies to improve WPRS and pediatric mortality.
Limitations
There were relatively few studies identified during the systematic review, and mortality is a relatively rare pediatric outcome, which limits the strength of the evidence or the robustness of the findings. We also acknowledge that the studies included in the meta-analysis are observational studies with possible residual confounding. Residual confounding cannot be completely eliminated in studies, especially in observational studies. Although plausible confounders have been adjusted for in the included studies, improper categorization, or misclassification of variables such as emergency department characteristics, patient demographics and severity of illness could distort the estimate reported in the included studies. Although all studies included children presenting to emergency departments, there is a slight variation in the characteristics of the emergency departments, patients admitted, severity of injury and the time frame used in estimating the mortality. Some studies included all critically ill children and others included injured children only, which may affect mortality rates and contribute to the relatively high heterogeneity of the model. These studies, however, presented considerably adjusted point estimates. We were also unable to conduct a meta-analysis for the secondary outcomes due to the wide range of outcomes in the included studies. However, the secondary outcomes consistently supported the improvement of WPRS in emergency departments. The WPRS is a relatively new score,6,7 which may explain why there is limited published data on readiness scores and pediatric outcomes. Although we performed a comprehensive literature search and had 2 individuals screening articles, there is a possibility that relevant studies could have been missed. As well, the study included articles published in English only, which could have limited the results. Another limitation was that 7 of the 8 included studies were from the US, which has a relatively different health care system from other countries.
Conclusion
The data suggest a potential inverse association between the WPRS of the emergency departments and mortality risk in children. More studies are needed to refute or confirm these findings.
Acknowledgements
The authors thank the administrative staff of The Children’s Hospital Research Institute of Manitoba for their support and contributions to our research work.
Footnotes
Competing interests: None declared.
This article has been peer reviewed.
Contributions: All authors contributed to the conception and design of the work, and the acquisition, analysis and interpretation of the data. All authors drafted the manuscript, revised it critically for important intellectual content, and provided final approval of the version to be published. All authors agreed to be accountable for all aspects of the work.
Funding: The study was supported by a grant from the Manitoba Medical Service Foundation awarded to Alex Aregbesola, and a grant from the Children’s Hospital Foundation of Manitoba awarded to Terry Klassen.
Data sharing: Data included in this systematic review can be requested by contacting the corresponding author.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/11/5/E956/suppl/DC1.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
- © 2023 CMA Impact Inc. or its licensors
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.