Article Text
PDF
Abstract
Objectives To investigate whether adding lactate to the quick Sequential (sepsis-related) Organ Failure Assessment (qSOFA) improves the prediction of mortality in adult hospital patients, compared with qSOFA alone.
Design Systematic review in accordance with Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines.
Data sources Embase, Medline, PubMed, SCOPUS, Web of Science, CINAHL and Open Grey databases were searched in November 2020.
Eligibility criteria Original research studies published after 2016 comparing qSOFA in combination with lactate (LqSOFA) with qSOFA alone in adult patients with sepsis in hospital. The language was restricted to English.
Data extraction and synthesis Title and abstract screening, full-text screening, data extraction and quality assessment (using Quality Assessment of Diagnostic Accuracy Studies-2) were conducted independently by two reviewers. Extracted data were collected into tables and diagnostic test accuracy was compared between the two tests.
Results We identified 1621 studies, of which 11 met our inclusion criteria. Overall, there was a low risk of bias across all studies. The area under the receiver operating characteristic (AUROC) curve for qSOFA was improved by the addition of lactate in 9 of the 10 studies reporting it. Sensitivity was increased in three of seven studies that reported it. Specificity was increased in four of seven studies that reported it. Of the six studies set exclusively within the emergency department, five published AUROCs, all of which reported an increase following the addition of lactate. Sensitivity and specificity results varied throughout the included studies. Due to insufficient data and heterogeneity of studies, a meta-analysis was not performed.
Conclusions LqSOFA is an effective tool for identifying mortality risk both in adult inpatients with sepsis and those in the emergency department. LqSOFA increases AUROC over qSOFA alone, particularly within the emergency department. However, further original research is required to provide a stronger base of evidence in lactate measurement timing, as well as prospective trials to strengthen evidence and reduce bias.
PROSPERO registration number CRD42020207648.
- infectious diseases
- health & safety
- risk management
- adult intensive & critical care
- chemical pathology
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This review search was comprehensive and systematic, using multiple databases and hand searching to investigate whether combining lactate with quick Sequential Organ Failure Assessment better stratified the risk of mortality in adult inpatients.
The review search was restricted to studies published in English, and thus placed a bias towards English-speaking countries.
Two team members worked independently on study screening, data extraction and the risk of bias assessment, with a third team member available to settle disagreements, reducing the risk of confirmatory bias.
Due to insufficient data and heterogeneity of the included studies, a valid meta-analysis could not be performed.
Introduction
Throughout the last few decades, sepsis has persisted as a leading cause of mortality worldwide.1–3 In 2017, sepsis was associated with an estimated 19.7% of all global deaths, and in 2013 cost the USA an estimated $24 billion.3–5 Initiating sepsis treatment as early as feasible is critical to reducing sepsis mortality and improving patient outcomes.6 7 As such, many clinical tools have been developed to assist in the rapid clinical recognition of sepsis upon patient presentation.8–10
The quick Sequential Organ Failure Assessment (qSOFA) pathway is a rapid bedside tool that ‘provides simple bedside criteria to identify adult patients with suspected infection who are likely to have poor outcomes’.10 The qSOFA score is classified as: (1) altered mentation; (2) a systolic blood pressure <100 mm Hg; and (3) a respiratory rate ≥22 breaths/min, with patients who meet two or more out of the three criteria considered higher risk.11 However, there have been recent concerns regarding the low sensitivity of qSOFA, resulting in attempts to improve its accuracy.12–15
Lactate, a metabolite resulting from cellular anaerobic metabolism, is associated with sepsis and septic shock.16 Hyperlactatemia in isolation has been shown to increase the risk of 90-day mortality in patients with sepsis by 1.7 times.17 As such, lactate alone has been postulated as a diagnostic and prognostic tool for sepsis.18 19 Point-of-care lactate has shown promise as a marker of sepsis severity and has been validated in several studies to improve the performance of bedside recognition algorithms.20 21 Combining biomarkers, such as lactate, with pre-existing clinical scores, such as qSOFA, could enhance swift sepsis recognition,22 23 improving overall patient mortality and morbidity. Kashyap et al investigated qSOFA and lactate during a larger review on the enhancement of sepsis identification scores, but did not study this tool in detail.24
This review aimed to identify and provide high-level evidence as to whether lactate as an additive to qSOFA (LqSOFA) will enhance the rapid stratification of sepsis risk in adult patients with sepsis in various hospital settings, compared with qSOFA alone.
Methods
This study is a systematic review searching multiple online databases using a PROSPERO-registered search strategy (CRD42020207648). We have followed the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines for reporting throughout the review.25 26 A completed PRISMA-DTA checklist is attached in online supplemental appendix 1.
Supplemental material
Search strategy
To identify relevant papers, we searched Embase, Medline (both via Ovid), Web of Science, Scopus, PubMed, and the Cumulative Index to Nursing and Allied Health Literature. To capture grey literature, we used the Open Grey database. The search strategy was developed by two authors (AG, KA) with the assistance of a clinical librarian, and reviewed by a third reviewer (LL). Search strategies for all databases can be seen in online supplemental appendix 2. Only papers written in English or with readily available English translations were considered for inclusion due to time and resource constraints.
Supplemental material
All references retrieved by the search were imported into an EndNote V.X9 library (Clarivate) and duplicates removed. Two reviewers (AG, KA) then independently applied the predefined eligibility criteria (below) for study selection through first title and abstract screening, followed by full-text retrieval and screening. Disagreements on included research were resolved by discussion, or by a third independent reviewer (LL) if required.
Eligibility criteria
Studies were included if they: (1) were original research; (2) were published after 2016; (3) examined adult hospital patient populations only; (4) reported combined qSOFA and lactate score results, as well as qSOFA alone as a comparator; (5) had a population defined as having a confirmed diagnosis or a suspected diagnosis of sepsis, systemic inflammatory response syndrome or septic shock; (6) were published in English.
Original research only was included, defined as published original research with relevant clinical results, including randomised controlled trials, observational studies, prospective and retrospective cohort analyses, as well as grey literature such as published conference abstracts. Opinion pieces, such as editorials and commentaries, reviews and other references containing unpublished non-original research were excluded. Papers published before 2016 were excluded as this was prior to the publication of the qSOFA criteria.11 Following discussion and full-text screening, the inclusion criteria were further clarified to require papers to look at the use of qSOFA and lactate specifically within the context of sepsis as defined in the included studies.
Quality assessment
Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool.27 Two reviewers (AG, KA) independently performed the quality assessment, with disagreements resolved through discussion or by a third reviewer (LL). Any biases of selected papers found, as well as the implications of bias, were discussed in narrative synthesis.
Patient and public involvement
This is a systematic review. No patients are involved.
Data extraction
Data were extracted from the selected studies using a data extraction form in Microsoft Excel. Data extracted included year of publication, study author(s), country of study setting, funding sources, number of participants, patient demographics, study design (eg, prospective or retrospective cohort), length of stay, intensive care unit (ICU) admission status and mortality. Data related to the implementation of qSOFA and combined qSOFA and lactate tools, and the threshold values of lactate used, were also extracted from papers. The type of mortality investigated, sensitivity, specificity, area under the receiver operating characteristic (AUROC) curve or data related to calculating these metrics were extracted if available. Seven authors were contacted to request missing data,21 28–33 three of which replied.21 29 31
Data synthesis
Data were analysed in the style of a narrative review with accompanying summary statistics due to the high variability within the included studies. Extracted data were collated within tables to allow for better comparison and analysis between studies.
We calculated sensitivity and specificity from data made available by authors if needed. The change in sensitivity, specificity and AUROC after the addition of lactate was calculated for each study. Studies involving the ICU only and emergency department (ED) were analysed as subgroups. Mortality outcomes were also stratified as subgroups and analysed. Additional data, including positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic OR (DOR) and raw outcome data (ie, true positive, false positive, true negative and false negative), were calculated from available data (reported and via author correspondence) and included in online supplemental tables 1–3.
Supplemental material
Results
Search results
Our search strategy was executed on the 2 November 2020, returning 2744 citations. After deduplication, 1621 articles remained. Following title and abstract screening, 26 articles remained, excluding 1595 articles. From this, full texts were retrieved, and 11 studies selected for inclusion.20 21 28–36 A flow chart demonstrating this process can be seen in figure 1.
PRISMA flow chart for study selection. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; qSOFA, quick Sequential Organ Failure Assessment.
Study characteristics
The characteristics of included studies are presented in table 1. Eleven studies were identified ranging from 2017 to 2020 in publication year. These studies were conducted in a diverse range of countries, with studies set in Australia (n=1), the USA (n=3), China (n=2), South Korea (n=2), Brazil (n=1) and across multiple countries (n=1). Only one study did not report the country it was set in. Six studies recruited only from the ED, with two recruiting from ICU, one from both ED and inpatient wards, and two from a hospital-wide cohort. Study population sizes ranged from 165 to 55 945.
Study characteristics
The main outcome for all studies was mortality. The type of mortality reported varied (table 1). Three studies used 30-day or 28-day mortality as their primary outcome,29 30 36 four studies used in-hospital mortality,20 31 33 35 one study used 7-day mortality32 and one other did not specify (table 2).34 Of the final two studies, one used a composite of ICU length of stay, vasopressor use and mortality within 72 hours of presentation,28 and the other looked at a composite of adverse outcomes, specified as either a prolonged stay in ICU ≥72 hours or in-hospital mortality.21
qSOFA versus LqSOFA mortality outcome measurement
LqSOFA characteristics
Study outcomes
Nine of the 10 studies reporting an AUROC found an improvement following the addition of lactate regardless of the type of mortality investigated (table 2), with the greatest increase being 23.9% demonstrated by Liu et al.33 In addition, sensitivity was increased by the addition of lactate in three of seven studies that reported sensitivity, and specificity was increased in the other four. The greatest change in sensitivity was demonstrated by Machado et al, with a 69.4% increase with the addition of lactate (table 2).35 The greatest change in specificity was demonstrated by Zhou et al, with a 63.4% increase in specificity in LqSOFA over qSOFA.36
Four studies reported data specifically on in-hospital mortality (table 1). The AUROC was improved by the addition of lactate in all four studies (table 2). Liu et al (2019) demonstrated the largest rise in AUROC, from 0.544 to 0.674 for qSOFA and LqSOFA, respectively; a 23.9% increase over qSOFA alone.33
Six studies were set exclusively within the ED, five of which recorded AUROCs (table 2). The AUROC increased after the addition of lactate in five studies. Chae et al demonstrated the largest increase in AUROC in LqSOFA, with a +16.7% increase.29 Sensitivity was increased by the addition of lactate in two of the five studies reporting it and specificity was increased in the other three (table 2). Given the heterogeneity across studies, for example, study population, lactate thresholds, outcome types and time points of outcome measurement, meta-analysis would not provide meaningful results. A summary ROC plot (sensitivity vs 1-specificity) of five ED studies shows that results of all studies are above the chance level (online supplemental figure 1).
Six studies reported the PPV and NPV following the addition of lactate to qSOFA with four studies reporting an increase to PPV, two reporting a decrease and vice versa for the NPV (online supplemental table 1). Similarly, the seven studies demonstrating a change in PLRs and NLRs are relatively evenly split with three studies demonstrating a decrease in PLR, four demonstrating an increase, four studies demonstrating an increase in NLR and three studies a decrease (online supplemental table 2). Of the seven studies with DOR, six reported an increase when lactate was added to qSOFA (online supplemental table 2).
qSOFA and LqSOFA cut-off thresholds and lactate measurement timing
Seven studies reported a lactate threshold of ≥2 mmol/L (table 3). Other lactate threshold levels used were ≥4 mmol/L (n=1), ≥3.225 mmol/L (n=1), and groups of <2 mmol/L, 2–4 mmol/L, and >4 mmol/L (n=1). Only one study did not report their lactate threshold. The definition of a positive ‘LqSOFA’ varied widely between studies, with seven studies reporting five different score thresholds (table 3). The time of lactate measurement and collection of clinical data also varied between studies (table 3).
Risk of bias
The risk of bias was generally low as seen in figure 2. Five studies demonstrated a low risk of bias among all domains. The index test domain was the most likely to introduce bias among the included studies, with two studies judged as high, and three as unclear risk of bias (figure 2). In addition, one study demonstrated a high risk of bias in the flow and timing domain (figure 2).
Risk of bias. In accordance with QUADAS-2 tool guidelines.27 Green or (√) symbol denotes low risk of bias. Red or (X) denotes high risk of bias. Yellow or (!) denotes unclear risk of bias. QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies-2.
Discussion
Overall, LqSOFA demonstrated an increased accuracy for the identification of patients at higher risk of mortality compared with qSOFA alone, as evidenced by an improvement in the AUROC upon the addition of lactate in 9 of the 11 included studies. This increased accuracy will translate to earlier identification of patients who would otherwise be overlooked, followed by earlier treatment initiation, escalation of care and better patient outcomes.6 7 In addition, most studies reported an increase in DOR following the addition of lactate to qSOFA, suggesting that there has been an increase in the test’s ability to diagnose the risk of mortality in patients with suspected sepsis correctly, further reinforcing the merit of adding lactate to qSOFA.
Our findings regarding changes in sensitivity and specificity following the addition of lactate to qSOFA are more inconsistent. Unsurprisingly, studies reporting an increase in sensitivity also reported a decrease in specificity, and vice versa. Of the included seven studies reporting sensitivity and specificity values, three21 34 35 reported an increase in sensitivity and decrease in specificity, and four28 29 31 36 reported an increase in specificity and decrease in sensitivity. High sensitivity of a sepsis recognition tool is beneficial as sepsis deteriorates rapidly.37 The earlier patients are identified, the earlier treatment begins, and therefore it is critical to identify as many sepsis cases as early as possible.38 However, a high-test specificity is also beneficial as it can reduce unnecessary antibiotic use or fluid resuscitation treatment, both of which have been linked to negative outcomes.39–41 This is particularly true in low-income to middle-income countries, as studies have shown unnecessary treatment to be associated with mortality in sepsis.42 Therefore, while our findings regarding the sensitivity and specificity of LqSOFA are heterogenic, both an increase in sensitivity or specificity would be beneficial in sepsis screening.
The qSOFA tool was designed to be fast, simple, and highly specific for predicting organ dysfunction and mortality, and is therefore ideal for use as a bedside tool in the ED.11 43–45 Lactate is often taken as part of normal screening and sepsis management within the ED, making it a viable and attractive candidate to add to the qSOFA score.46 47 Furthermore, lactate is a known marker of illness severity and septic shock, reinforcing its suitability for combination with qSOFA.10 47 We found all studies investigating LqSOFA in the ED demonstrated an increased AUROC, and hence ability to accurately predict mortality, after the addition of lactate (table 2). Thus, lactate would be a useful and valuable addition to qSOFA in the ED and other emergency situations.
Further to this, there is some evidence for the use of lactate in guiding sepsis therapy.47–49 Lactate levels have been shown to correlate with disease severity, and increased lactate clearance has long been known to correlate with survivability.47 49–51 Using lactate kinetics as part of lactate clearance-driven therapy could improve patient outcomes.52–55 Adding lactate to qSOFA would enhance the ability of qSOFA to not only identify patients at high risk of sepsis-related poor outcomes, but to also guide the management of patients treated as septic. Increasing the frequency of LqSOFA measurement could facilitate its use throughout the entirety of a patient’s hospitalisation. This would provide clinicians an additional tool for the simple and efficient assessment of a patient’s health, supporting regular patient monitoring. This may improve patient outcomes and is thus highlighted as an area for future research. However, lactate scores can be altered by several different mechanisms.47 49 Other conditions that interfere with tissue oxygenation or alter metabolic demand, such as diabetic ketoacidosis, seizures, metabolic liver disease as well as iatrogenic causes, can result in a raised lactate.47 49 56 Clinicians and researchers should be aware of these confounding factors when using or investigating the use of LqSOFA for both prognosis and sepsis management.
The studies included in this review use a range of lactate thresholds, including 2, 3.25 and 4 mmol/L (table 3). The lactate threshold used when screening a patient for sepsis risk will greatly affect the following risk estimation. Within the literature currently, there is a poor consensus for the appropriate lactate cut-off. Some studies indicate a lower lactate threshold, such as 1.3 mmol/L, can still have a significant correlation with prognosis, while others indicate a ≥2 mmol/L or ≥4 mmol/L concentration is the optimal threshold.57–59 Furthermore, we have highlighted considerable diversity in what lactate measurements are used when calculating the LqSOFA score (table 3). Some studies make use of the worst score within the first 24 hours, while others use parameters collected at ED triage. Given the dynamic nature of blood lactate levels,49 this difference in measurement timing will likely affect the final LqSOFA score. Furthermore, as many of these studies are retrospective, this does not give an accurate reflection of how LqSOFA will be measured when implemented in real time. Future research should aim to maintain pragmatic and consistent lactate thresholds and measurement timings, better reflecting the real-world use of the LqSOFA tool, and hence improving the evaluation validity.
Lastly, there is some evidence that venous lactate does not always correlate well with arterial and central lactate values, especially during hyperlactatemia.47 However, taking arterial lactate is difficult and not always practical, and venous lactate may be more suitable for a quick and simple tool such as LqSOFA.47 Clinicians and researchers should be aware of how the source of the lactate may influence the test result. Future large multicentre studies investigating the correlation between arterial and venous lactate would better elucidate this relationship and shed light on how this may influence LqSOFA.
To our knowledge, this systematic review is the first examining the addition of lactate to the qSOFA pathway. We used multiple databases to ensure a comprehensive and systematic search of the literature and completed this review to current PRISMA-DTA standards. However, we were unable to perform a valid meta-analysis due to insufficient data and heterogeneity of studies. Across the 11 included studies, there were six different measures of mortality reported. A total of four studies used the same time point to measure mortality; however, these four studies examined very different study populations, including all hospital patients, only ED patients or only ICU patients. Each of these populations is uniquely different, particularly in terms of sepsis severity, and therefore combining these studies for meta-analysis would not result in valid conclusions. Future meta-analyses could consider certain focused subsets as published studies accumulate on a specific mortality measure or among certain population (eg, ED patients). With enough studies, a valid meta-analysis could model sensitivity and specificity jointly using either the Hierarchical Summary Receiver Operating Characteristic Model or the Bivariate Model.60 In addition, we only reviewed trials available in English due to the time and resource constraints, and therefore may have missed articles published in other languages.
Conclusion
LqSOFA increased the accuracy of qSOFA alone in detecting patients at higher risk of sepsis-related mortality in many hospital settings, including the ED. Further research should investigate the use of LqSOFA prospectively, with focus on a more standardised or dynamic lactate measurement timing.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
This study did not receive nor require ethics approval, as it does not involve human and animal participants.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @lli_sydney
Contributors LL was responsible for the overall study as the guarantor. AG drafted the protocol. LL and KA reviewed the protocol. AG and KA conducted the literature search. AG, KA, LL, CH and VL all contributed to the production and review of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.