Can mid-regional pro-adrenomedullin (MR-proADM) increase the prognostic accuracy of NEWS in predicting deterioration in patients admitted to hospital with mild to moderately severe illness? A prospective single-centre observational study
=============================================================================================================================================================================================================================================

* Sara Graziadio
* Rachel Amie O’Leary
* Deborah D Stocken
* Michael Power
* A Joy Allen
* A John Simpson
* David Ashley Price

## Abstract

**Objective** To assess the value added to the National Early Warning Score (NEWS) by mid-regional pro-adrenomedullin (MR-proADM) blood level in predicting deterioration in mild to moderately ill people.

**Design** Prospective observational study.

**Setting** The Medical Admissions Suite of the Royal Victoria Infirmary, Newcastle.

**Participants** 300 adults with NEWS between 2 and 5 on admission. Exclusion criteria included receiving palliative care, or admitted for social reasons or self-harming. Patients were enrolled between September and December 2015, and followed up for 30 days after discharge.

**Outcome measure** The primary outcome measure was the proportion of patients who, within 72 hours, had an *acuity increase*, defined as any combination of an increase of at least 2 in the NEWS; transfer to a higher-dependency bed or monitored area; death; or for those discharged from hospital, readmission for medical reasons.

**Results** NEWS and MR-proADM together predicted *acuity increase* more accurately than NEWS alone, increasing the area under the curve (AUC) to 0.61 (95% CI 0.54 to 0.69) from 0.55 (95% CI 0.48 to 0.62). When the confounding effects of presence of chronic obstructive pulmonary disease or heart failure and interaction with MR-proADM were included, the prognostic accuracy further increased the AUC to 0.69 (95% CI 0.63 to 0.76).

**Conclusions** MR-proADM is potentially a clinically useful biomarker for deterioration in patients admitted to hospital with a mild to moderately severe acute illness, that is, with NEWS between 2 and 5. As a growing number of National Health Service hospitals are routinely recording the NEWS on their clinical information systems, further research should assess the practicality and use of developing a decision aid based on admission NEWS, MR-proADM level, and possibly other clinical data and other biomarkers that could further improve prognostic accuracy.

*   internal medicine

### Strengths and limitations of this study

*   This is the first study to use rigorous statistical methods to assess the value added by mid-regional pro-adrenomedullin (MR-proADM) to the admission National Early Warning Score for predicting clinically important deterioration in mild to moderately ill patients.

*   Overall prognostic accuracy might have been greater had more severely ill patients been included, but the aim of this study was to predict deterioration in less severely ill patients who could benefit from closer observation.

*   This was an observational study and thus could not directly assess the use of more accurate prediction of deterioration.

*   Initial evidence for MR-proADM as a biomarker for deterioration appears promising, but requires further validation for clinical use.

## Introduction  

The National Early Warning Score (NEWS) is recommended for assessing severity of illness in patients presenting in primary or secondary National Health Service (NHS) care and for surveillance of patients in hospital.1 2 Six physiological parameters (which can be measured at the bedside) are scored: respiratory rate, oxygen saturation, temperature, systolic blood pressure, pulse rate and level of consciousness. The scores are aggregated, and, if the patient requires oxygen, the total is increased. NEWS predicts death, cardiac arrest and unplanned intensive care unit (ICU) admission within 24 hours.3–5 However, NEWS does not identify all patients who turn out to be seriously ill,6–8 and there are also patients whose NEWS is usually elevated and who do not require the level of observation that the NEWS tool would suggest. For example, people with chronic obstructive pulmonary disease (COPD) or chronic heart failure (HF) have higher baseline NEWSs than those without these comorbidities. The prognostic accuracy of NEWS for patients presenting to the emergency department (ED) has been confirmed in a wide range of severity of illness,9 10 as has its reduced accuracy in people with COPD.11 However, no previous studies of the prognostic accuracy of NEWS in the ED/medical admissions unit (MAU, also called Assessment Suite (AS)) have focused on patients admitted with mild to moderately severe illness. Since a clinically important proportion of these patients do deteriorate unexpectedly, improved risk stratification would be useful.

Mid-regional pro-adrenomedullin (MR-proADM) is one of several promising biomarkers for severe illness and deterioration.12–16 

MR-proADM is a precursor of adrenomedullin (ADM), a member of the calcitonin peptide family. ADM is widely expressed and has roles in vasodilation, immune modulation and metabolic regulation. It is upregulated in severe infections, inflammation, vasodilation, stimulation of diuresis, increased cardiac output and stroke.17–19 ADM has a short half-life, but MR-proADM is more stable and directly reflects ADM concentrations in blood. Both ADM and MR-proADM levels are strongly associated with risk of mortality, regardless of aetiology.20–26 In people presenting with acute chest pain, MR-proADM has been reported to improve the Global Registry of Acute Coronary Events risk classification by 41%.27 As with the NEWS, people with COPD or chronic HF have higher baseline levels of MR-proADM.

The aim of this study was to assess whether the MR-proADM level used alongside the NEWS would improve prediction of deterioration over NEWS alone in patients admitted to the MAU with mild to moderately severe illness.

## Methods

### Patient and public involvement

Patients and the public were not specifically involved in the planning and execution of this study. However, the National Institute for Health Research now requires that the research it supports includes active involvement and engagement with patients and the public.

### Study participants and study design

This was a prospective observational cohort study. Patients were enrolled between September and December 2015 at the Royal Victoria Infirmary, Newcastle, and followed up for 30 days after discharge. If the patient died within 30 days of follow-up, this and the cause of death were recorded. Adults admitted to the MAU were recruited for the study between 9:00 and 16:00 on weekdays.

Sample size was based on a pragmatic recruitment target for a 3-month observational study. A recent unpublished audit conducted in the MAU at the Royal Victoria Infirmary found a deterioration rate of 20%. With 300 patients and complete data collection, 60 events would be anticipated. With this number of events, a multivariable prediction model could include up to six independent predictors. This is based on a widely accepted rule of thumb that models with fewer than 10 events per predictor tend to be overfitted.28 However, recent research suggests that the ‘ten events per variable’ rule of thumb may be optimistic.29 Because the aim of this study was to assess if further research would be indicated, 60 is considered an acceptable number of events, even if the rule of thumb is optimistic.

Patients were considered eligible for inclusion in the study if their NEWS on admission was at least 2 and not greater than 5, and all NEWS parameters were recorded. Patients were excluded from the study if they were receiving palliative care, were admitted for social reasons only, or were self-harming, or overdosing with drugs or other substances.

### Recorded data

Demographic and admission data included gender, year of birth, reason for admissions, diagnosis on discharge and the presence of comorbidities in which baseline MR-proADM levels are chronically raised: COPD with hypoxia (PaO2 <10 kPa)7; HF30; acute brain injury6; acute coronary syndrome27; acute venous thromboembolism21; high international normalised ratio (>2); acute kidney injury; electrolyte disturbances (Na+ <130 or >150 mmol/L; K+ <3.0 or >5.5 mmol/L); hyperglycaemia in type 1 diabetes (random glucose >10 mmol/L).

The NEWS was assessed at admission and over the next 72 hours, and the scores and assessment times were recorded. The seven clinical parameters used to determine the NEWS were recorded for the baseline (admission) assessment only. Baseline NEWSs were used to determine eligibility for this study. Subsequent NEWSs were used in the analyses to identify deterioration.

Blood samples were taken at hospital admission for assessment of MR-proADM, C reactive protein (CRP) and white blood cell (WBC) count.

### Laboratory tests

Plasma was obtained from blood samples (collected in EDTA) that were no longer clinically required. Plasma was stored in aliquots at –80°C.

MR-proADM was assayed in the on-site Blood Sciences Laboratory using the B·R·A·H·M·S Kryptor system according to the manufacturer’s instructions.

Blood samples were analysed in batches by personnel blinded with regard to the condition and NEWS of the patient. Nurses who assessed the NEWS and healthcare professionals managing patients in the MAU were blinded to MR-proADM results.

### Outcome measures

#### Outcome measure 1: *acuity increase*

A patient was classified as having an a*cuity increase* if one or more of the following occurred within 72 hours from admission:

1.  Transfer to a higher level of care (ICU or high-dependency unit).

2.  Readmission to hospital for reasons related to the initial admission.

3.  Death for reasons related to the initial admission.

4.  NEWS increased by at least 2 compared with the admission score.

#### Outcome measure 2: *deterioration event*

For most of the observed *acuity increase* cases, the reason for classification was an increase in the NEWS (table 1). An increase in NEWS reflects both measurement variation and physiological variation, so additional exploratory analyses were carried out to assess the performance of MR-proADM in predicting deterioration. *Deterioration events* were classified as the occurrence of one or more of the following:

View this table:
[Table 1](http://bmjopen.bmj.com/content/8/11/e020337/T1)

Table 1 
Characteristics of the study population, classified by outcome 1 (*acuity increase*), outcome 2 (*deterioration event*) and all patients

1.  Transfer to higher level of care within 72 hours from admission;

2.  Death (for reasons related to the admission) within 30 days;

3.  Readmission to hospital (for the same reason as the previous admission) within 30 days from first admission.

Classification based on this definition is unlikely to be subject to clinically important measurement variation. This analysis, therefore, should optimise the prognostic accuracy for events which are both clinically and economically important.

#### Outcome measure 3: *length of stay*

*Length of stay* was defined as the duration (in days) from admission to discharge or death.

### Statistical analysis

All data analyses were performed using the R language, V.3.2.0,31 with the support of RStudio, V.0.99.896 (RStudio). The following R packages were used: ggplot2, pROC, psych, PredictABEL, Hmisc and rms.

Logistic regression models were compared for their accuracy in predicting deterioration outcome measures as prespecified in an analysis plan. Analyses are presented as unadjusted parameter estimates of risk (OR, with CIs) and estimates adjusted for identified clinical confounding factors. The aims of the multivariable analyses were twofold: first, to estimate the effect size and significance adjusted for other identified influential predictors and interactions; and second, to investigate whether the addition of other predictors improved the goodness of fit and accuracy of prediction.

Only complete cases were analysed since missingness was minimal: 10 records without data on comorbidities (details in footnote in table 1).

For each measure of deterioration (*acuity increase*, *deterioration event* and *length of stay*), logistic regression models were compared for the following sets of predictor variables:

*Predictor set a.* Comparator (base case): NEWS on admission.

*Predictor set b.* Primary analysis: NEWS, MR-proADM.

*Predictor set c.* Secondary analyses: NEWS and MR-proADM always included. Age, gender, CRP, WBC, presence of COPD or HF, presence of other comorbidities and interactions between predictors when appropriate.

Predictors (and the underlying assumption of linearity of their relationship with the outcome of interest) were initially investigated through univariate analyses based on simple log and quadratic functions. Transformations were applied if they improved the goodness of fit as assessed by the Akaike information criterion (AIC) and were retained in the multivariable setting. NEWS was treated as an ordinal variable. We assessed interactions through visual data exploration without significance testing as the study was not powered for this. For the multivariable regression models, the set of predictors was assessed for independence through backward elimination, based on changes in AIC.

The analysis plan for the secondary outcome of *length of stay* was similar: using multiple linear regressions based on transformed outcomes to address non-normality. Dependent and exploratory variables were log-transformed if not normally distributed. Normality was assessed by visualising the data. More details on the methods used are reported in online supplementary material 1.

### Supplementary file 1

[[bmjopen-2017-020337-SP1.pdf]](pending:yes) 

Goodness of fit of logistic regression models was assessed with the C-statistic (which is the area under the receiver operating characteristic curve and is used as a measure of discrimination) presented with 95% CIs. To assess the value added by including the MR-proADM level with the NEWS in predicting deterioration, continuous net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.32 33 

For internal validation of the statistical models, the C-statistic was evaluated after correcting for optimistic predictions through bootstrapping with 10 000 resamples.

## Results

### Study enrolment

The process of recruitment and enrolment of patients for the study is shown in figure 1. The study recruited 300 patients, and 292 were included in the analysis. Five patients were excluded because the blood samples for MR-proADM were taken more than 12 hours from baseline NEWS assessment; three patients were excluded from the primary outcome due to missing follow-up NEWSs.

![Figure 1](http://bmjopen.bmj.com/http://bmjopenuat.stage.highwire.org/content/bmjopen/8/11/e020337/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/8/11/e020337/F1)

Figure 1 
Patient recruitment process. AS, Assessment Suite, also called as Medical Admissions Unit (MAU); GP, general practitioner; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.

### Patient characteristics

Patient demographics and mean biomarker levels for each covariate are reported in table 1. The cohort was evenly divided in gender and had a mean age of 63 years and mean NEWS on admission of 3, with the majority of patients having NEWS of 2. COPD or HF was present in 28%, and 25% had other comorbidities.

The study population was homogeneous across *acuity increase* and no *acuity increase* outcomes in terms of gender, age and NEWS on admission. Table 2 shows the frequencies of criteria determining *acuity increase* and *deterioration event*. Notably, around 95% of *acuity increases* were the result of an increase in NEWS, while readmission was the reason for around 70% of *deterioration event*s.

View this table:
[Table 2](http://bmjopen.bmj.com/content/8/11/e020337/T2)

Table 2 
Criteria met by patients classified with an *acuity increase* or *deterioration event* 

Patients who experienced *acuity increase* had higher MR-proADM and CRP levels at admission, and longer *length of stay* in the hospital and in the MAU.

The prevalence of *acuity increase* was 29% (somewhat higher than the anticipated 20%). The prevalence of *deterioration event*s was 11%. The numbers of events provided sufficient statistical power to assess statistical significance for the primary outcome, *acuity increase*, but not for the secondary outcome, *deterioration event*.

### Accuracy of MR-proADM for predicting *acuity increase*

Potentially useful predictors with univariate analysis of *acuity increase* were MR-proADM (OR 1.27, 95% CI 1.02 to 1.62; p=0.037), age2 (OR 1.00, 95% CI 0.99 to 1.00; p=0.023) and the presence of COPD or HF (OR 2.25, 95% CI 1.30 to 3.91; p=0.004; online supplementary figure S1). The prognostic accuracy of CRP, WBC and NEWS did not reach the threshold of significance (p=0.88, p=0.090, table 3, and p=0.416, table 4, respectively).

View this table:
[Table 3](http://bmjopen.bmj.com/content/8/11/e020337/T3)

Table 3 
Univariate regression analyses for predicting the three outcomes of interest: *acuity increase*, *deterioration event* and *length of stay* 

View this table:
[Table 4](http://bmjopen.bmj.com/content/8/11/e020337/T4)

Table 4 
Multivariable regression analyses for the outcomes of interest: *acuity increase*, *deterioration event* and *length of stay* (outcomes 1, 2 and 3, respectively) with NEWS comparator group

The prognostic accuracy for *acuity increase* of NEWS on its own was limited and not significant (area under the curve (AUC) 0.55, 95% CI 0.48 to 0.62), but when MR-proADM was included as an additional predictor, the accuracy of the model increased (AUC 0.61, 95% CI 0.54 to 0.69; OR 1.28, 95% CI 1.02 to 1.63; p=0.039) (tables 4 and 5, figure 2A). When including MR-proADM with NEWS, the reclassification of patients was also significant, especially for the NRI (NRI 0.3, SE 0.1, p=0.007, IDI 0.017; table 4).

View this table:
[Table 5](http://bmjopen.bmj.com/content/8/11/e020337/T5)

Table 5 
Model comparisons

![Figure 2](http://bmjopen.bmj.com/http://bmjopenuat.stage.highwire.org/content/bmjopen/8/11/e020337/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/8/11/e020337/F2)

Figure 2 
(A) Prognostic accuracy for *acuity increase.* Predictor set *a*: NEWS; predictor set *b*: NEWS, MR-proADM; predictor set *c*: NEWS, MR-proADM, COPD/HF, interaction between MR-proADM and COPD/HF. (B) Comparisons as for panel (A) but for predicting a *deterioration event.* Predictor set *c:* NEWS, MR-proADM level, age2, other comorbidities. (C) *Length of stay* predicted by MR-proADM level. COPD, chronic obstructive pulmonary disease; HF, heart failure; MR-proADM, mid-regional pro-adrenomedullin; NEWS, National Early Warning Score.

The prognostic accuracy of MR-proADM and the additional value it provides to the NEWS was confirmed for *deterioration event*s and *length of stay* (tables 4 and 5, and figure 2B,C).

For MR-proADM alone, the AUCs were for *acuity increase* 0.58 (0.51 to 0.66) and *deterioration event* 0.64 (0.54 to 0.74). For *length of stay*, the R2 was 0.12.

### Effect on prognostic accuracy when clinical information is added to the set of predictors

Secondary multivariable modelling evaluated the prognostic accuracy of MR-proADM when adjusted for the clinical factors in predictive set *c*: age, gender, CRP, WBC, presence of COPD or HF, and presence of other comorbidities,

For *acuity increase*, COPD or HF comorbidity status and its interaction with MR-proADM level improved the prognostic accuracy of the model: AUC increased from 0.61 (95% CI 0.54 to 0.69) to 0.69 (95% CI 0.63 to 0.76) and net reclassification index from 0.3 to 0.4 (table 5).

For *deterioration event*s, the presence of other comorbidities (excluding COPD and HF) and age2 increased the prognostic accuracy of MR-proADM (tables 4 and 5). The prognostic accuracy of *length of stay* (outcome 3) of MR-proADM is also increased by including age in the model (tables 4 and 5, online supplementary figure S2).

### Potential confounding effects

#### Shorter-term outcomes

NEWS and MR-proADM were less accurate in predicting *acuity increase* within 24 and 12 hours from admission than in predicting *acuity increase* within 72 hours (online supplementary table S1 and S2).

#### Interval between admission NEWS scoring and blood collection

Because ward processes did not allow the times of scoring NEWS and collecting blood to be specified for research, we assessed for a confounding effect from variation in the timings, but found no evidence for it (online supplementary table S3).

#### Correlations among biomarkers

Diagnostic plots, shown in online supplementary figure S1 and S3, show no multicollinearity in the data, no autocorrelation, no heteroscedasticity and no data points that stood out in terms of their influence on results.

### Sensitivity and specificity

As overall measures of accuracy, sensitivity and specificity were calculated (where appropriate) for each model using Youden’s index. The results are shown in online supplementary table S4. In practice, the trade-off between sensitivity and specificity would depend on the type of clinical decision to be made on the result (ie, ‘rule-in’ or ‘rule out’) and this would differ from the approach in Youden’s Index, which gives equal weight to false-positive and false-negative results.

### Internal validation

C-statistic values after correcting for optimistic predictions (ie, bootstrapped average of the AUC for each model) were as follows: for *acuity increase—*predictor set *a*, C-stat 0.53; predictor set *b*, C-stat 0.59; predictor set *c*, C-stat 0.66; for *deterioration events—*predictor set *a*, C-stat 0.52; predictor set *b*, C-stat 0.61; predictor set *c*, C-stat 0.68; for *length of stay*— predictor set *a*, R2=0.003; predictor set *b*, R2=0.12; predictor set *c*, R2=0.13. AUCs decreased slightly with the bootstrapped averages, but the differences between the AUCs for predictor sets *a*, *b* and *c* were constant. These results are an internal validation, and further validation on an external dataset is required.

## Discussion

### Accuracy of prediction of deterioration by MR-proADM

This study shows that MR-proADM may be a clinically useful biomarker for predicting deterioration (ie, *acuity increase*) within 72 hours from admission to hospital in mild to moderately ill patients with admission NEWS between 2 and 5. By design, NEWSs in this range imply a low risk of deterioration, and our data are consistent with this. Previous evaluations of the NEWS assessed on admission have found that it predicts deterioration,3–5 34 which may seem inconsistent. But these studies included all patients admitted to ED, whatever their NEWS.

For most of the observed *acuity increase* events, the reason for classification was an increase in the NEWS. Because an increase in NEWS reflects both measurement variation and physiological variation, additional exploratory analyses were carried out to assess the performance of MR-proADM, using an operational definition of deterioration, *deterioration event*, designed to minimise measurement variation. NEWS on its own had low prognostic accuracy for *deterioration event*s. However, MR-proADM level and NEWS together predicted *deterioration events* with an AUC of 0.65. Considering baseline patient characteristics further increased the accuracy of the model (AUC=0.73).

### Comorbidities and interactions with MR-proADM levels

MR-proADM levels in people with COPD and/or HF are chronically raised and are not predictive of deterioration. However, in other people whose MR-proADM levels are not chronically raised, high levels are predictive of *acuity increase* (online supplementary figure S1). Including these comorbidities and their interaction with MR-proADM level increased the prognostic accuracy of the logistic regression model.

### Limitations

This study included only patients who were admitted with a NEWS between 2 and 5. The prognostic accuracy of the MR-proADM would perhaps have been greater if more extreme cases had been included. However, patients with NEWSs more than 5 are already known to be severely ill and to require close monitoring and/or management at higher levels of care.

Internal validation found that the uncorrected C-statistics are optimistic, which implies that external validation in an independent study would be useful. However, after correction for optimistic predictions, the study’s conclusions remain unchanged.

### Interpretations and implications

The contributions of MR-proADM to the accuracy of the prognostic models suggests that it could provide additional prognostic information over and above NEWS.

Secondary analyses suggest that a potentially useful clinical decision aid could be based on the NEWS, MR-proADM level and clinical features.

### Future research and development

As a growing number of NHS hospitals are implementing the NEWS on their clinical information systems, it should be practical to develop a decision aid based on admission NEWS, MR-proADM level and clinical features. Other biomarkers may further improve prognostic accuracy for deterioration, for example, lactate,3 peroxiredoxin-4 and copeptin,22 35 36 and soluble urokinase plasminogen activator receptor.37 The feasibility, cost-effectiveness and acceptability of such decision aids needs to be evaluated in further research.

A rapid point-of-care test for MR-proADM could facilitate the assessment process and reduce delays in arranging optimal levels of care and intensity of monitoring. Future research could identify the threshold MR-proADM level corresponding to the optimal combination of sensitivity and specificity for a binary test (eg, ‘present’ or ‘absent’) for deterioration.

## Acknowledgments

The authors would like to thank the patients who accepted to be part of the study and the team of research nurses for the data collection: Laura Shewan, Louise Taylor, Janine Gradwell, Karen Martin, Katherine Cullen, Gerry Jones, Graham Soulsby, Carmen Scott and Helen Reed. We also thank the reviewers for their helpful suggestions and comments.

## Footnotes

*   Contributors AJS, DAP, MP and DDS devised the study. RAO, SG and AJA managed the project. SG performed the statistical analyses with advice from DS. All authors contributed to the final manuscript.

*   Funding The study was funded by B·R·A·H·M·S GmbH. The NIHR funds the NIHR Newcastle In Vitro Diagnostics Co-operative.

*   Disclaimer The views expressed are those of the authors and not necessarily those of B·R·A·H·M·S GmbH, the NHS, the NIHR or the Department of Health.

*   Competing interests None declared.

*   Patient consent Obtained.

*   Ethics approval The study was approved by the Newcastle and North Tyneside Research Ethics Committee (15/NE/0120), and by the R&D Committee of the Newcastle upon Tyne Hospitals NHS Foundation Trust (reference no. 7495).

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement No additional data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  1.Royal College of Physicians of London. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS—report of a working party. London: Royal College of Physicians of London, 2012.
    
    

2.  2. McGinley A , Pearse RM . A national early warning score for acutely ill patients. BMJ 2012;345:e5310.[doi:10.1136/bmj.e5310](http://dx.doi.org/10.1136/bmj.e5310) 
    
    

3.  3. Jo S , Yoon J , Lee JB , et al . Predictive value of the National Early Warning Score-Lactate for mortality and the need for critical care among general emergency department patients. J Crit Care 2016;36:60–8.[doi:10.1016/j.jcrc.2016.06.016](http://dx.doi.org/10.1016/j.jcrc.2016.06.016) 
    
    

4.  4. Smith GB , Prytherch DR , Meredith P , et al . The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013;84:465–70.[doi:10.1016/j.resuscitation.2012.12.016](http://dx.doi.org/10.1016/j.resuscitation.2012.12.016) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.resuscitation.2012.12.016&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23295778&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

5.  5. Smith ME , Chiovaro JC , O’Neil M , et al . Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc 2014;11:1454–65.[doi:10.1513/AnnalsATS.201403-102OC](http://dx.doi.org/10.1513/AnnalsATS.201403-102OC) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1513/AnnalsATS.201403-102OC&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25296111&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

6.  6. Teasdale GM . National Early Warning Score (NEWS) is not suitable for all patients. BMJ 2012;345:e587:e5875.[doi:10.1136/bmj.e5875](http://dx.doi.org/10.1136/bmj.e5875) 
    
    

7.  7. Eccles SR , Subbe C , Hancock D , et al . CREWS: improving specificity whilst maintaining sensitivity of the National Early Warning Score in patients with chronic hypoxaemia. Resuscitation 2014;85:109–11.[doi:10.1016/j.resuscitation.2013.08.277](http://dx.doi.org/10.1016/j.resuscitation.2013.08.277) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.resuscitation.2013.08.277&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24056390&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

8.  8. Badriyah T , Briggs JS , Meredith P , et al . Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS). Resuscitation 2014;85:418–23.[doi:10.1016/j.resuscitation.2013.12.011](http://dx.doi.org/10.1016/j.resuscitation.2013.12.011) 
    
    

9.  9. Abbott TE , Vaid N , Ip D , et al . A single-centre observational cohort study of admission National Early Warning Score (NEWS). Resuscitation 2015;92:89–93.[doi:10.1016/j.resuscitation.2015.04.020](http://dx.doi.org/10.1016/j.resuscitation.2015.04.020) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.resuscitation.2015.04.020&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25943011&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

10. 10. Corfield AR , Lees F , Zealley I , et al . Utility of a single early warning score in patients with sepsis in the emergency department. Emerg Med J 2014;31:482–7.[doi:10.1136/emermed-2012-202186](http://dx.doi.org/10.1136/emermed-2012-202186) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzMS82LzQ4MiI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi84LzExL2UwMjAzMzcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. 11. Bilben B , Grandal L , Søvik S . National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient—a prospective observational study. Scand J Trauma Resusc Emerg Med 2016;24:80.[doi:10.1186/s13049-016-0273-9](http://dx.doi.org/10.1186/s13049-016-0273-9) 
    
    

12. 12. Valenzuela-Sánchez F , Valenzuela-Méndez B , Rodríguez-Gutiérrez JF , et al . New role of biomarkers: mid-regional pro-adrenomedullin, the biomarker of organ failure. Ann Transl Med 2016;4:329.[doi:10.21037/atm.2016.08.65](http://dx.doi.org/10.21037/atm.2016.08.65) 
    
    

13. 13. Stokes NR , Dietz BW , Liang JJ . Cardiopulmonary laboratory biomarkers in the evaluation of acute dyspnea. Open Access Emerg Med 2016;8:35–45.[doi:10.2147/OAEM.S71446](http://dx.doi.org/10.2147/OAEM.S71446) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2147/OAEM.S71446&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=27307771&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

14. 14. Sinning C , Zengin E , Zeller T , et al . Candidate biomarkers in heart failure with reduced and preserved ejection fraction. Biomarkers 2015;20:258–65.[doi:10.3109/1354750X.2015.1068856](http://dx.doi.org/10.3109/1354750X.2015.1068856) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.3109/1354750X.2015.1068856&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=26301884&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

15. 15. Kutz A , Hausfater P , Amin D , et al . The TRIAGE-ProADM Score for an early risk stratification of medical patients in the emergency department—development based on a multi-national, prospective, observational study. PLoS One 2016;11:e0168076.[doi:10.1371/journal.pone.0168076](http://dx.doi.org/10.1371/journal.pone.0168076) 
    
    

16. 16. Albrich WC , Dusemund F , Rüegger K , et al . Enhancement of CURB65 score with proadrenomedullin (CURB65-A) for outcome prediction in lower respiratory tract infections: derivation of a clinical algorithm. BMC Infect Dis 2011;11:112.[doi:10.1186/1471-2334-11-112](http://dx.doi.org/10.1186/1471-2334-11-112) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2334-11-112&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21539743&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

17. 17. López J , Martínez A . Cell and molecular biology of the multifunctional peptide, adrenomedullin. Int Rev Cytol 2002;221:1–92.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0074-7696(02)21010-4&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12455746&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000179786500001&link_type=ISI) 

18. 18. Serrano-Ponz M , Rodrigo-Gasqué C , Siles E , et al . Temporal profiles of blood pressure, circulating nitric oxide, and adrenomedullin as predictors of clinical outcome in acute ischemic stroke patients. Mol Med Rep 2016;13:3724–34.[doi:10.3892/mmr.2016.5001](http://dx.doi.org/10.3892/mmr.2016.5001) 
    
    

19. 19. Zudaire E , Portal-Núñez S , Cuttitta F . The central role of adrenomedullin in host defense. J Leukoc Biol 2006;80:237–44.[doi:10.1189/jlb.0206123](http://dx.doi.org/10.1189/jlb.0206123) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1189/jlb.0206123&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16769769&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000243015600004&link_type=ISI) 

20. 20. Zuur-Telgen MC , Brusse-Keizer MG , VanderValk PD , et al . Stable-state midrange-proadrenomedullin level is a strong predictor of mortality in patients with COPD. Chest 2014;145:534–41.[doi:10.1378/chest.13-1063](http://dx.doi.org/10.1378/chest.13-1063) 
    
    

21. 21. Pedowska-Włoszek J , Kostrubiec M , Kurnicka K , et al . Midregional proadrenomedullin (MR-proADM) in the risk stratification of patients with acute pulmonary embolism. Thromb Res 2013;132:506–10.[doi:10.1016/j.thromres.2013.08.008](http://dx.doi.org/10.1016/j.thromres.2013.08.008) 
    
    

22. 22. Nickel CH , Messmer AS , Geigy N , et al . Stress markers predict mortality in patients with nonspecific complaints presenting to the emergency department and may be a useful risk stratification tool to support disposition planning. Acad Emerg Med 2013;20:670–9.[doi:10.1111/acem.12172](http://dx.doi.org/10.1111/acem.12172) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/acem.12172&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23859580&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

23. 23. Landman GW , van Dijk PR , Drion I , et al . Midregional fragment of proadrenomedullin, new-onset albuminuria, and cardiovascular and all-cause mortality in patients with type 2 diabetes (ZODIAC-30). Diabetes Care 2014;37:839–45.[doi:10.2337/dc13-1852](http://dx.doi.org/10.2337/dc13-1852) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo4OiIzNy8zLzgzOSI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi84LzExL2UwMjAzMzcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. 24. Funke-Kaiser A , Mann K , Colquhoun D , et al . Midregional proadrenomedullin and its change predicts recurrent major coronary events and heart failure in stable coronary heart disease patients: the LIPID study. Int J Cardiol 2014;172:411–8.[doi:10.1016/j.ijcard.2014.01.064](http://dx.doi.org/10.1016/j.ijcard.2014.01.064) 
    
    

25. 25. Gaggin HK , Januzzi JL . Biomarkers and diagnostics in heart failure. Biochim Biophys Acta 2013;1832:2442–50.[doi:10.1016/j.bbadis.2012.12.014](http://dx.doi.org/10.1016/j.bbadis.2012.12.014) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.bbadis.2012.12.014&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23313577&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

26. 26. Vigué B , Leblanc PE , Moati F , et al . Mid-regional pro-adrenomedullin (MR-proADM), a marker of positive fluid balance in critically ill patients: results of the ENVOL study. Crit Care 2016;20:363.[doi:10.1186/s13054-016-1540-x](http://dx.doi.org/10.1186/s13054-016-1540-x) 
    
    

27. 27. Tzikas S , Keller T , Ojeda FM , et al . MR-proANP and MR-proADM for risk stratification of patients with acute chest pain. Heart 2013;99:388–95.[doi:10.1136/heartjnl-2012-302956](http://dx.doi.org/10.1136/heartjnl-2012-302956) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiaGVhcnRqbmwiO3M6NToicmVzaWQiO3M6ODoiOTkvNi8zODgiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vOC8xMS9lMDIwMzM3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

28. 28. Peduzzi P , Concato J , Kemper E , et al . A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996;49:1373–9.[doi:10.1016/S0895-4356(96)00236-3](http://dx.doi.org/10.1016/S0895-4356(96)00236-3) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0895-4356(96)00236-3&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8970487&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1996VY92400008&link_type=ISI) 

29. 29. van Smeden M , de Groot JA , Moons KG , et al . No rationale for 1 variable per 10 events criterion for binary logistic regression analysis. BMC Med Res Methodol 2016;16:163.[doi:10.1186/s12874-016-0267-3](http://dx.doi.org/10.1186/s12874-016-0267-3) 
    
    

30. 30. Travaglino F , Russo V , De Berardinis B , et al . Thirty and ninety days mortality predictive value of admission and in-hospital procalcitonin and mid-regional pro-adrenomedullin testing in patients with dyspnea. Results from the VERyfing DYspnea trial. Am J Emerg Med 2014;32:334–41.[doi:10.1016/j.ajem.2013.12.045](http://dx.doi.org/10.1016/j.ajem.2013.12.045) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.ajem.2013.12.045&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24559907&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

31. 31.R: A language and environment for statistical computing. R development core team. Vienna, Austria.: R Foundation for Statistical Computing, 2011.
    
    

32. 32. Pencina MJ , D’Agostino RB , D’Agostino RB , et al . Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157–72.[doi:10.1002/sim.2929](http://dx.doi.org/10.1002/sim.2929) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.2929&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17569110&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000253098900001&link_type=ISI) 

33. 33. Steyerberg EW , Vedder MM , Leening MJ , et al . Graphical assessment of incremental value of novel markers in prediction models: from statistical to decision analytical perspectives. Biom J 2015;57:556–70.[doi:10.1002/bimj.201300260](http://dx.doi.org/10.1002/bimj.201300260) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/bimj.201300260&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25042996&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 

34. 34. Alam N , Vegting IL , Houben E , et al . Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department. Resuscitation 2015;90:111–5.[doi:10.1016/j.resuscitation.2015.02.011](http://dx.doi.org/10.1016/j.resuscitation.2015.02.011) 
    
    

35. 35. Nickel CH , Ruedinger J , Misch F , et al . Copeptin and peroxiredoxin-4 independently predict mortality in patients with nonspecific complaints presenting to the emergency department. Acad Emerg Med 2011;18:851–9.[doi:10.1111/j.1553-2712.2011.01126.x](http://dx.doi.org/10.1111/j.1553-2712.2011.01126.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1553-2712.2011.01126.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21843221&link_type=MED&atom=%2Fbmjopen%2F8%2F11%2Fe020337.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000293860400010&link_type=ISI) 

36. 36. Iversen K , Gøtze JP , Dalsgaard M , et al . Risk stratification in emergency patients by copeptin. BMC Med 2014;12:80.[doi:10.1186/1741-7015-12-80](http://dx.doi.org/10.1186/1741-7015-12-80) 
    
    

37. 37. Rasmussen LJ , Ladelund S , Haupt TH , et al . Soluble urokinase plasminogen activator receptor (suPAR) in acute care: a strong marker of disease presence and severity, readmission and mortality. A retrospective cohort study. Emerg Med J 2016;33:769–75.[doi:10.1136/emermed-2015-205444](http://dx.doi.org/10.1136/emermed-2015-205444) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo5OiIzMy8xMS83NjkiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vOC8xMS9lMDIwMzM3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)