What do we know about methods for predicting intensive care unit length of stay?

Most studies that evaluated models for ICU LOS prediction^(4-8) used multivariate linear regression and did not test other approaches to compare their accuracy. Linear regression has the advantage of clearer interpretation; however, the assumption of a linear relationship with the covariates is its major limitation. Verburg et al.⁽⁹⁾ performed a systematic review in 2017 and concluded that the models developed by these studies did not satisfy general requirements for the prediction of ICU LOS, either to plan resource allocation or to identify individual patient LOS.

We searched the literature to find studies that compared different prediction models for LOS prediction. From inception to October 6, 2020, we searched the MEDLINE, Embase and Scopus databases. The search was limited to the English language and the publication types “article”, “article in press”, and “review”. The search comprised the “title” and “keywords” fields, and no restriction was made for the publication period. We used the following queries: (“ICU” or “Intensive Care”) and (“length of stay”) and (“predict*”). The study selection was fourfold: (i) formulating eligibility criteria; (ii) abstract reading and selection for full-text reading; (iii) full-text reading and selection; and (iv) including new studies by backward and forward search. We considered the following eligibility criteria for study inclusion: studies that included and compared models for ICU LOS prediction, reporting statistics in terms of root mean square error (RMSE), mean absolute error (MAE) or R². Inference studies were not included. We found five prediction studies, and the characteristics of each one are summarized in table 1.

Table 1 - Characteristics of each prediction study

Verburg et al.⁽¹⁰⁾ compared six regression models to predict the ICU LOS for a dataset of 32,667 ICU admissions. The best models were the generalized linear model (GLM) with a Gaussian distribution and the GLM with a Poisson distribution, and the worst model was the Cox regression. The study tested the ICU LOS log transformation to reduce the skewness and improve the underlying variable distribution symmetry, which presented better results. The authors also tested the truncation at 30 days, which improved the model performance.

Moran et al.⁽⁴⁾ compared seven regression models to predict the ICU LOS for a dataset of 111,663 ICU admissions. The best was the linear mixed model (LMM). The authors also tested the log transformation in the ICU LOS, which presented better outputs.

Houthooft et al.⁽¹¹⁾ compared different data-driven models to predict the ICU LOS for patients remaining in the ICU on day 5. The best performing model was support vector regression (SVR), and the worst was artificial neural network (ANN). The authors included the log transformation of ICU LOS, feature normalization, and feature selection (using the random forest importance list and a backward elimination procedure with SVR) in the preprocessing methodology.

Li et al.⁽¹²⁾ created a predictive model using preprocessing techniques, exploratory data analysis, and the least absolute shrinkage and selection operator (LASSO) algorithm. In the preprocessing methodology, the authors included the treatment for missing data, the Box-Cox transformation (for ICU LOS and variables with skewness coefficients greater than 0.5), and Z-score normalization. In the exploratory data analysis step, the study explored new features and the collinearity between existing features.

We also found two prediction articles that compared data-driven models for hospital LOS and presented significant results. Muhlestein et al.⁽¹³⁾ developed a novel method to systematically rank, select, and combine different data-driven algorithms, building a model that predicts LOS following craniotomy for brain tumors. The top-performing algorithms were the gradient boosted tree (GBT) and SVR. These models were combined with an elastic net to create an ensemble model. The preprocessing methodology included the treatment for missing data and Z-score normalization. Caetano et al.⁽¹⁴⁾ used a data-driven method to predict the hospital LOS for a dataset of 26,431 admissions. The best model was random forest (RF), and the worst models were ordinary least square (OLS) and decision tree (DT). The methodology considered a preprocessing strategy, including k-nearest neighborhood (k-NN) imputation to deal with missing values and Z-score normalization to put the numeric values on the same scale. Moreover, a log transformation was applied to the covariate “previous LOS” and the outcome variable “LOS.”

The most common performance metric used to compare prediction models is the RMSE, followed by the MAE and the coefficient of determination (R²). From table 1, we can note that the studies with the best performance were those of Caetano et al.,⁽¹⁴⁾ Muhlestein et al.,⁽¹³⁾ and Li et al.⁽¹²⁾ The achieved results may be explained by the development of a structured data-driven methodology. These studies were included in the preprocessing step, the treatment for missing data, the log (or Box-Cox) transformation for ICU LOS, and Z-score normalization. Moreover, their methodology included splitting the dataset into training and testing cohorts and using a cross-validation step to analyze the model overfitting. Li et al.⁽¹²⁾ also explored new features and analyzed the collinearity between existing features.

Regarding the type of models tested in each study, we can separate them into statistical and data-driven models, as presented in table 2. We note that SVR, a state-of-art data-driven model, overcame the other models in two studies. Other models that presented good results were GBT, RF, GLM, LMM, and LASSO. Therefore, we suggest that future studies consider the following steps to achieve a reasonable prediction for ICU LOS: data extraction and feature engineering; treatment of missing data and outliers; data splitting into training and testing; data preprocessing, including collinearity analysis, feature selection, transformations to resolve skewness and normalization; cross-validation to analyze overfitting; and training, testing and comparing different types of models, including data-driven models. Moreover, future studies should report their results in terms of prediction error (RMSE and MAE), which can help researchers to make conclusions about the best models and make novel recommendations.

Table 2 - Models included in each prediction study

Regarding the distribution of ICU LOS, most authors tested the log transformation to reduce the distribution skewness. The truncation of ICU LOS data is a common measure to avoid extreme values. Therefore, truncation at high percentiles (95% or 99%) is an alternative to identify outliers. However, truncation can be unfair because there may be substantial differences in the truncated values, and the largest improvements in efficiency may be achieved in patients with the longest ICU LOS. Therefore, we recommend being careful when comparing models using truncated data with models using original data.

Clearly, no single model should be used in all situations. The best result will depend on each dataset, and the models should be trained specifically for each case. Moreover, it is crucial to extract the relevant covariates from the ICU database. Studies have demonstrated that there is a nonlinear relation between ICU LOS and patient severity. In other words, more severe patients tend to have a longer LOS. However, the sickest patients are also those at higher risk of death, which may decrease the expected ICU LOS. Therefore, it is important to include features related to patient severity. Peres et al.⁽¹⁵⁾ suggested a list of risk factors for ICU LOS that should be included in prediction models (e.g., comorbidities, invasive interventions, laboratory markers, and main reasons for ICU admission). The data-driven models will be able to understand this nonlinear relationship if relevant features are included in the analysis. Including irrelevant variables can increase the dimensionality of the problem, which may disturb the model results. On the other hand, excluding relevant features in advance may generate suboptimal results. Therefore, the extraction of variables from the dataset should be done with caution.

Our study has some limitations. First, the work of Houthooft et al.⁽¹¹⁾ and Li et al.⁽¹²⁾ analyzed restricted ICU populations. Houthooft et al.⁽¹¹⁾ examined a cohort including only medical patients, while Li et al.⁽¹²⁾ investigated a single ICU cohort. Second, we included two articles that focused their analysis on hospital LOS prediction instead of ICU LOS. The distribution of hospital LOS may be similar to that of ICU LOS; however, some assumptions may be different from each other. Third, one article⁽¹¹⁾ made the ICU LOS prediction on day 5 instead of at admission. The distribution of ICU LOS after day 5 was not the same compared to the original LOS, which may affect the comparison analysis.