Advertisement

Parsimonious machine learning models to predict resource use in cardiac surgery across a statewide collaborative

Open AccessPublished:April 20, 2022DOI:https://doi.org/10.1016/j.xjon.2022.04.017

      Abstract

      Objective

      We sought to several develop parsimonious machine learning models to predict resource utilization and clinical outcomes following cardiac operations using only preoperative factors.

      Methods

      All patients undergoing coronary artery bypass grafting and/or valve operations were identified in the 2015-2021 University of California Cardiac Surgery Consortium repository. The primary end point of the study was length of stay (LOS). Secondary endpoints included 30-day mortality, acute kidney injury, reoperation, postoperative blood transfusion and duration of intensive care unit admission (ICU LOS). Linear regression, gradient boosted machines, random forest, extreme gradient boosting predictive models were developed. The coefficient of determination and area under the receiver operating characteristic (AUC) were used to compare models. Important predictors of increased resource use were identified using SHapley summary plots.

      Results

      Compared with all other modeling strategies, gradient boosted machines demonstrated the greatest performance in the prediction of LOS (coefficient of determination, 0.42), ICU LOS (coefficient of determination, 0.23) and 30-day mortality (AUC, 0.69). Advancing age, reduced hematocrit, and multiple-valve procedures were associated with increased LOS and ICU LOS. Furthermore, the gradient boosted machine model best predicted acute kidney injury (AUC, 0.76), whereas random forest exhibited greatest discrimination in the prediction of postoperative transfusion (AUC, 0.73). We observed no difference in performance between modeling strategies for reoperation (AUC, 0.80).

      Conclusions

      Our findings affirm the utility of machine learning in the estimation of resource use and clinical outcomes following cardiac operations. We identified several risk factors associated with increased resource use, which may be used to guide case scheduling in times of limited hospital capacity.

      Graphical abstract

      Key Words

      Abbreviations and Acronyms:

      AKI (acute kidney injury), AUC (area under the receiver operating characteristic), CABG (coronary artery bypass grafting), GBM (gradient boosted machine), ICU (intensive care unit), LOS (length of stay), ML (machine learning), RF (random forest), STS (Society of Thoracic Surgeons), UCCSC (University of California Cardiac Surgery Consortium), XGBoost (extreme gradient boosting)
      Figure thumbnail fx2
      Observed length of stay versus predictions by machine learning model.
      Compared to traditional linear regression, machine learning yielded superior performance in the prediction of length of stay, mortality, acute kidney injury, and reoperation following cardiac operations.
      This study outlined the development of machine learning (ML) models to predict length of stay (LOS) following cardiac operations. Several clinical, operation-related, and hospital characteristics were found to be associated with increased LOS. Taken together, our findings suggest that ML models may be used to inform case scheduling strategies during times of limited hospital capacity.
      The novel COVID-19 pandemic has placed unprecedented strain on health care systems, influencing the allocation of personnel and resources. Several groups have reported cardiac surgery case volume reductions of 45% to 94%, with significant regional variability.
      • Gaudino M.
      • Chikwe J.
      • Hameed I.
      • Robinson N.B.
      • Fremes S.E.
      • Ruel M.
      Response of cardiac surgery units to COVID-19.
      • Farrington W.J.
      • Robinson N.B.
      • Rahouma M.
      • Lau C.
      • Hameed I.
      • Iannacone E.M.
      • et al.
      Cardiac surgery outcomes in an epicenter of the COVID-19 pandemic.
      • Ad N.
      • Luc J.G.Y.
      • Nguyen T.C.
      COVID-19 North American Cardiac Surgery Survey Working Group
      Cardiac surgery in North America and coronavirus disease 2019 (COVID-19): regional variability in burden and impact.
      • George I.
      • Salna M.
      • Kobsa S.
      • Deroo S.
      • Kriegel J.
      • Blitzer D.
      • et al.
      The rapid transformation of cardiac surgery practice in the coronavirus disease 2019 (COVID-19) pandemic: insights and clinical strategies from a centre at the epicentre.
      Subject to rates of “reopening” and patients’ desire to proceed with elective surgery, the projected time to equilibrium between back-logged cases and ongoing surgical need is estimated to be 12 to 22 months.
      • Bose S.K.
      • Dasani S.
      • Roberts S.E.
      • Chris W.
      • DeMatteo R.P.
      • Doherty G.M.
      • et al.
      The cost of quarantine: projecting the financial impact of canceled elective surgery on the nation’s hospitals.
      • Salenger R.
      • Etchill E.W.
      • Ad N.
      • Matthew T.
      • Alejo D.
      • Whitman G.
      • et al.
      The surge after the surge: cardiac surgery post–COVID-19.
      • Engelman D.T.
      • Lother S.
      • George I.
      • Ailawadi G.
      • Atluri P.
      • Grant M.C.
      • et al.
      Ramping up delivery of cardiac surgery during the COVID-19 pandemic: a guidance statement from the Society of Thoracic Surgeons COVID-19 Task Force.
      Furthermore, recovery from cessation of elective cases requires a nuanced approach to management of deferred and newly presenting patients as well as ongoing demands for perioperative resources. With estimates that operating volume must exceed 120% of baseline to accommodate deferred patients while concurrently preventing excess waitlist morbidity, rapid and accurate prediction of hospital bed occupancy and resource utilization are especially crucial.
      • Salenger R.
      • Etchill E.W.
      • Ad N.
      • Matthew T.
      • Alejo D.
      • Whitman G.
      • et al.
      The surge after the surge: cardiac surgery post–COVID-19.
      The Society of Thoracic Surgeons (STS), among others, has successfully implemented risk models to provide canonical estimates for parameters such as mortality, postoperative complications, and prolonged length of stay (LOS).
      • O’Brien S.M.
      • Feng L.
      • He X.
      • Xian Y.
      • Jacobs J.P.
      • Badhwar V.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
      However, as demonstrated by several reports of poor calibration when applied at the institutional level, these predictive tools are often cumbersome and require numerous data fields to yield a predicted risk without accounting for local variations in clinical practice.
      • Chan V.
      • Ahrari A.
      • Ruel M.
      • Elmistekawy E.
      • Hynes M.
      • Mesana T.G.
      Perioperative deaths after mitral valve operations may be overestimated by contemporary risk models.
      • Kirmani B.H.
      • Mazhar K.
      • Saleh H.Z.
      • Ward A.N.
      • Shaw M.
      • Fabri B.
      • et al.
      External validity of the Society of Thoracic Surgeons risk stratification tool for deep sternal wound infection after cardiac surgery in a UK population.
      • Sharkawi M.A.
      • Shah P.B.
      • Zenati M.
      • Kaneko T.
      • Ramadan R.
      Underclassification of predicted risk of mortality using the latest Society of Thoracic Surgeons Risk Models.
      Furthermore, most available models predict prolonged LOS in a binary manner, rather than an estimate of the actual duration of hospitalization in days.
      • O’Brien S.M.
      • Feng L.
      • He X.
      • Xian Y.
      • Jacobs J.P.
      • Badhwar V.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
      ,
      • Daghistani T.A.
      • Elshawi R.
      • Sakr S.
      • Ahmed A.M.
      • Al-Thwayee A.
      • Al-Mallah M.H.
      Predictors of in-hospital length of stay among cardiac patients: a machine learning approach.
      • Alshakhs F.
      • Alharthi H.
      • Aslam N.
      • Khan I.U.
      • Elasheri M.
      Predicting postoperative length of stay for isolated coronary artery bypass graft patients using machine learning.
      • Shahian D.M.
      • Jacobs J.P.
      • Badhwar V.
      • Kurlansky P.A.
      • Furnary A.P.
      • Cleveland Jr., J.C.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 1—background, design considerations, and model development.
      The classification of LOS into prolonged and routine reduces generalizability and limits the application of such tools in acute care settings.
      • Messaoudi N.
      • De Cocker J.
      • Stockman B.
      • Bossaert L.L.
      • Rodrigus I.E.R.
      Prediction of prolonged length of stay in the intensive care unit after cardiac surgery: the need for a multi-institutional risk scoring system.
      Machine learning (ML) algorithms allow for complex modeling of nonlinear relationships between predictive factors and have demonstrated superior discrimination and calibration in several clinical applications.
      • Kilic A.
      • Goyal A.
      • Miller J.K.
      • Gjekmarkaj E.
      • Tam W.L.
      • Gleason T.G.
      • et al.
      Predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery.
      • Tseng P.-Y.
      • Chen Y.-T.
      • Wang C.-H.
      • Chiu K.-M.
      • Peng Y.-S.
      • Hsu S.-P.
      • et al.
      Prediction of the development of acute kidney injury following cardiac surgery by machine learning.
      • Lee H.-C.
      • Yoon H.-K.
      • Nam K.
      • Cho Y.J.
      • Kim T.K.
      • Kim W.H.
      • et al.
      Derivation and validation of machine learning approaches to predict acute kidney injury after cardiac surgery.
      Therefore, we sought to develop ML-based models to predict LOS, 30-day mortality and select complications using an academic, statewide database. We hypothesized that a parsimonious ML model, containing few explanatory covariates, would yield superior discrimination and calibration compared with traditional linear and logistic regression.

      Methods

      Study Population

      All adults (aged 18 years or older) who underwent coronary artery bypass grafting (CABG) and/or valve operations were identified from the 2015 to 2021 University of California Cardiac Surgery Consortium (UCCSC) repository. Founded in 2013, the UCCSC is a collaborative among 5 academic hospitals across California. Data elements, including those submitted to the STS, are prospectively collected in compliance with policies of individual institutions and the University of California Systemwide Review Board (IRB No. 16-000558, approved May 6, 2016, renewed April 15, 2020). The need for patient written consent for the publication of the study data was waived by the institutional review board due to the de-identified nature of the UCCSC.
      Patients were stratified by the class of operation performed: isolated CABG, isolated valve, concomitant CABG/valve and multivalve operations. Those who required left ventricular assist device implantation, extracorporeal membrane oxygenation, or transcatheter procedures were excluded to maintain cohort homogeneity. Moreover, records with missing values for overall and intensive care unit (ICU) LOS as well as 30-day mortality were excluded (Figure E1). Patients with LOS or ICU LOS >95th percentile (>30 days for LOS, >259 hours in ICU) were similarly excluded.

      Variable and Outcome Definitions

      The primary end point was overall LOS. Mortality at 30 days, acute kidney injury (AKI), postoperative blood transfusion, reoperation and ICU LOS were also considered. Patient comorbidities, operative characteristics, and complications including AKI, postoperative blood transfusion and reoperation, were defined in accordance with the STS Adult Cardiac Database dictionary.
      Society of Thoracic Surgeons
      Adult cardiac surgery database data collection.
      Annual operative caseload, number of adult cardiac surgeons, total number of low acuity and cardiothoracic ICU beds were tabulated for each institution. Variables with missing values in >20% of patients were not considered for inclusion. For retained features with missing data, values were imputed using the median and mode for continuous and categorical variables, respectively. The number of records with missing data for each variable is reported in Table E1.

      Modeling Techniques

      We compared 3 ML algorithms to traditional, multivariable linear, and logistic regression: gradient boosted machines (GBM), extreme gradient boosting (XGBoost) and random forest (RF). These algorithms autonomously generate a large set of decision trees to capture nuanced patterns in training data. In the case of RF, the development of every decision tree occurs independently, and the final output of the model is the arithmetic mean of the output from each decision tree. However, the XGBoost and GBM algorithms train decision trees in a stepwise manner to compensate for errors of the prior trees, and the output is the weighted average of each decision tree's estimate.
      • Chen T.
      • Guestrin C.
      XGBoost: a scalable tree boosting system.
      A brief schematic highlighting the differences between boosting (XGBoost and GBM) and bagging (RF) classifiers is shown in Figure E2. Hyperparameters, which are used to control the learning process of ML models, were selected using the GridSearchCV function in the Python sklearn library (Python Software Foundation). This technique exhaustively evaluates a wide range of hyperparameters and selects values that optimize model performance. Selected hyperparameters for each model are shown in Table E2.

      Model Development

      Thirty-seven preoperative patient and hospital characteristics were chosen as candidate predictors. Clinical variables were selected from the STS risk score variable list based on clinical relevance and are listed in Table 1.
      • O’Brien S.M.
      • Feng L.
      • He X.
      • Xian Y.
      • Jacobs J.P.
      • Badhwar V.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
      Hospital factors were incorporated to account for variation in practice across participating institutions. Variable selection was performed using recursive feature elimination, a ML technique that is used to reduce collinearity and eliminate covariates with low variance. In recursive feature elimination, cross-validation is used to exhaustively evaluate variable sets of different sizes and select the best collection of features. Given that transportability and ease of use is an important aspect of risk tools, we identified the smallest set of variables that retained maximum predictive performance. This algorithm was independently applied using linear regression and GBM to ascertain any differences between modeling strategies. Selected variables were used for all subsequent model development (Table E3). We also compared the performance of ML against the STS risk scores for 30-day mortality, AKI, and reoperation.
      Table 1Baseline patient characteristics of the study cohort
      ParameterOverall (n = 6316)Derivation (n = 5028)Validation (n = 1288)P value
      Age (y)63 ± 1363 ± 1364 ± 13<.001
      Elective admission (%)58.558.359.3.52
      Female (%)27.527.726.6.45
      Height (cm)171 ± 11171 ± 11171 ± 10.29
      Weight (kg)82 ± 1982 ± 1981 ± 20.57
      Ethnicity (%)19.719.919.3.68
      Operative type (%)
       Isolated CABG50.551.347.4.012
       Isolated valve operation33.331.341.2<.001
       CABG + valve10.611.18.3.003
       Multiple valve5.86.33.6<.001
      Medical conditions (%)
       Atrial fibrillation17.617.617.7.91
       Cancer6.97.16.4.37
       Cerebrovascular disease17.017.216.2.38
       Severe lung disease3.33.23.9.23
       Congestive heart failure36.133.845.0<.001
       Diabetes38.137.639.7.18
       Home oxygen3.03.12.8.59
       Hypertension77.477.278.0.57
       Infectious endocarditis5.95.96.1.84
       Liver disease6.46.85.0.017
       Peripheral vascular disease9.08.511.1.003
       Prior myocardial infarction31.231.729.3.09
      Laboratory values
       Hematocrit (% blood volume)39 ± 639 ± 639 ± 6.01
       International normalized ratio1.13 ± 0.31.13 ± 0.31.12 ± 0.2.26
       Serum albumin (g/dL)3.9 ± 0.63.9 ± 0.63.9 ± 0.6.008
       Preoperative creatinine (mg/dL)1.4 ± 1.71.4 ± 1.61.5 ± 1.9<.001
       Ejection fraction (%)56 ± 1256 ± 1257 ± 12.21
      Hospital of operation (%)
       Center 132.731.736.5.001
       Center 224.024.721.4.014
       Center 319.119.118.9.84
       Center 414.014.910.5<.001
       Center 510.29.512.7<.001
      Values are presented as mean ± SD or n. CABG, Coronary artery bypass grafting.
      The derivation cohort consisted of operations performed before March 2020, whereas the remainder comprised the validation dataset. To obtain cross-validated performance metrics, models were fit using 50% of the derivation cohort and tested using the remainder. This process was repeated 100 times to acquire model performance metrics, which are reported as means with 95% CIs. To account for potential differences in case-mix due to the COVID-19 pandemic, we assessed the stability of model performance in the pre- (derivation) and post-COVID–19 (validation) eras.

      Model Evaluation and Interpretation

      Linear regression, GBM, RF, and XGBoost models were compared using the coefficient of determination (R2) between observed and predicted values. Binary classifiers were evaluated using the area under the receiver operating characteristic (AUC). The accuracy of probabilistic predictions was assessed using the Brier score, for which lower values denote superior calibration. Model R2 and Brier scores were analyzed using a paired t test, which allowed for comparison of model performance across cross-validation folds. Similarly, model AUCs were compared using DeLong's test, which specifically accounts for the influence of model evaluation on a common test set. SHapley additive values were calculated to estimate the marginal influence of each covariate on the output of a decision tree model.
      • Tseng P.-Y.
      • Chen Y.-T.
      • Wang C.-H.
      • Chiu K.-M.
      • Peng Y.-S.
      • Hsu S.-P.
      • et al.
      Prediction of the development of acute kidney injury following cardiac surgery by machine learning.
      Baseline characteristics are reported as means with SD or medians with interquartile range (IQR), as appropriate. Means were analyzed using the adjusted Wald test, whereas medians were analyzed with the Mann-Whitney U test. Categorical variables are reported as frequencies and were compared using the Pearson χ2 test. Statistical significance was set at α = 0.05. Statistical analysis was conducted using Stata 16.0 (StataCorp) and Python version 3.9. The sklearn, shap, and xgboost packages of Python were used to develop and assess ML models as described above.
      • Lundberg S.M.
      • Lee S.-I.
      A unified approach to interpreting model predictions.
      ,
      • Pedregosa F.
      • Varoquaux G.
      • Gramfort A.
      • Michel V.
      • Thirion B.
      • Grisel O.
      • et al.
      Scikit-learn: machine learning in Python.

      Results

      Population Characteristics

      Across 5 participating centers, 6,316 patients met study criteria. The study cohort was predominantly male (72.5%), with mean age 63 years. A significant proportion of patients had preexisting medical conditions such as diabetes, congestive heart failure, and atrial fibrillation (Table 1). The most frequent operation was isolated CABG (50.5%), followed by isolated valve (33.3%) and concomitant CABG/valve operations (10.6%). The majority of operations were performed electively. Over the study period, the highest volume center performed 1,205 operations, whereas the lowest volume center performed 626 operations. The 30-day mortality rate was 0.9%. Overall, 27.7% of patients received postoperative transfusions, and 1.5% developed AKI. Median LOS was 8 days (IQR, 6-13 days) with a median ICU LOS of 74 hours (IQR, 47-116 hours).
      Comparison of baseline characteristics and outcomes between the derivation and validation cohorts is shown in Tables 1 and 2. Patients in the validation cohort were marginally older (64 ± 13 vs 63 ± 13 years; P <.001) and had greater rates of congestive heart failure (45.0% vs 33.8%; P < .001) and peripheral vascular disease (11.1% vs 8.3%; P = .003). Valve operations were more frequent in the validation group, compared with derivation. Although rates of 30-day mortality and AKI were similar, the incidence of reoperation (6.9% vs 9.1%; P = .014) and postoperative blood transfusion (23.1% vs 28.8%; P < .001) was lower in the validation cohort. The distribution of LOS and ICU LOS was statistically different between the derivation and validation datasets (Table 2).
      Table 2Resource utilization and clinical outcomes stratified by derivation and validation cohorts
      OutcomeOverall (n = 6316)Derivation (n = 5028)Validation (n = 1288)P value
      Resource use
       Length of stay (d)8 (6-13)8 (6-13)8 (5-12).008
       ICU length of stay (h)74 (47-116)75 (47-117)68 (43-99)<.001
      Clinical end points
       Mortality0.91.00.7.39
       Acute kidney injury1.51.51.7.54
       Postoperative transfusion27.728.823.1<.001
       Reoperation8.69.16.9.014
      Values are presented as median (interquartile range) or %. ICU, Intensive care unit.

      Variable Selection

      Recursive feature elimination was applied to 37 candidate variables to determine the optimal covariate set in the prediction of overall LOS. Figure 1 demonstrates the cross-validated R2 versus the number of covariates included in each model. The GBM model outperformed linear regression, regardless of feature set size. Notably, after the inclusion of 23 features, no appreciable increase in performance was observed from the GBM or linear regression model. Thus, all models were developed using the 23 features that were most strongly associated with LOS (Table E3).
      Figure thumbnail gr1
      Figure 1Coefficient of determination (R2) versus covariate set size in the prediction of in-hospital length of stay. LR, Linear regression; GBM, gradient boosted machine.

      Resource Utilization

      Linear regression, GBM, RF, and XGBoost models were developed to predict in-hospital LOS. Compared to linear regression, the GBM model yielded a higher R2 (0.42 vs 0.41; P < .001, Table 3). As shown in Figure E3, predictions by the GBM model were more strongly correlated with observed values for LOS, compared with linear regression. Although the difference in cross-validated R2 between the 2 strategies was subtle, the GBM model greatly outperformed linear regression in the validation dataset (R2, 0.47 vs 0.42) (Table 4). When assessing cumulative model error in the validation cohort, the GBM model resulted in a 197-day reduction in error across all patients relative to linear regression.
      Table 3Performance of each algorithm when predicting resource utilization and clinical outcomes in the validation cohort
      OutcomeLinearLogisticGBMRFXGBoostSTS
      Resource use
      Regressions were evaluated using the coefficient of determination (R2).
       Length of stay0.420.470.470.47
       ICU length of stay0.0170.0780.0540.082
      Clinical end point
      Binary classifiers were assessed with the area under the receiver operating characteristic.
       Mortality0.680.680.70.720.91
       Acute kidney injury0.770.790.80.80.84
       Postoperative transfusion0.690.680.680.67
       Reoperation0.780.790.80.780.76
      GBM, Gradient boosted machine; RF, random forest; XGBoost, extreme gradient boosting; STS, Society of Thoracic Surgeons risk score; ICU, intensive care unit.
      Regressions were evaluated using the coefficient of determination (R2).
      Binary classifiers were assessed with the area under the receiver operating characteristic.
      Table 4Cross-validated model performance metrics for each algorithm and outcome
      OutcomeLinearLogisticGBMRFXGBoostSTSP valueP value
      Resource use
      Models with continuous output were evaluated using the coefficient of determination (R2).
       Length of stay0.41 (0.41-0.41)0.42 (0.42-0.42)0.41 (0.40-0.41)0.42 (0.42-0.42)<.001
       ICU length of stay0.15 (0.15-0.15)0.23 (0.23-0.23)0.21 (0.21-0.21)0.22 (0.22-0.22)<.001
      Clinical end point
      Binary classifiers were assessed with the area under the receiver operating characteristic.
       Mortality0.67 (0.67-0.68)0.69 (0.68-0.70)0.69 (0.68-0.70)0.69 (0.69-0.70)0.91 (0.91-0.92)<.001<.001
       Acute kidney injury0.67 (0.67-0.68)0.76 (0.75-0.77)0.76 (0.76-0.77)0.74 (0.73-0.75)0.84 (0.83-0.86)<.001<.001
       Postoperative transfusion0.71 (0.71-0.72)0.73 (0.73-0.73)0.71 (0.71-0.71)0.73 (0.73-0.74)<.001
       Reoperation0.81 (0.80-0.81)0.8 (0.79-0.80)0.80 (0.80-0.80)0.79 (0.79-0.80)0.76 (0.76-0.77).99<.001
      Values are presented as mean (95% CI). GBM, Gradient boosted machine; RF, random forest; XGBoost, extreme gradient boosting; STS, Society of Thoracic Surgeons risk score; ICU, intensive care unit.
      Models with continuous output were evaluated using the coefficient of determination (R2).
      Binary classifiers were assessed with the area under the receiver operating characteristic.
      The GBM model was interpreted using SHapley summary plots, and the most salient predictors of LOS were ranked by their relative importance (y-axis). Figure 2 depicts how high (red dot) and low (blue dot) feature values corresponded to a change in LOS prediction. Elective admission had the highest feature importance and was associated with significantly decreased LOS. In addition, we found decreased hematocrit and serum albumin levels to increase the estimated LOS. Certain procedures, such as concomitant CABG/valve and multivalve operations, were found to confer longer LOS. Notably, an increased number of floor beds conferred greater estimated LOS (Figure 2).
      Figure thumbnail gr2
      Figure 2Interpretation of gradient boosted machine (GBM)-based model for prediction of length of stay (LOS) (days) using SHapley summary plots. The y-axis is ordered by increasing feature importance, and the x-axis is the marginal effect of each parameter on predicted LOS. Red dots show the influence of high feature values on predicted LOS, whereas blue dots show the influence of low feature values. CABG, Coronary artery bypass grafting.
      In the prediction of ICU LOS, the GBM model demonstrated significantly increased cross-validated R2 compared with linear regression (0.23 vs 0.15; P < .001). However, in the validation dataset, the XGBoost model demonstrated the highest performance (Table 4). Decreased preoperative creatinine level, low ejection fraction, and preexisting congestive heart failure were associated with greater predicted ICU LOS. Notably, increased annual hospital volume and a higher number of low-acuity beds were associated with lower estimated ICU LOS (Figure 3).
      Figure thumbnail gr3
      Figure 3Interpretation of gradient boosted machine (GBM)-based model for prediction of intensive care unit length of stay (ICU LOS) (hours) using SHapley summary plots. The y-axis is ordered by increasing feature importance, and the x-axis is the marginal effect of each parameter on predicted ICU LOS. Red dots show the influence of high feature values on predicted ICU LOS, whereas blue dots show the influence of low feature values. CABG, Coronary artery bypass grafting.

      Clinical Outcomes

      The GBM, RF, and XGBoost models outperformed logistic regression in the prediction of 30-day mortality (AUC, 0.69 vs 0.67; P < .001). Furthermore, the GBM and RF models outperformed logistic regression and XGBoost in the prediction of AKI (Table 3). Whereas postoperative blood transfusion was best predicted by GBM and XGBoost, all modeling strategies displayed similar discrimination in the estimation of reoperation (Table 3). The STS risk score for 30-day mortality and AKI outperformed ML models. However, ML displayed greater discrimination than the STS model in the prediction of reoperation (Table 3). These comparisons were consistent when evaluating the Brier score for each model (Tables E4 and E5).

      Discussion

      Reliable estimation of hospitalization duration remains a challenge for surgeons and administrators alike. The present study developed several parsimonious ML models to develop a readily useful prediction instrument for LOS (Video 1). This work entails one of the largest applications of ML to discretely model LOS using a multicenter, academic dataset. Compared with linear and logistic regression, we found ML algorithms to exhibit higher performance for prognostication of LOS, 30-day mortality, AKI, postoperative transfusion and ICU LOS. Using autonomous techniques, we identified several key predictors of increased resource use, including existing comorbidities, decreased preoperative hematocrit and serum albumin levels. And finally, we noted a significant influence of hospital characteristics on ICU LOS, suggesting the need for incorporation of center-specific characteristics in predictive tools.
      Several clinical characteristics, including preoperative anemia, renal dysfunction, and operative complexity, were associated with increased overall and ICU LOS. These findings are expected because laboratory values such as hematocrit level, international normalized ratio, creatinine level, and albumin level are incorporated in virtually every clinical risk score calculator.
      • O’Brien S.M.
      • Feng L.
      • He X.
      • Xian Y.
      • Jacobs J.P.
      • Badhwar V.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
      Moreover, these clinical factors influence the development of postoperative complications, including pneumonia and AKI, which are drivers of hospital LOS and costs.
      • O’Brien S.M.
      • Feng L.
      • He X.
      • Xian Y.
      • Jacobs J.P.
      • Badhwar V.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
      ,
      • Daghistani T.A.
      • Elshawi R.
      • Sakr S.
      • Ahmed A.M.
      • Al-Thwayee A.
      • Al-Mallah M.H.
      Predictors of in-hospital length of stay among cardiac patients: a machine learning approach.
      ,
      • Tseng P.-Y.
      • Chen Y.-T.
      • Wang C.-H.
      • Chiu K.-M.
      • Peng Y.-S.
      • Hsu S.-P.
      • et al.
      Prediction of the development of acute kidney injury following cardiac surgery by machine learning.
      ,
      • Almashrafi A.
      • Elmontsri M.
      • Aylin P.
      Systematic review of factors influencing length of stay in ICU after adult cardiac surgery.
      SHapley interpretation revealed that more complex operations were associated with greater LOS. The relatively higher incidence of complications in the setting of complex cardiac surgery, such as pacemaker placement, need for blood transfusion, and a greater need for ICU-level care, may explain this observation. Taken together, our findings validate the utilization of ML methods to reduce bias, enhance external validity, and autonomously select features associated with increasing LOS. Furthermore, our results demonstrate that during times of limited hospital capacity, clinical characteristics such as organ dysfunction and operative complexity should be considered when predicting hospitalization duration.
      • Tzeng C.-W.D.
      • Teshome M.
      • Katz M.H.G.
      • Weinberg J.S.
      • Lai S.Y.
      • Antonoff M.B.
      • et al.
      Cancer surgery scheduling during and after the COVID-19 first wave: the MD Anderson Cancer Center experience.
      In addition to patient factors, we found certain hospital structural characteristics to influence ICU LOS. For example, increasing cardiac institutional volume and a greater number of low-acuity beds was associated with reduced ICU LOS. Several factors may contribute to this important finding. Higher institutional cardiac surgery volume may represent greater expertise, the presence of standardized care pathways, and more efficient hospital throughput for these cases. Moreover, greater availability of low-acuity beds may lead to less delay in transitioning out of the ICU when clinical milestones are met.
      • Messaoudi N.
      • De Cocker J.
      • Stockman B.
      • Bossaert L.L.
      • Rodrigus I.E.R.
      Prediction of prolonged length of stay in the intensive care unit after cardiac surgery: the need for a multi-institutional risk scoring system.
      Consistent with this notion, several prior studies have demonstrated wide variation in hospital practices that may influence LOS, such as expedited discharge after lung resection and CABG.
      • Tran Z.
      • Chervu N.
      • Williamson C.
      • Verma A.
      • Hadaya J.
      • Gandjian M.
      • et al.
      The impact of expedited discharge on 30-day readmission following lung resection: a national study.
      ,
      • Afflu D.K.
      • Seese L.
      • Sultan I.
      • Gleason T.
      • Wang Y.
      • Navid F.
      • et al.
      Very early discharge after coronary artery bypass grafting does not affect readmission or survival.
      A nationwide study of minimally invasive esophagectomy in the Netherlands demonstrated great heterogeneity in ICU LOS, pointing to differences in use of early extubation protocols and analgesic modalities as contributing factors.
      • Voeten D.M.
      • van der Werf L.R.
      • Gisbertz S.S.
      • Ruurda J.P.
      • van Berge Henegouwen M.I.
      • van Hillegersberg R.
      • et al.
      Postoperative intensive care unit stay after minimally invasive esophagectomy shows large hospital variation. Results from the Dutch Upper Gastrointestinal Cancer Audit.
      Investigation at a broader scale is necessary to confirm the generalizability of our findings and to identify modifiable practice patterns that increase LOS.
      In the present work, ML models exhibited superior accuracy in the prognostication of overall and ICU length of stay, compared with linear regression. A single-center study similarly compared linear regression and artificial neural networks, finding the latter to have enhanced LOS prediction for patients undergoing isolated CABG.
      • Triana A.J.
      • Vyas R.
      • Shah A.S.
      • Tiwari V.
      Predicting length of stay of coronary artery bypass grafting patients using machine learning.
      Furthermore, LaFaro and colleagues
      • LaFaro R.J.
      • Pothula S.
      • Kubal K.P.
      • Inchiosa M.E.
      • Pothula V.M.
      • Yuan S.C.
      • et al.
      Neural network prediction of ICU length of stay following cardiac surgery based on pre-incision variables.
      used a sample of 185 patients undergoing cardiac surgery to show that artificial neural networks yield more accurate estimates of ICU LOS compared with linear regression. The improved performance of ML models is likely attributable to their ability to capture nonlinear interactions between covariates and outcomes of interest. Although the decision-tree structure evaluates such interactions autonomously, linear regression models can only accommodate explicitly included interaction terms, making the development of an equivalent model cumbersome and more prone to bias. Our findings are in congruence with the growing body of literature, which demonstrates increased performance of ML models in the clinical setting.
      • Kilic A.
      • Goyal A.
      • Miller J.K.
      • Gjekmarkaj E.
      • Tam W.L.
      • Gleason T.G.
      • et al.
      Predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery.
      • Tseng P.-Y.
      • Chen Y.-T.
      • Wang C.-H.
      • Chiu K.-M.
      • Peng Y.-S.
      • Hsu S.-P.
      • et al.
      Prediction of the development of acute kidney injury following cardiac surgery by machine learning.
      • Lee H.-C.
      • Yoon H.-K.
      • Nam K.
      • Cho Y.J.
      • Kim T.K.
      • Kim W.H.
      • et al.
      Derivation and validation of machine learning approaches to predict acute kidney injury after cardiac surgery.
      Thus, ML algorithms should be considered as a viable and potentially superior alternative modeling approach in surgical care applications.
      Although ML methods outperformed linear strategies for prediction of reoperation, the STS models outperformed ML for 30-day mortality and AKI. This observation is most attributable to the large sample used to derive the STS risk scores as well as the incorporation of more than 100 data fields.
      • Shahian D.M.
      • Jacobs J.P.
      • Badhwar V.
      • Kurlansky P.A.
      • Furnary A.P.
      • Cleveland Jr., J.C.
      • et al.
      The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 1—background, design considerations, and model development.
      Nonetheless, the STS models are limited to operations either involving CABG or single-valve replacement, and do not provide risk estimates for aortic surgery or multivalve procedures. Such operations present a more heterogeneous risk profile and may reduce the performance of predictive models. We opted to include such operations in our modeling attempts to develop a tool that accurately reflects the case-mix at our 5 academic institutions. Indeed, procedures not accounted for by the STS comprised approximately 5% of our study cohort. Regardless, ML approaches are gradually being incorporated into the STS models to provide more bespoke estimates, an effort that will certainly improve risk prediction across cases performed in the United States.
      The predictive models developed in the present work have considerable utility in the clinical and administrative settings. Their mode of application is tunable to an institution's needs, and the insights that they provide have the potential to enhance clinical outcomes. A landmark randomized control trial by Shimabukuro and colleagues
      • Shimabukuro D.W.
      • Barton C.W.
      • Feldman M.D.
      • Mataraso S.J.
      • Das R.
      Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial.
      found the implementation of ML models to reduce ICU mortality and LOS, demonstrating that such tools can tangibly improve clinical outcomes and decrease resource utilization. Our group has chosen to make the ML models with the greatest R2 and AUC available for public use. This online tool may be used by clinicians when evaluating patient risk or by administrators who wish to apply our predictive model at the programmatic level. However, a model that continuously incorporates postoperative events into the estimated LOS would be most pertinent to patient care in the perioperative setting. Further efforts to develop such tools are warranted.
      Given the premium placed on low-acuity and ICU beds during the COVID-19 pandemic, hospitals transiently reduced surgical volume. Prachand and colleagues
      • Prachand V.N.
      • Milner R.
      • Angelos P.
      • Posner M.C.
      • Fung J.J.
      • Agrawal N.
      • et al.
      Medically necessary, time-sensitive procedures: scoring system to ethically and efficiently manage resource scarcity and provider risk during the COVID-19 pandemic.
      proposed a widely used framework for triaging medically necessary, time sensitive procedures. It highlighted several key factors, such as operating-room time, estimated LOS, and anticipated blood loss, when determining resource allocation. In the event of significant reduction in operating capacity, the development of algorithms that balance risk associated with delay in operative management as well as estimated resource use may be necessary. Our proposed ML based models may better inform decisions about scheduling and optimizing case-mix to ensure sufficient hospital throughput. With wide availability of ML present and use of few explanatory variables, prospective studies may readily determine the pragmatic influence of such models in optimizing hospital efficiency.
      The present study has several limitations. As a multicenter study confined to a group of academic centers, our findings are not generalizable to the cardiac surgical population at large. In addition, although the consortium makes a concerted effort to homogenize practice patterns across participating institutions, certain clinical factors may vary by center and surgeon, such as the threshold for blood transfusion. Transfer status was similarly not captured in the UCCSC and could not be accounted for in our predictive models. Furthermore, despite the relatively large size of the dataset, prospective application of the ML models is required to externally validate their utility. Nonetheless, we used robust statistical methods and a sparse set of autonomously selected variables to enhance the generalizability.

      Conclusions

      We have demonstrated the superior performance of ML models in providing accurate predictions for LOS using a multi-institutional, cardiac surgery database. Derived from few variables, such models can estimate resource use and better inform projected hospital census. Leveraging the information derived from machine learning models may be especially useful in reducing the influence of pandemic-related disruptions in cardiac surgical programs (Figure 4).
      Figure thumbnail gr4
      Figure 4Compared with linear regression, machine learning models exhibited superior performance in the estimation of length of stay following cardiac operations. CABG, Coronary artery bypass grafting; LOS, length of stay; ICU LOS, intensive care unit length of stay; EF, ejection fraction.

      Conflict of Interest Statement

      Dr Shemin serves as a consultant to the Edwards Lifesciences Advisory Board and as a co-principal investigator on the PARTNER II trial. All other authors reported no conflicts of interest.
      The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they have a conflict of interest. The editors and reviewers of this article have no conflicts of interest.
      Contributing members of the University of California Cardiac Surgery Consortium include University of California, Los Angeles: Peyman Benharash, MD, FACS; Richard J. Shemin, MD, FACS; and Nancy Satou. University of California, San Francisco: Tom Nguyen, MD; and Carolyn Clary. University of California, San Diego: Michael Madani, MD, FACS; Jill Higgins; and Dawna Steltzner. University of California, Davis: Bob Kiaii, MD, FRCSC, FACS; J. Nilas Young, MD, FACS; Kathleen Behan; Heather Houston; and Cindi Matsumoto. University of California, Irvine: Jack C. Sun, MD, MS, FRCSC; Lisha Flavin; and Patria Fopiano. University of California Office of the President: Maricel Cabrera. Biome Analytics: Rakan Khaki, MPH; and Polly Washabaugh, BS.

      Supplementary Data

      • Video 1

        Authors briefly discuss the performance of machine learning models in the prediction of resource use following cardiac operations. Video available at: https://www.jtcvs.org/article/S2666-2736(22)00171-1/fulltext.

      Appendix 1

      Figure thumbnail fx4
      Figure E1Study Consolidated Standards of Reporting Trials diagram. CABG, Coronary artery bypass grafting; UCCSC, University of California Cardiac Surgery Consortium; LOS, length of stay; ICU LOS, intensive care unit length of stay.
      Figure thumbnail fx5
      Figure E2Schematic representing the algorithmic design of random forest, gradient boosted machines (GBM) and extreme gradient boosting (XGBoost).
      Figure thumbnail fx6
      Figure E3Calibration plot of observed versus predicted length of stay in days. R2, Coefficient of determination; GBM, gradient boosted machine; LR, Linear regression.
      Table E1Distribution of missing variables in the entire study cohort, as well as between the derivation and validation cohorts
      ParameterOverall (n = 6316)Derivation (n = 5028)Validation (n = 1288)
      Age000
      Elective admission000
      Female000
      Height (cm)220
      Weight (kg)110
      Ethnicity000
      Operative type
       Isolated CABG000
       Isolated valve operation000
       CABG + valve000
       Multiple valve000
      Medical conditions
       Atrial fibrillation000
       Cancer000
       Cerebrovascular disease000
       Mild lung disease000
       Severe lung disease000
       Congestive heart failure000
       Diabetes000
       Home oxygen000
       Hypertension000
       Infectious endocarditis000
       Intra-aortic balloon pump000
       Liver disease000
       Mediastinal radiation000
       Peripheral vascular disease000
       Prior myocardial infarction000
       Syncope000
       Thoracic aortic disease000
       Current tobacco use000
      Laboratory values
       Hematocrit (%)39372
       International normalized ratio27523233
       Serum albumin (g/dL)940793147
       Preoperative creatinine (mg/dL)25223
       Ejection fraction (%)19817820
      Hospital of operation
       Center 1000
       Center 2000
       Center 3000
       Center 4000
       Center 5000
      CABG, Coronary artery bypass grafting.
      Table E2Hyperparameters for each machine learning model developed in the present study. Parameters not mentioned here were set as the default value
      OutcomeGBMRFXGBoost
      Length of stay{'max_depth': 3, 'max_features': 8, 'n_estimators': 60}{'max_depth': 6, 'max_features': 7, 'n_estimators': 40}{'alpha': 1, 'max_depth': 2, 'n_estimators': 30}
      ICU length of stay{'max_depth': 5, 'max_features': 2, 'n_estimators': 70}{'max_depth': 6, 'max_features': 6, 'n_estimators': 40}{'alpha': 10, 'max_depth': 3, 'n_estimators': 20}
      Mortality{'max_depth': 2, 'max_features': 7, 'n_estimators': 20}{'max_depth': 3, 'max_features': 2, 'n_estimators': 10}{'alpha': 10, 'max_depth': 2, 'n_estimators': 10}
      Acute kidney injury{'max_depth': 3, 'max_features': 10, 'n_estimators': 30}{'max_depth': 4, 'max_features': 9, 'n_estimators': 50}{'alpha': 1, 'max_depth': 6, 'n_estimators': 30}
      Postoperative transfusion{'max_depth': 2, 'max_features': 5, 'n_estimators': 10}{'max_depth': 2, 'max_features': 5, 'n_estimators': 10}{'alpha': 10, 'max_depth': 2, 'n_estimators': 10}
      Reoperation{'max_depth': 2, 'max_features': 8, 'n_estimators': 10}{'max_depth': 3, 'max_features': 10, 'n_estimators': 40}{'alpha': 0.1, 'max_depth': 2, 'n_estimators': 10}
      GBM, Gradient boosted machine; RF, Random forest; XGBoost, extreme gradient boosting; ICU, intensive care unit.
      Table E3Covariates selected for final model development
      Selected covariates
      Baseline characteristics
       Age
       Elective operation
       Height (cm)
       Weight (kg)
      Operation type
       Isolated CABG
       Isolated valve operation
       CABG + valve
       Multiple valve
      Medical conditions
       Preoperative atrial fibrillation
       Cerebrovascular disease
       Mild lung disease
       Severe lung disease
       Congestive heart failure
       Home oxygen
       Previous myocardial infarction
       Hematocrit
       International normalized ratio
       Serum albumin
       Preoperative creatinine
       Ejection fraction
      Hospital characteristics
       Annual hospital volume
       No. of cardiac surgeons on staff
       No. of low-acuity beds
      CABG, Coronary artery bypass grafting.
      Table E4Cross-validated Brier scores for binary classifiers
      OutcomeLogisticGBMRFXGBoostSTS
      Mortality0.0096 (0.0095-0.0096)0.0105 (0.0104-0.0107)0.0095 (0.0095-0.0095)0.0095 (0.0094-0.0095)0.0075 (0.0073-0.0077)
      Acute kidney injury0.0146 (0.0145-0.0146)0.0161 (0.0159-0.0163)0.0144 (0.0143-0.0144)0.0145 (0.0145-0.0146)0.0175 (0.0175-0.0175)
      Postoperative transfusion0.1827 (0.1823-0.1831)0.1781 (0.1777-0.1784)0.1894 (0.1892-0.1896)0.1755 (0.1751-0.1759)
      Reoperation0.0727 (0.0725-0.0729)0.076 (0.0759-0.0761)0.0739 (0.0738-0.0741)0.0746 (0.0744-0.0747)0.0603 (0.0601-0.0604)
      GBM, Gradient boosted machine; RF, Random forest. XGBoost, extreme gradient boosting; STS, Society of Thoracic Surgeon Risk Score.
      Table E5Evaluation of Brier score for each predictive model using data from the validation cohort
      OutcomeLogisticGBMRFXGBoostSTS
      Mortality0.00690.00830.00690.00670.0070
      Acute kidney injury0.01680.01650.01620.01750.0170
      Postoperative transfusion0.17330.17080.17150.1751
      Reoperation0.05860.06050.05940.06120.0597
      GBM, Gradient boosted machine; RF, Random forest; XGBoost, extreme gradient boosting; STS, Society of Thoracic Surgeon Risk Score.

      References

        • Gaudino M.
        • Chikwe J.
        • Hameed I.
        • Robinson N.B.
        • Fremes S.E.
        • Ruel M.
        Response of cardiac surgery units to COVID-19.
        Circulation. 2020; 142: 300-302https://doi.org/10.1161/CIRCULATIONAHA.120.047865
        • Farrington W.J.
        • Robinson N.B.
        • Rahouma M.
        • Lau C.
        • Hameed I.
        • Iannacone E.M.
        • et al.
        Cardiac surgery outcomes in an epicenter of the COVID-19 pandemic.
        Semin Thorac Cardiovasc Surg. 2022; 34: 182-188https://doi.org/10.1053/J.SEMTCVS.2021.01.005
        • Ad N.
        • Luc J.G.Y.
        • Nguyen T.C.
        • COVID-19 North American Cardiac Surgery Survey Working Group
        Cardiac surgery in North America and coronavirus disease 2019 (COVID-19): regional variability in burden and impact.
        J Thorac Cardiovasc Surg. 2021; 162: 893-903.e4https://doi.org/10.1016/J.JTCVS.2020.06.077
        • George I.
        • Salna M.
        • Kobsa S.
        • Deroo S.
        • Kriegel J.
        • Blitzer D.
        • et al.
        The rapid transformation of cardiac surgery practice in the coronavirus disease 2019 (COVID-19) pandemic: insights and clinical strategies from a centre at the epicentre.
        Eur J Cardiothorac Surg. 2020; 58: 667-675https://doi.org/10.1093/EJCTS/EZAA228
        • Bose S.K.
        • Dasani S.
        • Roberts S.E.
        • Chris W.
        • DeMatteo R.P.
        • Doherty G.M.
        • et al.
        The cost of quarantine: projecting the financial impact of canceled elective surgery on the nation’s hospitals.
        Ann Surg. 2021; 273: 844-849https://doi.org/10.1097/SLA.0000000000004766
        • Salenger R.
        • Etchill E.W.
        • Ad N.
        • Matthew T.
        • Alejo D.
        • Whitman G.
        • et al.
        The surge after the surge: cardiac surgery post–COVID-19.
        Ann Thorac Surg. 2020; 110: 2020-2025https://doi.org/10.1016/J.ATHORACSUR.2020.04.018
        • Engelman D.T.
        • Lother S.
        • George I.
        • Ailawadi G.
        • Atluri P.
        • Grant M.C.
        • et al.
        Ramping up delivery of cardiac surgery during the COVID-19 pandemic: a guidance statement from the Society of Thoracic Surgeons COVID-19 Task Force.
        Ann Thorac Surg. 2020; 110: 712-717https://doi.org/10.1016/J.ATHORACSUR.2020.05.002
        • O’Brien S.M.
        • Feng L.
        • He X.
        • Xian Y.
        • Jacobs J.P.
        • Badhwar V.
        • et al.
        The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 2—statistical methods and results.
        Ann Thorac Surg. 2018; 105: 1419-1428https://doi.org/10.1016/J.ATHORACSUR.2018.03.003
        • Chan V.
        • Ahrari A.
        • Ruel M.
        • Elmistekawy E.
        • Hynes M.
        • Mesana T.G.
        Perioperative deaths after mitral valve operations may be overestimated by contemporary risk models.
        Ann Thorac Surg. 2014; 98: 605-610https://doi.org/10.1016/J.ATHORACSUR.2014.05.011
        • Kirmani B.H.
        • Mazhar K.
        • Saleh H.Z.
        • Ward A.N.
        • Shaw M.
        • Fabri B.
        • et al.
        External validity of the Society of Thoracic Surgeons risk stratification tool for deep sternal wound infection after cardiac surgery in a UK population.
        Interact Cardiovasc Thorac Surg. 2013; 17: 479-484https://doi.org/10.1093/ICVTS/IVT222
        • Sharkawi M.A.
        • Shah P.B.
        • Zenati M.
        • Kaneko T.
        • Ramadan R.
        Underclassification of predicted risk of mortality using the latest Society of Thoracic Surgeons Risk Models.
        Structural Heart. 2021; 3: 330-331
        • Daghistani T.A.
        • Elshawi R.
        • Sakr S.
        • Ahmed A.M.
        • Al-Thwayee A.
        • Al-Mallah M.H.
        Predictors of in-hospital length of stay among cardiac patients: a machine learning approach.
        Int J Cardiol. 2019; 288: 140-147https://doi.org/10.1016/J.IJCARD.2019.01.046
        • Alshakhs F.
        • Alharthi H.
        • Aslam N.
        • Khan I.U.
        • Elasheri M.
        Predicting postoperative length of stay for isolated coronary artery bypass graft patients using machine learning.
        Int J Gen Med. 2020; 13: 751https://doi.org/10.2147/IJGM.S250334
        • Shahian D.M.
        • Jacobs J.P.
        • Badhwar V.
        • Kurlansky P.A.
        • Furnary A.P.
        • Cleveland Jr., J.C.
        • et al.
        The Society of Thoracic Surgeons 2018 adult cardiac Surgery risk models: part 1—background, design considerations, and model development.
        Ann Thorac Surg. 2018; 105: 1411-1418https://doi.org/10.1016/J.ATHORACSUR.2018.03.002
        • Messaoudi N.
        • De Cocker J.
        • Stockman B.
        • Bossaert L.L.
        • Rodrigus I.E.R.
        Prediction of prolonged length of stay in the intensive care unit after cardiac surgery: the need for a multi-institutional risk scoring system.
        J Card Surg. 2009; 24: 127-133https://doi.org/10.1111/J.1540-8191.2008.00716.X
        • Kilic A.
        • Goyal A.
        • Miller J.K.
        • Gjekmarkaj E.
        • Tam W.L.
        • Gleason T.G.
        • et al.
        Predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery.
        Ann Thorac Surg. 2020; 109: 1811-1819https://doi.org/10.1016/J.ATHORACSUR.2019.09.049
        • Tseng P.-Y.
        • Chen Y.-T.
        • Wang C.-H.
        • Chiu K.-M.
        • Peng Y.-S.
        • Hsu S.-P.
        • et al.
        Prediction of the development of acute kidney injury following cardiac surgery by machine learning.
        Crit Care. 2020; 24: 1-13https://doi.org/10.1186/S13054-020-03179-9
        • Lee H.-C.
        • Yoon H.-K.
        • Nam K.
        • Cho Y.J.
        • Kim T.K.
        • Kim W.H.
        • et al.
        Derivation and validation of machine learning approaches to predict acute kidney injury after cardiac surgery.
        J Clin Med. 2018; 7: 322https://doi.org/10.3390/JCM7100322
        • Society of Thoracic Surgeons
        Adult cardiac surgery database data collection.
        • Chen T.
        • Guestrin C.
        XGBoost: a scalable tree boosting system.
        • Lundberg S.M.
        • Lee S.-I.
        A unified approach to interpreting model predictions.
        https://arxiv.org/abs/1705.07874
        Date accessed: April 26, 2022
        • Pedregosa F.
        • Varoquaux G.
        • Gramfort A.
        • Michel V.
        • Thirion B.
        • Grisel O.
        • et al.
        Scikit-learn: machine learning in Python.
        J Mach Learn Res. 2011; 12: 2825-2830
        • Almashrafi A.
        • Elmontsri M.
        • Aylin P.
        Systematic review of factors influencing length of stay in ICU after adult cardiac surgery.
        BMC Heal Serv Res. 2016; 16: 1-12https://doi.org/10.1186/S12913-016-1591-3
        • Tzeng C.-W.D.
        • Teshome M.
        • Katz M.H.G.
        • Weinberg J.S.
        • Lai S.Y.
        • Antonoff M.B.
        • et al.
        Cancer surgery scheduling during and after the COVID-19 first wave: the MD Anderson Cancer Center experience.
        Ann Surg. 2020; 272: e106https://doi.org/10.1097/SLA.0000000000004092
        • Tran Z.
        • Chervu N.
        • Williamson C.
        • Verma A.
        • Hadaya J.
        • Gandjian M.
        • et al.
        The impact of expedited discharge on 30-day readmission following lung resection: a national study.
        Ann Thorac Surg. 2022; 113: 1274-1281https://doi.org/10.1016/J.ATHORACSUR.2021.04.009
        • Afflu D.K.
        • Seese L.
        • Sultan I.
        • Gleason T.
        • Wang Y.
        • Navid F.
        • et al.
        Very early discharge after coronary artery bypass grafting does not affect readmission or survival.
        Ann Thorac Surg. 2021; 111: 906-913https://doi.org/10.1016/J.ATHORACSUR.2020.05.159
        • Voeten D.M.
        • van der Werf L.R.
        • Gisbertz S.S.
        • Ruurda J.P.
        • van Berge Henegouwen M.I.
        • van Hillegersberg R.
        • et al.
        Postoperative intensive care unit stay after minimally invasive esophagectomy shows large hospital variation. Results from the Dutch Upper Gastrointestinal Cancer Audit.
        Eur J Surg Oncol. 2021; 47: 1961-1968https://doi.org/10.1016/J.EJSO.2021.01.005
        • Triana A.J.
        • Vyas R.
        • Shah A.S.
        • Tiwari V.
        Predicting length of stay of coronary artery bypass grafting patients using machine learning.
        J Surg Res. 2021; 264: 68-75https://doi.org/10.1016/J.JSS.2021.02.003
        • LaFaro R.J.
        • Pothula S.
        • Kubal K.P.
        • Inchiosa M.E.
        • Pothula V.M.
        • Yuan S.C.
        • et al.
        Neural network prediction of ICU length of stay following cardiac surgery based on pre-incision variables.
        PLoS One. 2015; 10: e0145395https://doi.org/10.1371/JOURNAL.PONE.0145395
        • Shimabukuro D.W.
        • Barton C.W.
        • Feldman M.D.
        • Mataraso S.J.
        • Das R.
        Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial.
        BMJ Open Respir Res. 2017; 4: e000234https://doi.org/10.1136/BMJRESP-2017-000234
        • Prachand V.N.
        • Milner R.
        • Angelos P.
        • Posner M.C.
        • Fung J.J.
        • Agrawal N.
        • et al.
        Medically necessary, time-sensitive procedures: scoring system to ethically and efficiently manage resource scarcity and provider risk during the COVID-19 pandemic.
        J Am Coll Surg. 2020; 231: 281-288https://doi.org/10.1016/J.JAMCOLLSURG.2020.04.011