- Journal List
- HHS Author Manuscripts
- PMC10272088
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
Surgery. Author manuscript; available in PMC 2024 Jul 1.
Published in final edited form as:
Surgery. 2023 Jul; 174(1): 66–74.
Published online 2023 May 5. doi:10.1016/j.surg.2023.02.024
PMCID: PMC10272088
NIHMSID: NIHMS1880910
PMID: 37149424
Emily M Mason, MSCS,a,b William G Henderson, PhD, MPH,b,c,d Michael R Bronsert, PhD, MS,b,c Kathryn L Colborn, PhD, MSPH,b,d Adam R Dyas, MD,b Anne Lambert-Kerzner, PhD, MSPH,b,d and Robert A Meguid, MD, MPH, FACSb,c,*
Author information Copyright and License information PMC Disclaimer
The publisher's final edited version of this article is available at Surgery
Associated Data
- Supplementary Materials
Abstract
Background
Postoperative length of stay (LOS) is a meaningful patient-centered outcome and an important determinant of healthcare costs. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict postoperative LOS has not been assessed. We aimed to determine whether the SURPAS variables could accurately predict postoperative LOS up to 30 days in a broad inpatient surgical population.
Methods
This was a retrospective analysis of the American College of Surgeons’ National Surgical Quality Improvement Program (ACS-NSQIP) adult database, 2012–2018. A model using the SURPAS variables and a 28-variable “full” model, incorporating all available ACS-NSQIP preoperative nonlaboratory variables, were fit to the analytical cohort (2012–2018) using multiple linear regression and compared using model performance metrics. Internal chronological validation of the SURPAS model was conducted using training (2012–2017) and test (2018) datasets.
Results
We analyzed 3,295,028 procedures. The adjusted R2 for the SURPAS model fit to this cohort was 93.3% of that for the full model (0.347 vs. 0.372). In the internal chronological validation of the SURPAS model, the adjusted R2 for the test dataset was 97.1% of that for the training dataset (0.3389 vs. 0.3489).
Conclusions
The parsimonious SURPAS model can preoperatively predict postoperative LOS up to 30 days for inpatient surgical procedures almost as accurately as a model using all 28 ACS-NSQIP preoperative nonlaboratory variables and has shown acceptable internal chronological validation.
Two-Sentence Article Summary
We aimed to determine whether the 7-variable Surgical Risk Preoperative Assessment System (SURPAS) model can accurately predict postoperative length of stay (LOS) up to 30 days in a broad inpatient surgical population. The parsimonious SURPAS model can preoperatively predict postoperative LOS almost as accurately as a model using all 28 American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) preoperative nonlaboratory variables and has shown acceptable internal chronological validation.
INTRODUCTION
Postoperative length of stay (LOS) is a meaningful patient-centered outcome and an important determinant of healthcare costs.1,2 Current discussions between the surgeon and patient about possible outcomes including length of stay are usually informed by surgeon experience.3 However, patients report increased comfort with the decision for surgery and decreased anxiety over upcoming surgery when engaged in data-driven discussions about risk of postoperative outcomes.4 Accurate preoperative prediction of this outcome could potentially improve presurgical patient counseling, surgical case planning and risk mitigation, and accuracy of healthcare institution cost projections and resource planning.5–11
We identified 20 recent articles examining preoperative predictors of postoperative LOS. Of these, 13 (65%) articles described predictors of LOS as a primary outcome,1,2,12–22 four (20%) articles incorporated only preoperative predictor variables,2,13,14,21 and only one (5%) used a broad surgical population.12 However, no single article met all of these three criteria. We are aware of only one risk calculator available for preoperative prediction of postoperative LOS in a broad surgical population, specifically the American College of Surgeons’ (ACS) surgical risk calculator.23–26 However, details of the development and characteristics of the ACS surgical risk calculator model for postoperative LOS have not been published. Further, the ACS surgical risk calculator models require the input of 22 predictor variables and are limited by an ACS policy restricting their release.26
The Surgical Risk Preoperative Assessment System (SURPAS) is a parsimonious risk assessment tool using eight preoperative predictor variables to estimate a patient’s risk for 12 postoperative adverse events.27–30 It was developed from the ACS National Surgical Quality Improvement Program (ACS-NSQIP) dataset27–30 and is applicable to more than 3000 operations in adults in nine surgical specialties: general surgery, gynecology, neurosurgery, orthopedic, otolaryngology, plastic, thoracic, urology, and vascular. The eight predictor variables include four related to the operation (Current Procedural Terminology [CPT] specific variable for the outcome being assessed, inpatient versus outpatient operation setting, primary surgeon specialty, and work relative value unit [RVU] as a measure of surgical procedure complexity) and four related to the patient (patient age, American Society of Anesthesiology Physical Status Classification [ASA class], functional health status before the operation, and emergent versus non-emergent surgery status). The twelve 30-day postoperative outcomes include mortality, overall morbidity, unplanned readmission, non-home discharge, and eight specific types of postoperative complications: infection, cardiac, pulmonary, renal, urinary tract infection, venous thromboembolism, bleeding, and stroke. The SURPAS models have been extensively validated.29–37 In pilot studies, SURPAS has been well received by surgeons and patients38,39 and has improved patient satisfaction, comfort, and decreased anxiety about the pending operation.4 At the University of Colorado Health System (UCHealth), SURPAS has been incorporated into the electronic health record. Additionally, the SURPAS model equations are publicly available in Predictive Model Markup Language to facilitate SURPAS incorporation into other institutions’ electronic health records,40 and the risk calculator tool is available for use at https://surpas.agilemd.com/surpas.41 The ability of SURPAS to preoperatively predict a patient’s postoperative LOS has not yet been assessed.
The purpose of this study was to determine whether the SURPAS model would: (1) predict postoperative LOS for inpatient procedures as accurately as a model using all of the 28 preoperative nonlaboratory predictor variables available in the ACS-NSQIP dataset (i.e., the “full” model), and (2) demonstrate acceptable performance in a test of internal chronological validation.
METHODS
Study Setting and Data Source
This study was a retrospective cohort analysis using the 2012–2018 ACS-NSQIP Participant Use Data Files (PUFs). These contain patient-level, aggregate data from approximately 700 participating hospitals (the participant list can be found at https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/about-acs-nsqip/participants/).42 The dataset contains preoperative, operative, and 30-day postoperative data for adult patients in nine surgical specialties: general surgery, gynecology, neurosurgery, orthopedic, otolaryngology, plastic, thoracic, urology, and vascular. Definitions of all ACS-NSQIP variables can be found at https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/participant-use-data-file/.43 Data are collected by certified Surgical Clinical Reviewers through manual chart review and direct contact with patients and families at 30 days after the operation and are periodically audited. The ACS-NSQIP protocol specifies systematic sampling of surgical cases for participating institutions based on the institution’s size, urban-rural designation, and extent of surgical specialties reported (e.g., general/vascular surgery, multispecialty surgery, or procedure-targeted data). Full sampling details are provided in a PUF User Guide for each annual dataset and can be found at https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/participant-use-data-file/.43
Because postoperative LOS is generally relevant only to inpatients, the study was conducted only with inpatients in the ACS-NSQIP database. Also, because inpatient/outpatient setting is one of the eight SURPAS preoperative predictor variables and therefore was a constant in the analytical cohort, a seven-variable SURPAS model (excluding the inpatient/outpatient variable) was developed in this study. This study was deemed exempt from review by the Colorado Multiple Institutional Review Board because the ACS-NSQIP data were de-identified and publicly available.
Study Sample
The study sample included operations from the 2012–2018 ACS-NSQIP database. Operations were excluded if they were not within the original nine target surgical specialties of the ACS-NSQIP, missing key preoperative data, outpatient procedures, operations with LOS greater than 30 days or missing LOS, and patients who died during the index hospitalization or within 30 days after surgery.
Primary Outcome
The primary outcome assessed in this study was postoperative LOS up to 30 days. We eliminated cases with LOS >30 days, because our interest was to develop accurate LOS estimates for average patients undergoing a given surgical operation rather than for patients at risk of developing serious complications that require extended periods of hospitalization.
Statistical Analyses
Selection of the analytical cohort was described using a Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) flow diagram (Figure 1). We excluded observations not within the original nine target surgical specialties of the ACS-NSQIP, missing key preoperative data, outpatient procedures, observations with LOS >30 days or missing LOS, and patients who died during the index hospitalization or within 30 days of surgery. The analytical cohort was separated into training (2012–2017) and test (2018) datasets. Distribution measures for LOS were calculated.
Figure 1.
Strengthening the Reporting of Observational Studies in Epidemiology flow diagram. Abbreviations: ACS-NSQIP PUF, American College of Surgeons National Surgical Quality Improvement Program Participant Use File; ASA class, American Society of Anesthesiology Physical Status Classification.
*Target surgical specialties included general surgery, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery.
The bivariable association between each categorical ACS-NSQIP preoperative predictor variable and postoperative LOS was examined by comparing the mean LOS between categories of the variable using an independent sample t-test (variables with two categories) or analysis of variance (variables with more than two categories). For the continuous ACS-NSQIP preoperative predictor variables, Pearson correlation coefficients (PCC) were calculated between the preoperative variable and postoperative LOS. The preoperative variable “CPT-specific mean LOS” was calculated by taking all operations with a specific CPT code for the primary operation and calculating the mean LOS for those operations. This was done for each individual CPT code of the primary operation.
Multiple linear regression was used to fit the LOS prediction models, with LOS up to 30 days as the dependent variable and the ACS-NSQIP preoperative nonlaboratory variables as the independent variables. We treated LOS as having a Gaussian distribution after comparing the goodness of fit of a linear model to a Poisson regression model and a negative binomial regression model and finding that the Gaussian distribution was a better fit when comparing adjusted R2 and calibration.
For fitting of the full model, the 28 ACS-NSQIP preoperative nonlaboratory variables were used as the independent variables. Preoperative laboratory data were not used because it was previously found that they do not contribute to risk prediction beyond the nonlaboratory data and are missing not at random.29 The seven SURPAS preoperative predictor variables described previously were used to fit the SURPAS model; however, the inpatient/outpatient SURPAS variable was not used because only inpatient operations were included in this analysis.30 Model performance was compared between the full model and SURPAS model using measures for overall fit (adjusted R2) and calibration (scatterplot of observed vs. predicted LOS characterized by intercept and slope). Intercept values closer to 0 and slope values closer to 1 indicated better calibration.
Internal chronological validation testing of the SURPAS model was performed, developing the model with the training dataset (2012–2017), applying this model to both the training and test (2018) datasets, and comparing model performance measures (adjusted R2, calibration intercept, and calibration slope) between the two datasets. Statistical analyses were conducted in SAS version 9.4 (SAS Institute; Cary, NC, USA).
RESULTS
A total of 5,881,881 surgical procedures were available in the ACS-NSQIP PUF 2012–2018 source dataset. Figure 1 shows the STROBE flow diagram for inpatient procedure inclusion for the analytical cohort (2012–2018; N=3,295,028), training dataset (2012–2017; N=2,750,226), and test dataset (2018; N=544,802).
Table 1 presents the bivariable associations between preoperative patient characteristics and postoperative LOS. Tests of association between LOS and all 28 preoperative variables were statistically significant at p <0.0001. For most of the preoperative variables, the associations were in the expected direction. Some notable differences in LOS were: male vs. female (4.1 days vs. 3.6 days); black vs. white (4.2 vs. 3.8); underweight vs. normal weight (5.7 vs. 4.3); transfer from acute or chronic care vs. admission from home (6.3, 6.6, vs. 3.7); emergency vs. non-emergency operation (5.5 vs. 3.6); surgical specialty (thoracic 5.2 to gynecology 2.4); and ASA class (I, 2.1; II, 2.9; III, 4.3; IV, 6.7; V, 12.4). For the comorbidities, the longest LOS durations were noted for the following: ascites (7.9 days) vs. none (3.8 days); totally dependent functional health status (7.4) vs. partially dependent (6.4) and independent (3.7); septic shock (13.6) vs. sepsis (7.4), systemic inflammatory response syndrome (5.2), and none (3.6); acute renal failure (8.4) vs. none (3.8); preoperative transfusion (8.1) vs. none (3.8); ventilator dependence (13.8) vs. none (3.8); and greater than 10% loss of body weight over the prior six months (7.4) vs. none (3.8). Finally, all continuous predictor variables showed a positive correlation with LOS, indicating that LOS increased with increasing age (PCC=0.1529), greater procedural complexity as represented by work RVU (PCC=0.2465), and CPT-specific mean LOS (PCC=0.5609).
Table 1.
Bivariable association between preoperative patient characteristics and postoperative length of stay up to 30 days using the analytical cohort (2012–2018; N=3,295,028).
| Characteristic* | Procedures n (%) | LOS† Mean (SD) | LOS† Median [IQR] |
|---|---|---|---|
| Total Sample | 3,295,028 (100.0) | 3.8 (3.9) | 3 [1, 5] |
| Sex | |||
| Female | 1,859,106 (56.4) | 3.6 (3.6) | 3 [1, 4] |
| Male | 1,435,922 (43.6) | 4.1 (4.2) | 3 [1, 5] |
| Race/ethnicity | |||
| American Indian or Alaska Native | 16,977 (0.5) | 4.1 (4.1) | 3 [2, 5] |
| Asian or Pacific Islander | 101,684 (3.1) | 3.9 (4.0) | 3 [1, 5] |
| Black, not Hispanic origin | 333,070 (10.1) | 4.2 (4.3) | 3 [2, 5] |
| Hispanic origin | 169,410 (5.1) | 3.5 (3.7) | 2 [1, 4] |
| White, not Hispanic origin | 2,161,490 (65.6) | 3.8 (3.8) | 3 [2, 5] |
| Null/unknown | 512,397 (15.6) | 3.9 (4.3) | 3 [1, 5] |
| BMI category | |||
| Underweight (< 18.5 kg/m2) | 60,218 (1.8) | 5.7 (5.1) | 4 [2, 7] |
| Normal weight (18.5 – 24.9 kg/m2) | 732,880 (22.2) | 4.3 (4.3) | 3 [2, 5] |
| Overweight (25.0 – 29.9 kg/m2) | 969,730 (29.4) | 3.8 (3.8) | 3 [1, 5] |
| Obese class 1 (30.0 – 34.9 kg/m2) | 693,217 (21.0) | 3.6 (3.6) | 3 [1, 4] |
| Obese class 2 (35.0 – 39.9 kg/m2) | 382,412 (11.6) | 3.5 (3.5) | 2 [1, 4] |
| Obese class 3 (> 39.9 kg/m2) | 377,974 (11.5) | 3.3 (3.4) | 2 [1, 4] |
| Null/unknown | 78,597 (2.4) | 4.9 (5.4) | 3 [1, 6] |
| Ascites (within 30 days) | |||
| No | 3,281,045 (99.6) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 13,983 (0.4) | 7.9 (6.1) | 6 [4, 10] |
| Bleeding disorder requiring hospitalization | |||
| No | 3,118,294 (94.6) | 3.7 (3.8) | 3 [1, 5] |
| Yes | 176,734 (5.4) | 5.4 (5.1) | 4 [2, 7] |
| Diabetes mellitus | |||
| No | 2,704,812 (82.1) | 3.7 (3.8) | 3 [1, 4] |
| Oral/non-insulin agent | 362,862 (11.0) | 4.0 (3.9) | 3 [2, 5] |
| Insulin | 227,354 (6.9) | 5.1 (4.8) | 4 [2, 7] |
| Dialysis or hemofiltration (within 2 weeks) | |||
| No | 3,250,121 (98.6) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 44,907 (1.4) | 6.4 (5.7) | 5 [2, 8] |
| Disseminated cancer | |||
| No | 3,187,363 (96.7) | 3.8 (3.8) | 3 [1, 5] |
| Yes | 107,665 (3.3) | 6.3 (5.0) | 5 [3, 8] |
| Dyspnea (within 30 days) | |||
| None | 3,079,353 (93.5) | 3.8 (3.9) | 3 [1, 5] |
| Moderate exertion | 199,340 (6.1) | 4.5 (4.4) | 3 [2, 6] |
| At rest | 16,335 (0.5) | 6.4 (5.8) | 5 [2, 8] |
| Functional health status before operation | |||
| Independent | 3,176,030 (96.4) | 3.7 (3.8) | 3 [1, 5] |
| Partially dependent | 99,557 (3.0) | 6.4 (5.4) | 5 [3, 8] |
| Totally dependent | 19,441 (0.6) | 7.4 (6.1) | 5 [3, 10] |
| Congestive heart failure (within 30 days) | |||
| No | 3,262,097 (99.0) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 32,931 (1.0) | 6.8 (5.8) | 5 [3, 9] |
| Severe COPD | |||
| No | 3,118,789 (94.7) | 3.8 (3.8) | 3 [1, 5] |
| Yes | 176,239 (5.4) | 5.3 (4.9) | 4 [2, 7] |
| Blood pressure >140/90 mmHg or taking antihypertension medication | |||
| No | 1,614,858 (49.0) | 3.5 (3.7) | 2 [1, 4] |
| Yes | 1,680,170 (51.0 | 4.1 (4.1) | 3 [2, 5] |
| Systemic sepsis (within 48 hours) | |||
| None | 3,062,335 (92.9) | 3.6 (3.6) | 3 [1, 4] |
| SIRS | 122,778 (3.7) | 5.2 (5.2) | 4 [2, 7] |
| Sepsis | 98,412 (3.0) | 7.4 (5.9) | 6 [3, 10] |
| Septic shock | 11,503 (0.4) | 13.6 (7.1) | 13 [8, 18] |
| Acute renal failure (rising creatinine to >3 mg/dL within 24 hours) | |||
| No | 3,281,932 (99.6) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 13,096 (0.4) | 8.4 (6.7) | 7 [3, 12] |
| Cigarette smoker (within 1 year) | |||
| No | 2,706,669 (82.1) | 3.8 (3.8) | 3 [1, 4] |
| Yes | 588,359 (17.9) | 4.2 (4.3) | 3 [1, 5] |
| Steroid use for chronic condition | |||
| No | 3,147,420 (95.5) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 147,608 (4.5) | 5.1 (4.6) | 4 [2, 6] |
| Transfusion of packed RBCs (within 72 hours) | |||
| No | 3,253,796 (98.8) | 3.8 (3.8) | 3 [1, 5] |
| Yes | 41,232 (1.3) | 8.1 (6.1) | 6 [4, 11] |
| Ventilator-dependent (within 48 hours) | |||
| No | 3,286,491 (99.7) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 8,537 (0.3) | 13.8 (7.5) | 13 [8, 19] |
| Open wound with or without infection | |||
| No | 3,170,790 (96.2) | 3.7 (3.8) | 3 [4, 1] |
| Yes | 124,238 (3.8) | 6.9 (5.7) | 5 [3, 9] |
| >10% loss of body weight (within 6 months) | |||
| No | 3,237,613 (98.3) | 3.8 (3.9) | 3 [1, 5] |
| Yes | 57,415 (1.7) | 7.4 (5.5) | 6 [4, 9] |
| Transfer status | |||
| Admitted directly from home | 3,106,620 (94.3) | 3.7 (3.7) | 3 [1, 4] |
| Acute care hospital | 147,117 (4.5) | 6.3 (5.7) | 5 [2, 8] |
| Chronic care facility | 41,291 (1.3) | 6.6 (5.5) | 5 [3, 8] |
| Primary surgeon specialty | |||
| General surgery | 1,413,573 (42.9) | 4.5 (4.5) | 3 [2, 6] |
| Gynecology | 206,537 (6.3) | 2.4 (2.2) | 2 [1, 3] |
| Neurosurgery | 217,860 (6.6) | 3.8 (3.8) | 3 [2, 4] |
| Orthopedic | 857,801 (26.0) | 3.1 (2.7) | 3 [2, 3] |
| Otolaryngology | 45,496 (1.4) | 3.4 (4.2) | 2 [1, 4] |
| Plastic | 44,868 (1.4) | 3.6 (4.0) | 3 [1, 4] |
| Thoracic | 63,642 (1.9) | 5.2 (4.4) | 4 [2, 7] |
| Urology | 177,184 (5.4) | 3.2 (3.3) | 2 [1, 4] |
| Vascular | 268,067 (8.1) | 4.2 (4.5) | 3 [1, 6] |
| Emergency operation | |||
| No | 2,919,164 (88.6) | 3.6 (3.6) | 3 [1, 4] |
| Yes | 375,864 (11.4) | 5.3 (5.5) | 4 [1, 7] |
| ASA class | |||
| I (normal healthy patient) | 158,045 (4.8) | 2.1 (2.2) | 1 [1, 3] |
| II (patient with mild systemic disease) | 1,278,284 (38.8) | 2.9 (2.8) | 2 [1, 4] |
| III (patient with severe systemic disease) | 1,612,399 (48.9) | 4.3 (4.1) | 3 [2, 5] |
| IV (patient with severe systemic disease that is a constant threat) | 242,142 (7.4) | 6.7 (5.8) | 5 [3, 9] |
| V (moribund patient who is not expected to survive) | 4,158 (0.1) | 12.4 (7.5) | 11 [7, 18] |
Open in a separate window
Abbreviations: ASA class, American Society of Anesthesiologists Physical Status Classification; BMI, body mass index; COPD, chronic obstructive pulmonary disease; CPT, current procedural terminology; IQR, interquartile range; LOS, length of stay in days; RBC, red blood cell; RVU, relative value unit; SD, standard deviation; SIRS, systemic inflammatory response syndrome.
*Independent sample t-tests (for categorical predictor variables with 2 categories), one-factor analyses of variance (for categorical predictor variables with >2 categories), and Pearson correlation analyses (for continuous predictor variables) were conducted and resulted in p-values <0.0001. Pearson correlation coefficients (PCCs) for continuous variables were as follows: age (PCC=0.1529), work RVU (PCC=0.2465), and CPT-specific mean LOS (PCC=0.5609).
†Length of stay (LOS) is reported in days. LOS range was 0–30 days across all characteristics.
Comparison of the multiple linear regression to the Poisson and negative binomial regressions using the full models (all 28 ACS-NSQIP preoperative nonlaboratory variables) and analytical cohort showed that the multiple linear regression accounted for more of the variability in observed LOS (adjusted R2 0.3720 vs. 0.3396 and 0.3294, respectively), lower systematic bias (intercept −1.65E-11 vs. 0.351 and 0.764), better discrimination (slope 1.000 vs. 0.909 and 0.791), and better calibration (both intercept and slope nearer their target values). Therefore, multiple linear regression was chosen as the regression method for prediction model development and assessment in this study.
Table 2 shows the results of the multiple linear regression analysis using the 28 variable full model. The adjusted R2 was 0.372, the calibration intercept was −1.65E-11, and the calibration slope was 1.00. Each regression coefficient (with 95% confidence interval [CI]) represents either the change in LOS for having the characteristic vs. not having the characteristic for categorical variables or the change in LOS per 1-unit change in the predictor for continuous variables, after adjusting for all other predictors. Among the 28 predictor variables included in the full model, the changes in LOS were statistically significant (p<0.05) for all variables except presence of diabetes mellitus requiring oral/non-insulin agent (LOS change of −0.01 days [95% CI: −0.021, 0.001] vs. none; p=0.07) and blood pressure greater than 140/90 mmHg or taking antihypertension medication (LOS change of 0.01 [95% CI: −0.001, 0.015] vs. none; p=0.08). The risk factors producing the greatest increases in LOS were septic shock (4.04 days), ventilator dependence (3.51), ASA class V (3.25), sepsis (1.72), ASA class IV (1.51), ascites (1.35), and transfusion (1.07).
Table 2.
Multiple linear regression analysis of the full model for postoperative length of stay up to 30 days (dependent variable) and the 28 American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) preoperative nonlaboratory predictor variables (independent variables) using the analytical cohort (2012 to 2018; N=3,295,028).
| Characteristic* | Regression Coefficient† | 95% CI |
|---|---|---|
| Sex, male vs female | 0.008 | (0.001, 0.016) |
| Age | 0.015 | (0.014, 0.015) |
| Race/ethnicity vs White, not Hispanic origin | ||
| American Indian or Alaska Native | 0.290 | (0.243, 0.337) |
| Asian or Pacific Islander | 0.295 | (0.275, 0.315) |
| Black, not Hispanic origin | 0.454 | (0.442, 0.466) |
| Hispanic origin | 0.127 | (0.112, 0.143) |
| Null/unknown | 0.539 | (0.530, 0.549) |
| BMI category vs normal weight (18.5 – 24.9 kg/m2) | ||
| Underweight (< 18.5 kg/m2) | 0.261 | (0.235, 0.287) |
| Overweight (25.0 – 29.9 kg/m2) | −0.053 | (−0.062, −0.043) |
| Obese class 1 (30.0 – 34.9 kg/m2) | −0.025 | (−0.036, −0.015) |
| Obese class 2 (35.0 – 39.9 kg/m2) | 0.024 | (0.012, 0.037) |
| Obese class 3 (> 39.9 kg/m2) | 0.112 | (0.098, 0.125) |
| Null/unknown | 0.381 | (0.358, 0.405) |
| Ascites (within 30 days) | 1.353 | (1.301, 1.405) |
| Bleeding disorder requiring hospitalization | 0.291 | (0.275, 0.306) |
| Diabetes mellitus vs none | ||
| Oral/non-insulin agent | −0.010‡ | (−0.021, 0.001)‡ |
| Insulin | 0.170 | (0.156, 0.184) |
| Dialysis or hemofiltration (within 2 weeks) | 0.405 | (0.374, 0.436) |
| Disseminated cancer | 0.396 | (0.376, 0.415) |
| Dyspnea (within 30 days) vs none | ||
| Moderate exertion | 0.159 | (0.145, 0.174) |
| At rest | 0.502 | (0.453, 0.550) |
| Functional health status before operation vs independent | ||
| Partially dependent | 0.730 | (0.709, 0.751) |
| Totally dependent | 0.535 | (0.489, 0.581) |
| Congestive heart failure (within 30 days) | 0.654 | (0.619, 0.689) |
| Severe COPD | 0.366 | (0.350, 0.381) |
| Blood pressure >140/90 mmHg or taking antihypertension medication | 0.007‡ | (−0.001, 0.015)‡ |
| Systemic sepsis (within 48 hours) vs none | ||
| SIRS | 0.540 | (0.521, 0.558) |
| Sepsis | 1.724 | (1.703, 1.745) |
| Septic shock | 4.043 | (3.982, 4.103) |
| Acute renal failure (rising creatinine to >3 mg/dL within 24 hours) | 0.993 | (0.937, 1.048) |
| Cigarette smoker (within 1 year) | 0.045 | (0.036, 0.054) |
| Steroid use for chronic condition | 0.211 | (0.194, 0.227) |
| Transfusion of packed RBCs (within 72 hours) | 1.065 | (1.034, 1.096) |
| Ventilator-dependent (within 48 hours) | 3.507 | (3.437, 3.576) |
| Open wound with or without infection | 0.671 | (0.651, 0.690) |
| >10% loss of body weight (within 6 months) | 0.701 | (0.675, 0.727) |
| Transfer status vs admitted directly from home | ||
| Acute care hospital | 0.709 | (0.693, 0.726) |
| Chronic care facility | 0.119 | (0.086, 0.151) |
| Primary surgeon specialty vs general surgery | ||
| Gynecology | 0.058 | (0.043, 0.073) |
| Neurosurgery | −0.078 | (−0.092, −0.063) |
| Orthopedic | −0.181 | (−0.191, −0.172) |
| Otolaryngology | 0.187 | (0.158, 0.216) |
| Plastic | 0.179 | (0.150, 0.209) |
| Thoracic | −0.070 | (−0.095, −0.045) |
| Urology | −0.207 | (−0.224, −0.191) |
| Vascular | −0.726 | (−0.740, −0.711) |
| Emergency operation | 0.416 | (0.404, 0.428) |
| ASA class vs I (normal healthy patient) | ||
| II (patient with mild systemic disease) | 0.240 | (0.222, 0.257) |
| III (patient with severe systemic disease) | 0.688 | (0.670, 0.706) |
| IV (patient with severe systemic disease that is a constant threat) | 1.511 | (1.489, 1.534) |
| V (moribund patient who is not expected to survive) | 3.252 | (3.155, 3.350) |
| Work RVU | 0.025 | (0.025, 0.026) |
| CPT-specific mean LOS | 0.794 | (0.792, 0.796) |
Open in a separate window
Abbreviations: ASA class, American Society of Anesthesiologists Physical Status Classification; BMI, body mass index; CI, confidence interval; COPD, chronic obstructive pulmonary disease; CPT, current procedural terminology; LOS, length of stay in days; RBC, red blood cell; RVU, relative value unit; SIRS, systemic inflammatory response syndrome.
*The model’s y-intercept was −1.534 (95% CI: [−1.556, −1.512]). Adjusted R2 = 0.3720. Regression line of observed versus predicted values: intercept = −1.651E-11, slope = 1.000.
†Regression coefficients are reported in days and represent either the change in LOS for having the characteristic vs. not having the characteristic for categorical variables or the change in LOS per 1-unit change in the predictor for continuous variables, after adjusting for all other predictors. Regression coefficients were statistically significant (p<0.05) except where noted.
‡The regression coefficients were not statistically significant for diabetes mellitus requiring oral/non-insulin agent (p=0.0732) and blood pressure greater than 140/90 mmHg or taking antihypertension medication (p=0.0785).
Table 3 shows the results of the multiple linear regression analysis using the SURPAS seven-variable model. All associations were statistically significant (p<0.05). Overall, performance metrics for the SURPAS model were similar to those for the full model; specifically, adjusted R2 was 0.347 (93.3% of that of the full model), the calibration intercept was 1.310E-10, and the calibration slope was 1.00. These results demonstrate that the SURPAS model has similar overall fit, calibration, bias, and discrimination as compared to the full model.
Table 3.
Multiple linear regression analysis of the Surgical Risk Preoperative Assessment System (SURPAS) model for postoperative length of stay up to 30 days (dependent variable) and the seven SURPAS preoperative predictor variables (independent variables) using the analytical cohort (2012 to 2018; N=3,295,028).
| Characteristic* | Regression Coefficient† | 95% CI |
|---|---|---|
| Age | 0.012 | (0.011, 0.012) |
| Functional health status before operation vs independent | ||
| Partially dependent | 1.047 | (1.027, 1.068) |
| Totally dependent | 1.205 | (1.160, 1.250) |
| Primary surgeon specialty vs general surgery | ||
| Gynecology | 0.145 | (0.130, 0.160) |
| Neurosurgery | −0.098 | (−0.113, −0.083) |
| Orthopedic | −0.208 | (−0.217, −0.198) |
| Otolaryngology | 0.211 | (0.182, 0.241) |
| Plastic | 0.237 | (0.207, 0.267) |
| Thoracic | −0.106 | (−0.131, −0.080) |
| Urology | −0.152 | (−0.168, −0.135) |
| Vascular | −0.665 | (−0.678, −0.651) |
| Emergency operation | 0.867 | (0.856, 0.879) |
| ASA physical status classification vs I (normal healthy patient) | ||
| II (patient with mild systemic disease) | 0.262 | (0.245, 0.279) |
| III (patient with severe systemic disease) | 0.865 | (0.848, 0.883) |
| IV (patient with severe systemic disease that is a constant threat) | 2.228 | (2.206, 2.250) |
| V (moribund patient who is not expected to survive) | 5.496 | (5.398, 5.594) |
| Work RVU | 0.012 | (0.012, 0.013) |
| CPT-specific mean LOS | 0.875 | (0.873, 0.877) |
Open in a separate window
Abbreviations: ASA class, American Society of Anesthesiologists Physical Status Classification; CI, confidence interval; CPT, current procedural terminology; LOS, length of stay in days; RVU, relative value unit.
*The model’s y-intercept was −1.186 (95% CI: [−1.206, −1.166]). Adjusted R2 = 0.3470. Regression line of observed versus predicted values: intercept = 1.310E-10, slope = 1.000.
†Regression coefficients are reported in days and represent either the change in LOS for having the characteristic vs. not having the characteristic for categorical variables or the change in LOS per 1-unit change in the predictor for continuous variables, after adjusting for all other predictors. All reported regression coefficients were statistically significant (p<0.05).
In the internal chronological validation of the SURPAS model, the adjusted R2 in the test dataset was 97.1% of that in the training dataset (0.3389 vs. 0.3489, respectively). The calibration intercept was further from zero (−0.2011 vs. 2.339E-11), indicating slightly increased systematic bias (i.e., LOS slightly underestimated across all predictions). The calibration slope was slightly further from 1.0 (0.994 vs. 1.000), indicating slightly decreased discrimination (i.e., LOS slightly overestimated at shorter observed LOS and slightly underestimated at longer observed LOS). Collectively, the intercept and slope were slightly further from their target values for the test dataset versus the training dataset, thus indicating a slight decrease in calibration performance.
We performed a sensitivity analysis, including the patients with LOS>30 days. As might be expected, the performances of the models declined. The adjusted R2 for the full model declined from 0.3720 to 0.2885, and for the SURPAS model from 0.3470 to 0.2594. In the internal validation of the SURPAS model, the R2 declined in the training dataset from 0.3489 to 0.2546, and in the test dataset from 0.3389 to 0.3087. The intercepts and slopes of the different models did not change appreciably. Details of the full and SURPAS models including patients with LOS>30 days are presented in Supplemental Tables S1 and S2.
DISCUSSION
The results of this study indicate that the parsimonious seven-variable SURPAS model predicts postoperative LOS for inpatient surgical procedures almost as accurately as a full model using all 28 ACS-NSQIP preoperative nonlaboratory variables. Additionally, the SURPAS model performed almost as accurately in the test dataset as in the training dataset in internal chronological validation testing. Therefore, we conclude that postoperative LOS up to 30 days for inpatient surgical procedures can be added to the existing SURPAS tool as an additional postoperative outcome predicted by the model. In addition to estimating a patient’s postoperative LOS, the SURPAS model also estimates a number of binary outcomes, including 30-day mortality, overall morbidity, unplanned readmission, non-home discharge, and eight specific postoperative complications (infection, cardiac, pulmonary, renal, urinary tract infection, venous thromboembolism, bleeding, and stroke). The SURPAS model’s ability to predict these outcomes varies by outcome, as assessed by the c-index: mortality (0.928), non-home discharge (0.912), pulmonary (0.893), bleeding (0.875), cardiac (0.871), renal (0.863), stroke (0.840), overall morbidity (0.823), infection (0.805), venous thromboembolism (0.788), urinary tract infection (0.776), and readmission (0.728).
This study is unique because it reports on a preoperative prediction model for postoperative LOS that is applicable to a broad inpatient surgical population. Postoperative LOS is an important indicator of patient severity, care quality, and institutional efficiency and is frequently used to compare surgical intervention effectiveness, inform cost analyses and operational evaluations, and support resource and discharge planning.5–11 Most of the existing literature focuses on risk factors for prolonged LOS, and available prediction models are limited to specific operations and/or patient populations. Additionally, the use of limited datasets, predictor variables that do not represent routinely collected data, and binary LOS outcomes (e.g., LOS ≥ 14 days or upper quartile of a LOS distribution) result in models that lack generalizability or utility outside of the context in which they were developed.5 In this study, a universal prediction model was developed using the large ACS-NSQIP dataset, which encompasses a national sample across a broad range of surgical specialties and procedures. Additionally, the SURPAS model (1) uses a parsimonious set of medically relevant, readily available predictor variables; (2) has been successfully implemented into an institution’s electronic health record; and (3) is available online for other institutions to use.41 Lastly, the choice to model postoperative LOS as a continuous, rather than binary, outcome in this study provides more information to patients, providers, and healthcare administrators by estimating the patient’s exact LOS rather than just indicating whether the patient might have a “prolonged LOS.”
In interpreting the findings of this study, we need to address the clinical meaningfulness of the SURPAS model’s adjusted R2 value of 0.347 and its difference of 6.7% with the full model and the 2.9% difference between the test and training datasets. A potential limitation of this study is that the SURPAS model’s adjusted R2 value of 0.347 indicates that the model accounts for only 34.7% of the variability in observed LOS for inpatient surgical procedures. It has been reported that studies trying to explain human behavior typically have R2 values <50%.44 In the few studies identified that treat LOS as a continuous variable, Achanta et al found R2 values of 0.28 to 0.44 for predicting postoperative LOS in six different operations in emergency general surgery,12 while Verburg et al found an R2 of 0.266 in predicting LOS in ICU survivors.45 These R2 values are similar to what we found. Although this result appears relatively low in relation to the maximum achievable R2 value of 100%, postoperative LOS is a complex outcome influenced not only by preoperative patient characteristics but also more substantially by intra- and post-operative factors (e.g., complications, surgical and patient care quality, socioeconomic factors) as well as a multitude of hospital-level characteristics and process elements (e.g., hospital and procedure volume, payer mix, discharge protocols, regional variations and colloquial factors).8–10 Including preoperative hospital-level factors, such as hospital/procedure volume or payer mix, may have increased the adjusted R2 value in this study; however, these variables are not available in the ACS-NSQIP database. Regarding the small 6.7% difference in the adjusted R2 between the SURPAS and full models, and the 2.9% difference in the adjusted R2 between the test and training datasets, we are not aware of any publications addressing the clinical relevance of differences in R2 of different prediction models. However, there have been publications about minimal clinically important differences (MCID) in other research contexts indicating differences of 15–20% are considered to be MCID.46
CONCLUSIONS
The parsimonious SURPAS model predicts postoperative LOS up to 30 days for inpatient surgical procedures as accurately as a model using all 28 ACS-NSQIP preoperative nonlaboratory variables and has shown acceptable internal chronological validation. Therefore, prediction of postoperative LOS up to 30 days for inpatient surgical procedures can be integrated into the existing SURPAS tool to preoperatively predict this important outcome and potentially support presurgical patient counseling, surgical case planning and risk mitigation, and healthcare institution cost projections and resource planning.
Supplementary Material
1
Click here to view.(32K, docx)
Acknowledgments:
We would like to thank Drs. Catherine Battaglia and Allan V Prochazka with the University of Colorado Anschutz Medical Campus Clinical Science Program for providing essential feedback on the study proposal and manuscript preparation.
Funding/Financial Support:
This publication is supported in part by National Institutes of Health/National Center for Advancing Translational Sciences Colorado Clinical and Translational Science Award (grant number UL1 TR002535). Contents are the authors’ sole responsibility and do not necessarily represent official National Institutes of Health views.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest/Disclosures:
Emily M Mason reports having received salary and educational assistance from Terumo BCT, Lakewood, Colorado. Terumo BCT had no involvement in study conduct or publication. All other authors reported no competing interests or potential conflicts of interest. The ACS-NSQIP and participating hospitals are the source of these data; the ACS has not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.
REFERENCES
1. Chona D, Lakomkin N, Bulka C, et al. Predicting the post-operative length of stay for the orthopaedic trauma patient. Int Orthop. 2017;41(5):859–868. [PubMed] [Google Scholar]
2. Elsamna ST, Hasan S, Shapiro ME, Merchant AM. Factors Contributing to Extended Hospital Length of Stay in Emergency General Surgery(dagger). J Invest Surg. 2021;34(12):1399–1406. [PubMed] [Google Scholar]
3. Aasen DM, Wiesen BM, Singh AB, et al. Systematic Review of Preoperative Risk Discussion in Practice. J Surg Educ. 2020;77(4):911–920. [PubMed] [Google Scholar]
4. Wiesen BM, Bronsert MR, Aasen DM, et al. Use of Surgical Risk Preoperative Assessment System (SURPAS) and Patient Satisfaction During Informed Consent for Surgery. J Am Coll Surg. 2020;230(6):1025–1033 e1021. [PubMed] [Google Scholar]
5. Stone K, Zwiggelaar R, Jones P, Mac Parthaláin N. A systematic review of the prediction of hospital length of stay: Towards a unified framework. PLOS Digit Health. 2022;1(4): e0000017. [PMC free article] [PubMed] [Google Scholar]
6. Baek H, Cho M, Kim S, Hwang H, Song M, Yoo S. Analysis of length of hospital stay using electronic health records: A statistical and data mining approach. PLoS One. 2018;13(4):e0195901. [PMC free article] [PubMed] [Google Scholar]
7. Cohen ME, Bilimoria KY, Ko CY, Richards K, Hall BL. Variability in length of stay after colorectal surgery: assessment of 182 hospitals in the national surgical quality improvement program. Ann Surg. 2009;250(6):901–907. [PubMed] [Google Scholar]
8. Giambrone GP, Smith MC, Wu X, et al. Variability in length of stay after uncomplicated pulmonary lobectomy: is length of stay a quality metric or a patient metric?dagger. Eur J Cardiothorac Surg. 2016;49(4):e65–71. [PMC free article] [PubMed] [Google Scholar]
9. Hughes AH, Horrocks D Jr., Leung C, Richardson MB, Sheehy AM, Locke CFS. The increasing impact of length of stay “outliers” on length of stay at an urban academic hospital. BMC Health Serv Res. 2021;21(1):940. [PMC free article] [PubMed] [Google Scholar]
10. Peterson ED, Coombs LP, Ferguson TB, et al. Hospital variability in length of stay after coronary artery bypass surgery: results from the Society of Thoracic Surgeon’s National Cardiac Database. Ann Thorac Surg. 2002;74(2):464–473. [PubMed] [Google Scholar]
11. Siddique SM, Tipton K, Leas B, et al. Interventions to Reduce Hospital Length of Stay in High-risk Populations: A Systematic Review. JAMA Netw Open. 2021;4(9):e2125846. [PMC free article] [PubMed] [Google Scholar]
12. Achanta A, Nordestgaard A, Kongkaewpaisan N, et al. Most of the variation in length of stay in emergency general surgery is not related to clinical factors of patient care. J Trauma Acute Care Surg. 2019;87(2):408–412. [PubMed] [Google Scholar]
13. Carr CJ, Mears SC, Barnes CL, Stambough JB. Length of Stay After Joint Arthroplasty is Less Than Predicted Using Two Risk Calculators. J Arthroplasty. 2021;36(9):3073–3077. [PMC free article] [PubMed] [Google Scholar]
14. Dasenbrock HH, Liu KX, Devine CA, et al. Length of hospital stay after craniotomy for tumor: a National Surgical Quality Improvement Program analysis. Neurosurg Focus. 2015;39(6):E12. [PubMed] [Google Scholar]
15. DeLuzio MR, Keshava HB, Wang Z, Boffa DJ, Detterbeck FC, Kim AW. A model for predicting prolonged length of stay in patients undergoing anatomical lung resection: a National Surgical Quality Improvement Program (NSQIP) database study. Interact Cardiovasc Thorac Surg. 2016;23(2):208–215. [PubMed] [Google Scholar]
16. Gruskay JA, Fu M, Basques BA, et al. Factors Affecting Length of Stay and Complications After Elective Anterior Cervical Discectomy and Fusion: A Study of 2164 Patients From The American College of Surgeons National Surgical Quality Improvement Project Database (ACS NSQIP). Clin Spine Surg. 2016;29(1):E34–42. [PubMed] [Google Scholar]
17. Park KU, Rubinfeld I, Hodari A, Hammoud Z. Prolonged Length of Stay after Esophageal Resection: Identifying Drivers of Increased Length of Stay Using the NSQIP Database. J Am Coll Surg. 2016;223(2):286–290. [PubMed] [Google Scholar]
18. Schneider AM, Denyer S, Brown NM. Risk Factors Associated With Extended Length of Hospital Stay After Geriatric Hip Fracture. J Am Acad Orthop Surg Glob Res Rev. 2021;5(5):e2100073. [PMC free article] [PubMed] [Google Scholar]
19. Smith CG, Davenport DL, Hoffman MR. Characteristics Associated with Prolonged Length of Stay after Myomectomy for Uterine Myomas. J Minim Invasive Gynecol. 2019;26(7):1303–1310. [PubMed] [Google Scholar]
20. Stefanova D, Ullmann TM, Limberg J, et al. Risk Factors for Prolonged Length of Stay and Readmission After Parathyroidectomy for Renal Secondary Hyperparathyroidism. World J Surg. 2020;44(11):3751–3760. [PubMed] [Google Scholar]
21. Tornese D, Robustelli A, Ricci G, Rancoita PMV, Maffulli N, Peretti GM. Predictors of postoperative hospital length of stay after total knee arthroplasty. Singapore Med J. 2021. [PMC free article] [PubMed] [Google Scholar]
22. Zeidan M, Goz V, Lakomkin N, Spina N, Brodke DS, Spiker WR. Predictors of Readmission and Prolonged Length of Stay After Cervical Disc Arthroplasty. Spine (Phila Pa 1976). 2021;46(8):487–491. [PubMed] [Google Scholar]
23. Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833–842 e831–833. [PMC free article] [PubMed] [Google Scholar]
24. Cohen ME, Liu Y, Ko CY, Hall BL. An Examination of American College of Surgeons NSQIP Surgical Risk Calculator Accuracy. J Am Coll Surg. 2017;224(5):787–795 e781. [PubMed] [Google Scholar]
25. Liu Y, Cohen ME, Hall BL, Ko CY, Bilimoria KY. Evaluation and Enhancement of Calibration in the American College of Surgeons NSQIP Surgical Risk Calculator. J Am Coll Surg. 2016;223(2):231–239. [PubMed] [Google Scholar]
26. Liu Y, Cohen ME, Ko CY, Bilimoria KY, Hall BL. Considerations in Releasing Equations for the American College of Surgeons NSQIP Surgical Risk Calculator: In Reply to Wanderer and Ehrenfeld. J Am Coll Surg. 2016;223(4):674–675. [PubMed] [Google Scholar]
27. Hammermeister KE, Henderson WG, Bronsert MR, Juarez-Colunga E, Meguid RA. Bringing Quantitative Risk Assessment Closer to the Patient and Surgeon: A Novel Approach to Improve Outcomes. Ann Surg. 2016;263(6):1039–1041. [PubMed] [Google Scholar]
28. Meguid RA, Bronsert MR, Juarez-Colunga E, Hammermeister KE, Henderson WG. Surgical Risk Preoperative Assessment System (SURPAS): I. Parsimonious, Clinically Meaningful Groups of Postoperative Complications by Factor Analysis. Ann Surg. 2016;263(6):1042–1048. [PubMed] [Google Scholar]
29. Meguid RA, Bronsert MR, Juarez-Colunga E, Hammermeister KE, Henderson WG. Surgical Risk Preoperative Assessment System (SURPAS): II. Parsimonious Risk Models for Postoperative Adverse Outcomes Addressing Need for Laboratory Variables and Surgeon Specialty-specific Models. Ann Surg. 2016;264(1):10–22. [PubMed] [Google Scholar]
30. Meguid RA, Bronsert MR, Juarez-Colunga E, Hammermeister KE, Henderson WG. Surgical Risk Preoperative Assessment System (SURPAS): III. Accurate Preoperative Prediction of 8 Adverse Outcomes Using 8 Predictor Variables. Ann Surg. 2016;264(1):23–31. [PubMed] [Google Scholar]
31. Gibula DR, Singh AB, Bronsert MR, et al. Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables. Surgery. 2019;166(5):812–819. [PubMed] [Google Scholar]
32. Henderson WG, Bronsert MR, Hammermeister KE, Lambert-Kerzner A, Meguid RA. Refining the predictive variables in the “Surgical Risk Preoperative Assessment System” (SURPAS): a descriptive analysis. Patient Saf Surg. 2019;13:28. [PMC free article] [PubMed] [Google Scholar]
33. Henderson WG, Rozeboom PD, Bronsert MR, et al. Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas. Surgery. 2021. [PubMed] [Google Scholar]
34. Khaneki S, Bronsert MR, Henderson WG, et al. Comparison of accuracy of prediction of postoperative mortality and morbidity between a new, parsimonious risk calculator (SURPAS) and the ACS Surgical Risk Calculator. Am J Surg. 2020;219(6):1065–1072. [PubMed] [Google Scholar]
35. Rozeboom PD, Bronsert MR, Henderson WG, et al. The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery. Am J Surg. 2021;222(3):643–649. [PubMed] [Google Scholar]
36. Rozeboom PD, Bronsert MR, Velopulos CG, et al. A comparison of the new, parsimonious tool Surgical Risk Preoperative Assessment System (SURPAS) to the American College of Surgeons (ACS) risk calculator in emergency surgery. Surgery. 2020;168(6):1152–1159. [PubMed] [Google Scholar]
37. Singh AB, Bronsert MR, Henderson WG, Lambert-Kerzner A, Hammermeister KE, Meguid RA. Accurate Preoperative Prediction of Discharge Destination Using 8 Predictor Variables: A NSQIP Analysis. J Am Coll Surg. 2020;230(1):64–75 e62. [PubMed] [Google Scholar]
38. Bronsert MR, Lambert-Kerzner A, Henderson WG, et al. The value of the “Surgical Risk Preoperative Assessment System” (SURPAS) in preoperative consultation for elective surgery: a pilot study. Patient Saf Surg. 2020;14:31. [PMC free article] [PubMed] [Google Scholar]
39. Lambert-Kerzner A, Ford KL, Hammermeister KE, Henderson WG, Bronsert MR, Meguid RA. Assessment of attitudes towards future implementation of the “Surgical Risk Preoperative Assessment System” (SURPAS) tool: a pilot survey among patients, surgeons, and hospital administrators. Patient Saf Surg. 2018;12:12. [PMC free article] [PubMed] [Google Scholar]
40. Colborn KL. SURPAS: Code repository for SURPAS Risk Model. GitHub. https://github.com/katiecolborn/SURPAS.AccessedNovember 1, 2021. [Google Scholar]
41. SURPAS: Surgical Risk Preoperative Assessment System [Internet]. AgileMD. AccessedJune 21, 2022. Available from: https://surpas.agilemd.com/surpas. [Google Scholar]
42. ACS NSQIP Hospitals [Internet]. American College of Surgeons. AccessedJune 21, 2022. Available from: https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/about-acs-nsqip/participants/. [Google Scholar]
43. ACS NSQIP Participant Use Data File [Internet]. American College of Surgeons. AccessedJune 21, 2022. Available from: https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/participant-use-data-file/. [Google Scholar]
44. How To Interpret R-squared in Regression Analysis [Internet]. Statistics by Jim.AccessedJuly 18, 2022. Available from: https://statisticsbyjim.com/regression/interpret-r-squared-regression/.
45. Verburg IW, de Keizer NF, de Jonge E, Peek N. Comparison of regression methods for modeling intensive care length of stay. PLoS One. 2014;9(10):e109684. [PMC free article] [PubMed] [Google Scholar]
46. Dvir Z. Difference, significant difference and clinically meaningful difference: The meaning of change in rehabilitation. J Exerc Rehabil. 2015;11(2):67–73. [PMC free article] [PubMed] [Google Scholar]
