Search
Close this search box.

A multimodal stacked ensemble model for cardiac output prediction utilizing cardiorespiratory interactions during general anesthesia – Scientific Reports

In this study, we proposed a multimodal stacking ensemble that combines data from non-invasive cardiovascular monitoring and MV parameters, including SpO2 and EtCO2. A fundamental principle of the proposed model is that stacking makes the prediction accuracy better than that of a single machine learning algorithm, and stacking several algorithms significantly improves the prediction accuracy. We demonstrate that the multimodal stacked ensemble model predicts accurate and valid CO values with marginal bias and a narrow CO limit of agreement compared with those obtained using pulse contour technique devices.

Ensemble stacking regression leverages multimodal information gathered from anesthesia machine and patient monitors, deriving benefits from the RF, GBM, and XGBoost base models. It effectively captures the nonlinear relationships in the interplay between the heart and lungs during positive-pressure ventilation. Non-linear interactions of cardiopulmonary features may explain why GLM base models exhibit inferior performance compared with other base models in hemodynamic and respiratory data. An additional advantage of ensemble stacking regression is the interpretability of the final predictions obtained using the GLM metalearner. Furthermore, it demonstrates robustness by harnessing the strengths of the multiple base models.

The Bland–Altman plot is widely recognized as the standard statistical method for assessing the agreement between two consecutive measurements of the same clinical variable21. When using a clinical CO measurement device, Bland–Altman plots do not indicate whether the LoAs are acceptable22. For example, an agreement limitation of ± 1 L/min may not be acceptable in patients with low CO syndrome. Additionally, our results include percentage difference plots demonstrating that multimodal stacked ensemble models accurately predict CO, with predictions falling within the acceptable clinical criteria (± 30%) of the proportional mean difference when compared those obtained using the Vigileo and EV1000 devices.

In previous studies the calibrated pulse wave analysis device EV1000 has proven to be accurate and consistent, and was thus used for our reference CO measurement. The results showed good agreement and interchangeability with TD CO measurement, with a bias of − 0.07 L/min, LoA of 2.0 L/min, and a percentage of 29%23. In addition, the uncalibrated FloTrac/Vigileo provides clinically acceptable accuracy under stable hemodynamic conditions, with an average error below 30% for CO compared with that obtained via TD11. However, severe sepsis and septic shock uncalibrated FloTrac/Vigileo vs. TD revealed no clinically acceptable tracking capability with a bias of − 0.86 L/min, LoA of − 4.48 to 2.77 L/min, and a percentage error of 48%24.

Our study was based entirely on non-cardiac surgery. Accordingly, NIBP was selected because it is a standard measurement for patients with ASA I and II and for intermediate-risk surgery. In addition, NIBP appears to be in acceptable agreement with invasively measured BP in patients with cardiogenic shock25, MV, and arrhythmia26. However, NIBP is not always well calibrated with invasive BP measurement, particularly in hypothermia and pronounced hypotension25. Although invasive BP, known as beat-by-beat measurement, is considered the gold standard method of diagnosis, NIBP is associated with fewer complications, particularly catheter-associated artery pseudoaneurysms, occlusions, and infections27. Occasionally, a measurement can be inaccurate owing to kinking or damping of the arterial line.

The HR was extracted from finger photoplethysmography and may represent acceptable accuracy based on electrocardiography (ECG) during normal breathing. Photoplethysmography and ECG-derived heart rates can differ moderately, and photoplethysmography shows an advantage in monitoring changes in ITP caused by ventilation, sleep apnea, and even changes in respiratory rate during deep breathing28,29.

Using the respiratory rate based on capnography, the expiratory tidal volume, and the expiratory Vm enabled us to obtain the exact delivered volume per breathing cycle recorded in the anesthesia machine (Fig. 6a–c). Noteworthy differences between the set and delivered tidal volumes have been demonstrated in several clinical situations, such as patient lung size, lung compliance, airway resistance, and maintenance of spontaneous breathing during general anesthesia through invasively assisted spontaneous ventilation30,31.

Visualizing cardiopulmonary interactions and variable importance in a multimodal stacked ensemble model

Providing decision support using a functional hemodynamic machine learning model based on the complex relationship between the heart and lungs during general anesthesia should be understood by the medical environment. The predictability of the model was quantified in our work using partial dependence plots (PDPs)32, model parameter importance, and interaction variables33.

The symmetric matrix, derived from the calculation of variable importance and interactions using the RF model, was utilized to visualize the interaction variables in Table Fig. 3c, importance variables in Fig. 3b, and to construct a network graph in Fig. 3a. Variable importance is assessed exclusively based on changes in MSE. In difference, variable interactions are evaluated using the square root of the mean unnormalized version of the H-statistic, yielding a value on a scale of 0 to 1. This approach reduces the identification of spurious interactions and presents results by quantifying changes in the RMSE, which are measured on the same scale as CO in L/min. An RF model incorporating NIBP/HR/MV/SpO2/EtCO2 and CO measured by the Vigileo device was chosen to visualize the interaction and importance variables because it displayed the highest performance, with an R2 of 0.973 and an MSE of 0.074 compared to other base models.

Figure 3
figure 3

(a) The two-way interaction (Vint) represents the unnormalized Friedman’s H-statistic between variables, depicted by connecting lines in the RF model for predicting the CO. The stronger the interaction, the thicker and darker the indigo line. The node’s size and green intensity indicate the variable’s importance (Vimp). (b) Contributions of explanatory variables to the RF model, measured in mean squared error “%IncMSE”. (c) The table matrix presents the numerical values of the unnormalized Friedman’s H-statistic, indicating the interacting variables within the RF model for predicting the CO.

All demographic, hemodynamic, and respiratory parameters displayed interactions to varying degrees with a range of H-statistic values (Fig. 3a and c). Hence, these plots facilitate the interpretation of cardiopulmonary interactions, particularly concerning total interactions and interactions between pairs of features, where one feature remains constant while others change, thereby influencing the accuracy of the cardiac output prediction. Demographic and hemodynamic variables, specifically weight and HR, were identified as the most important interactions, exhibiting an H-statistic value of 0.091. This finding suggests that an increase in the accuracy of the CO prediction corresponds to a reduction in the RMSE of 0.091 L/min. The constant pairs variable, HR, demonstrated the strongest reciprocal interactions with age (H-statistic = 0.058), NIBP-SBP (H-statistic = 0.045), height (H-statistic = 0.07), and EtCO2 (H-statistic = 0.056). The six variables that contributed the most to the prediction of CO in the RF model were HR, age, height, weight, NIBP-SBP, and minute volume (Fig. 3a and b).

The results of our study were consistent with well-established data demonstrating that CO levels decrease with age by approximately 1% per year after the third decade (Fig. 4a). Age-related decline in the stroke index is accompanied by decreased body size and HR, which reduces CO34. We found the exact relationship between body size and CO in a straight-line regression, as observed in the last century35 (Fig. 4c,d). According to our findings, in females, one-way PDPs from the RF, GBM, and XGBoost models showed a decrease in CO of approximately 10% compared to those in males during intraoperative measurements. However, this difference was smaller than the 22% difference reported during the resting state36 (Fig. 4b).

Figure 4
figure 4

Partial dependence plots for variables in the multimodal stacking ensemble model for CO measured by Vigileo monitoring device. Partial Dependence Multimodel Plot gives a graphical depiction of the distributed random forest (DRF), gradient boosting machine (GBM), generalized linear model (GLM), and extreme gradient boosting (XGBoost). The effect of a variable is measured as the change in the mean cardiac output. HR plethysmographic heart rate, NIBP SBP systolic non-invasive blood pressure.

HR is crucial to determining the diastolic filling time, influencing the SV via the Frank–Starling mechanism. For cardiopulmonary interactions during MV, venous return can be reduced, which can further compromise diastolic filling, particularly at high heart rates. Our study revealed a linear relationship between CO and HR up to 90/min, where deceleration began (Fig. 4e). Early curve deceleration is well documented in impaired right heart filling37. However, here, this may have been influenced by factors, such as autonomic nervous system activity, blood volume, and heart contractility, which were beyond the scope of this study.

The relationships between SBP, DBP, and CO during general anesthesia are complex and dynamic. In our study, we observed an increase in SBP corresponding to an increase in CO of up to 120 mmHg following the onset of the deceleration curve (Fig. 4f). The decreased CO level during high intraoperative SBP may be caused by increased vascular resistance, stiffened large arteries38, and reduced SV owing to elevated afterload. Our study demonstrated a decrease in DBP with a marginal increase in CO (Fig. 5a). An increase in pulse pressure might elucidate the observed increase in CO. An increase in SV owing to volume substitution results in increased CO, causing an increase in pulse pressure. Cardiopulmonary interactions and additional interventions such as vasopressor administration or adjustments to ventilator settings may play a substantial role. Additionally, the nonlinear relationship between pulse pressure, cardiac index (CI), and deceleration curve starting at a CI of 3 L/min/m2 has been well documented39.

Figure 5
figure 5

Partial dependence plots for variables in the multimodal stacking ensemble model for CO measured by Vigileo monitoring device. Partial dependence multimodel plot gives a graphical depiction of the distributed random forest (DRF), gradient boosting machine (GBM), generalized linear model (GLM), and extreme gradient boosting (XGBoost). The effect of a variable is measured as the change in the mean cardiac output. NIBPDBP diastolic non-invasive blood pressure, FiO2 fraction of inspired oxygen, SPO2 oxygen saturation, EtCO2 infrared spectrometry capnography, which measures end-tidal CO2, PIP peak inspiratory pressure, PEEP positive end-expiratory pressure.

One-way PDPs revealed an inverse relationship between CO and airway pressure (Fig. 5e). A decrease in SV and venous return is the primary mechanism by which increasing airway pressure reduces CO. The application of airway pressure levels at 10, 20, and 30 cm H2O led to a variation in the CI between + 6% and − 43%, which was associated with corresponding changes in the SV index (p < 0001, r2 = 0.89)40. Our findings align with those of earlier studies, as they indicated an increase in airway pressure during lung inflation and a reduction in CO at a rate of 0.5 L/min per 10 mbar increase in PIP.

PEEP increases ITP during the entire respiratory cycle to restore normal end-expiratory lung volume during MV. Increasing the PEEP levels allowed for greater lung expansion. PEEP during MV may also displace blood from the pulmonary circulation, increase mean systemic pressure, reduce venous return, and decrease CO and tissue perfusion41. Our model exhibits a decrease in the CO rate of 0.1 L/min by raising PEEP to 2.5 mbar (Fig. 5f). This decrease in CO with increasing PEEP in a curvilinear relationship has been previously reported42.

A reduction in TV increases CO; nevertheless, the degree of improvement in hemodynamics depends largely on ITP43. Reducing the tidal volume increases chest wall compliance by decreasing ITP during MV and increasing venous return, leading to increased left ventricular preload and CO. This is consistent with our finding; our model showed an increase in CO of 0.03 L/min per 1 mL/kg of TV reduction (Fig. 6b). A tidal volume > 15 mL/kg markedly decreases HR and blood pressure and reduces CO44. However, we could not evaluate this observation with limited training data for tidal volumes > 15 mL/kg, and a machine learning model could not make meaningful predictions.

Figure 6
figure 6

Partial dependence plots for variables in the multimodal stacking ensemble model for CO measured by Vigileo monitoring device. Partial dependence multimodel plot gives a graphical depiction of the distributed random forest (DRF), gradient boosting machine (GBM), generalized linear model (GLM), and extreme gradient boosting (XGBoost). The effect of a variable is measured as the change in the mean cardiac output. RR respiratory rate based on capnography, TV expiratory tidal volume, Vm expiratory minute volume.

Changes in exhaled carbon dioxide during general anesthesia with stable ventilation correspond to changes in CO and metabolic CO2 production45. At ETCO2 levels > 30 mmHg, RF, GBM, and XGBoost models predict a satisfactory CO increase of 0.5 L/min per 10 mmHg of ETCO2 (Fig. 5d). A similar correlation between ETCO2 and CO has been reported in previous studies46. An animal model during cardiopulmonary resuscitation showed a correlation coefficient of 0.79 between EtCO2 and CI47. This finding is consistent with that of the GLM model. However, the GLM model had a lower performance than that of the RF, GBM, and XGBoost models and had less training data with EtCO2 < 30 mmHg.

A decline in SpO2 was observed with decreasing CO in all base models in our study (Fig. 5c). Decreased CO caused by cardiopulmonary interactions is the primary factor in the reduction of arterial oxygen content observed during MV48. Hypovolemia and vasodilation, which are commonly observed during general anesthesia, may also contribute to this phenomenon. However, our data did not allow us to determine whether the increased inspired O2 fraction reflected an increase in CO (Fig. 5b). It is widely recognized that increases in FiO2 at fixed values of CO fail to detect conditions of low oxygen supply during central venous oxygen saturation49.

This study may be more compelling if the model was applied to a dataset that included direct CO measurements obtained through thermodilution using a pulmonary artery catheter. Nevertheless, the interpretability of the developed multimodal stacking ensemble is a notable strength of the proposed system. By offering valuable insights into the interpretation of the model, we deepen our understanding of all purely physiological inputs implicated in CO estimation. This not only enhances scholarly comprehension within the discipline, but also promotes the endorsement and integration of the system among healthcare practitioners. The architecture of this model aligns with the characteristics of “locked” algorithms as defined in the proposed regulatory framework for modifications to Artificial intelligence/machine learning (AI/ML)-based Software as a Medical Device by the food and drug administration (FDA)50. Training the complex algorithm with numerical data enhanced its versatility, allowing the model to be saved, exported, and deployed in diverse medical environments for production use.

Further limitations of this study are as follows. The data analyzed was from one source only and focused solely on adult patients. During data mining, we could not find synchronized records of sudden blood loss or vasoactive infusions. This limitation has an impact on our model’s ability to assess fluid responsiveness and requires thorough evaluation when our model undergoes testing in real-time general anesthesia scenarios. The perioperative clinical information dataset contained data on estimating intraoperative blood loss and cumulative intraoperative use of vasoactive medications (ephedrine, phenylephrine, and epinephrine). However, this information lacks a recorded time, making it unsuitable for inclusion in our model. Mechanical ventilation without spontaneous effort may affect hemodynamics differently; nonetheless, the ventilation modes were not documented in the VitalDB data, leading to their exclusion from this study. In addition, the small number of patients with obstructive or restrictive lung diseases made it difficult to include them in the data subset. Although constant ventilation was ensured during surgery in this study, it is important to recognize that the period from the onset of anesthesia to the start of surgery and the time between the end of surgery and extubation are important for comprehending the influence of cardiopulmonary interactions on hemodynamics. During extubation or weaning, spontaneous inspiratory efforts in patients with obstructive and restrictive lung disease may strongly decrease CO by increasing the left ventricular afterload, especially if left ventricular function is already impaired43. Our model should be improved in the future to address these cardiopulmonary interactions.