Evaluating methods for risk prediction of Covid-19 mortality in nursing home residents before and after vaccine availability: a retrospective cohort study.

Publication date: Mar 27, 2024

SARS-CoV-2 vaccines are effective in reducing hospitalization, COVID-19 symptoms, and COVID-19 mortality for nursing home (NH) residents. We sought to compare the accuracy of various machine learning models, examine changes to model performance, and identify resident characteristics that have the strongest associations with 30-day COVID-19 mortality, before and after vaccine availability. We conducted a population-based retrospective cohort study analyzing data from all NH facilities across Ontario, Canada. We included all residents diagnosed with SARS-CoV-2 and living in NHs between March 2020 and July 2021. We employed five machine learning algorithms to predict COVID-19 mortality, including logistic regression, LASSO regression, classification and regression trees (CART), random forests, and gradient boosted trees. The discriminative performance of the models was evaluated using the area under the receiver operating characteristic curve (AUC) for each model using 10-fold cross-validation. Model calibration was determined through evaluation of calibration slopes. Variable importance was calculated by repeatedly and randomly permutating the values of each predictor in the dataset and re-evaluating the model’s performance. A total of 14,977 NH residents and 20 resident characteristics were included in the model. The cross-validated AUCs were similar across algorithms and ranged from 0. 64 to 0. 67. Gradient boosted trees and logistic regression had an AUC of 0. 67 pre- and post-vaccine availability. CART had the lowest discrimination ability with an AUC of 0. 64 pre-vaccine availability, and 0. 65 post-vaccine availability. The most influential resident characteristics, irrespective of vaccine availability, included advanced age (≥ 75 years), health instability, functional and cognitive status, sex (male), and polypharmacy. The predictive accuracy and discrimination exhibited by all five examined machine learning algorithms were similar. Both logistic regression and gradient boosted trees exhibit comparable performance and display slight superiority over other machine learning algorithms. We observed consistent model performance both before and after vaccine availability. The influence of resident characteristics on COVID-19 mortality remained consistent across time periods, suggesting that changes to pre-vaccination screening practices for high-risk individuals are effective in the post-vaccination era.

Open Access PDF

Concepts Keywords
Canada Cohort study
Hospitalization COVID-19
July Electronic Health Record
Models Long-term care
Machine learning
Nursing home
Older adults
Prediction modeling


Type Source Name
disease MESH Covid-19
disease VO vaccine
disease VO effective
disease VO population
disease VO Canada
drug DRUGBANK Flunarizine
disease VO time
disease VO vaccination
disease MESH Long Covid
pathway REACTOME Reproduction
disease VO dose
disease MESH death
drug DRUGBANK Serine
disease IDO susceptibility
disease VO efficient
disease IDO facility
drug DRUGBANK Trestolone
drug DRUGBANK Esomeprazole
disease VO frequency
disease MESH infections
drug DRUGBANK Ranitidine
disease IDO infection
disease MESH dementia

Original Article

(Visited 1 times, 1 visits today)