Comparing the performance of statistical, machine learning, and deep learning algorithms to predict time-to-event: A simulation study for conversion to mild cognitive impairment

被引:0
|
作者
Billichova, Martina [1 ]
Coan, Lauren Joyce [2 ]
Czanner, Silvester [1 ,2 ]
Kovacova, Monika [1 ]
Sharifian, Fariba [2 ]
Czanner, Gabriela [1 ,2 ]
机构
[1] Slovak Univ Technol Bratislava, Fac Informat & Informat Technol, Bratislava, Slovakia
[2] Liverpool John Moores Univ, Sch Comp Sci & Math, Liverpool, England
来源
PLOS ONE | 2024年 / 19卷 / 01期
关键词
MODELS;
D O I
10.1371/journal.pone.0297190
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Mild Cognitive Impairment (MCI) is a condition characterized by a decline in cognitive abilities, specifically in memory, language, and attention, that is beyond what is expected due to normal aging. Detection of MCI is crucial for providing appropriate interventions and slowing down the progression of dementia. There are several automated predictive algorithms for prediction using time-to-event data, but it is not clear which is best to predict the time to conversion to MCI. There is also confusion if algorithms with fewer training weights are less accurate. We compared three algorithms, from smaller to large numbers of training weights: a statistical predictive model (Cox proportional hazards model, CoxPH), a machine learning model (Random Survival Forest, RSF), and a deep learning model (DeepSurv). To compare the algorithms under different scenarios, we created a simulated dataset based on the Alzheimer NACC dataset. We found that the CoxPH model was among the best-performing models, in all simulated scenarios. In a larger sample size (n = 6,000), the deep learning algorithm (DeepSurv) exhibited comparable accuracy (73.1%) to the CoxPH model (73%). In the past, ignoring heterogeneity in the CoxPH model led to the conclusion that deep learning methods are superior. We found that when using the CoxPH model with heterogeneity, its accuracy is comparable to that of DeepSurv and RSF. Furthermore, when unobserved heterogeneity is present, such as missing features in the training, all three models showed a similar drop in accuracy. This simulation study suggests that in some applications an algorithm with a smaller number of training weights is not disadvantaged in terms of accuracy. Since algorithms with fewer weights are inherently easier to explain, this study can help artificial intelligence research develop a principled approach to comparing statistical, machine learning, and deep learning algorithms for time-to-event predictions.
引用
收藏
页数:20
相关论文
共 50 条
  • [1] A scoping methodological review of simulation studies comparing statistical and machine learning approaches to risk prediction for time-to-event data
    Smith, Hayley
    Sweeting, Michael
    Morris, Tim
    Crowther, Michael
    [J]. DIAGNOSTIC AND PROGNOSTIC RESEARCH, 2022, 6 (01)
  • [2] Machine learning can predict mild cognitive impairment in Parkinson's disease
    Amboni, Marianna
    Ricciardi, Carlo
    Adamo, Sarah
    Nicolai, Emanuele
    Volzone, Antonio
    Erro, Roberto
    Cuoco, Sofia
    Cesarelli, Giuseppe
    Basso, Luca
    D'Addio, Giovanni
    Salvatore, Marco
    Pace, Leonardo
    Barone, Paolo
    [J]. FRONTIERS IN NEUROLOGY, 2022, 13
  • [3] Machine-Learning Algorithms Based on Screening Tests for Mild Cognitive Impairment
    Park, Jin-Hyuck
    [J]. AMERICAN JOURNAL OF ALZHEIMERS DISEASE AND OTHER DEMENTIAS, 2020, 35
  • [4] Comparing the performance of machine learning and deep learning algorithms classifying messages in Facebook learning group
    Huang-Fu, Cheng-Yo
    Liao, Chen-Hsuan
    Wu, Jiun-Yu
    [J]. IEEE 21ST INTERNATIONAL CONFERENCE ON ADVANCED LEARNING TECHNOLOGIES (ICALT 2021), 2021, : 347 - 349
  • [5] Comparison of Machine Learning-based Approaches to Predict the Conversion to Alzheimer?s Disease from Mild Cognitive Impairment
    Franciotti, Raffaella
    Nardini, Davide
    Russo, Mirella
    Onofrj, Marco
    Sensi, Stefano L.
    [J]. NEUROSCIENCE, 2023, 514 : 143 - 152
  • [6] A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer's disease
    Spasov, Simeon
    Passamonti, Luca
    Duggento, Andrea
    Lio, Pietro
    Toschi, Nicola
    [J]. NEUROIMAGE, 2019, 189 : 276 - 287
  • [7] A hybrid machine learning approach for prediction of conversion from mild cognitive impairment to dementia
    Bucholc, Magda
    Titarenko, Sofya
    Ding, Xuemei
    Canavan, Callum
    Chen, Tianhua
    [J]. EXPERT SYSTEMS WITH APPLICATIONS, 2023, 217
  • [8] Machine learning approaches to studying the role of cognitive reserve in conversion from mild cognitive impairment to dementia
    Facal, David
    Valladares-Rodriguez, Sonia
    Lojo-Seoane, Cristina
    Pereiro, Arturo X.
    Anido-Rifon, Luis
    Juncos-Rabadan, Onesimo
    [J]. INTERNATIONAL JOURNAL OF GERIATRIC PSYCHIATRY, 2019, 34 (07) : 941 - 949
  • [9] Predictive performance of machine learning compared to statistical methods in time-to-event analysis of cardiovascular disease: a systematic review protocol
    Suliman, Abubaker
    Masud, Mohammad
    Serhani, Mohamed Adel
    Abdullahi, Aminu S.
    Oulhaj, Abderrahim
    [J]. BMJ OPEN, 2024, 14 (04):
  • [10] Commentary on Comparison of Machine Learning-based Approaches to Predict the Conversion to Alzheimer's Disease from Mild Cognitive Impairment
    Rossini, Paolo Maria
    Miraglia, Francesca
    Vecchio, Fabrizio
    [J]. NEUROSCIENCE, 2023, 514 : 141 - 142