Transparent Ensembles for Covid-19 Prognosis

A natural method aiming at explaining the answers of a black-box model is by means of propositional rules. Nevertheless, rule extraction from ensembles of Machine Learning models was rarely achieved. Moreover, experiments in this context have rarely been evaluated by cross-validation trials. Based on stratified tenfold cross-validation, we performed experiments with several ensemble models on Covid-19 prognostic data. Specifically, we compared the characteristics of the propositional rules generated from: Random Forests; Shallow Trees trained by Gradient Boosting; Decision Stumps trained by several variants of Boosting; and ensembles of transparent neural networks trained by Bagging. The Discretized Interpretable Multi Layer Perceptron (DIMLP) allowed us to generate rules from all the used ensembles by transforming Decision Trees into DIMLPs. Our rule extraction technique simply determines whether an axis-parallel hyperplane is discriminative or not, with a greedy algorithm that progressively removes rule antecedents. Rules extracted from Decision Stumps trained by modest Adaboost were the simplest with the highest fidelity. Our best average predictive accuracy result was equal to 96.5%. Finally, we described a particular ruleset extracted from an ensemble of Decision Stumps and it turned out that the rule antecedents seem to be plausible with respect to several recent works related to the Covid-19 virus.