Learning Entropy: On Shannon vs. Machine-Learning-Based Information in Time Series

The paper discusses the Learning-based information (L) and Learning Entropy (LE) in contrast to classical Shannon probabilistic Information (I) and probabilistic entropy (H). It is shown that L corresponds to the recently introduced Approximate Individual Sample-point Learning Entropy (AISLE). For data series, then, the LE should be defined as the mean value of L that is finally in proper accordance with Shannon's concept of entropy H. The distinction of L against I is explained by the real-time anomaly detection of individual time series data points (states). First, the principal distinction of the information concept of Ivs.L is demonstrated in respect to data governing law that L considers explicitly (while I does not). Second, it is shown that L has the potential to be applied on much shorter datasets than I because of the learning system being pre-trained and being able to generalize from a smaller dataset. Then, floating window trajectories of the covariance matrix norm, the trajectory of approximate variance fractal dimension, and especially the windowed Shannon Entropy trajectory are compared to LE on multichannel EEG featuring epileptic seizure. The results on real time series show that L, i.e., AISLE, can be a useful counterpart to Shannon entropy allowing us also for more detailed search of anomaly onsets (change points).