Transformer based Early Classification for Real-time Human Activity Recognition in Smart Homes

Human activity recognition (HAR) plays a key role in intelligent systems. Ambient sensors are utilized to avoid privacy concerns and to collect data streams in a less intrusive manner in smart homes. In scenarios requiring immediate intervention, the systems must perform HAR in real-time. Since it is difficult to segment the exact transition point in real-time, data unrelated to the target activity can appear at the beginning of the time series, which we call unrefined data. It leads us to a new challenge that the HAR model recognizes a user's activity as early as possible with unrefined data. To explore the impact of unrefined data on real-time HAR, we design an experimental system that consists of a Transformer-based filtering network and an LSTM-based early classifier. We evaluate the experimental system with 3 public datasets collected on testbeds with ambient sensors installed. Our results reveal that unrefined data degrade HAR performance in terms of accuracy and earliness, and the use of the filtering network that filters out unrefined data improves recognition performance.