HIAWare: Speculate Handwriting on Mobile Devices with Built-In Sensors

A variety of sensors are built into intelligent mobile devices. However, these sensors can be used as side channels for inferring information. Researchers have shown that some touchscreen information, such as PIN and unlock pattern, can be speculated by background applications with motion sensors. Those attacks mainly focus on the restricted-area input interface (e.g., virtual keyboard). To date, the privacy risk in the unrestricted-area input interface does not receive sufficient attention. In this paper, we investigate such privacy risk and design an unrestricted-area information speculation framework, called Handwritten Information Awareness (HIAWare). HIAWare exploits the sensors' signals that are affected by handwriting actions to speculate the handwritten characters. To alleviate the impact of different handwriting habits, we utilize the generality patterns of characters. Furthermore, to mitigate the impact of holding posture in handwriting, we propose a user-independent posture-aware approach. As a result, HIAWare can attack any victim without obtaining the victim's information in advance. The experiments show that the speculation accuracy of HIAWare is close to 90.0%, demonstrating the viability of HIAWare.