An acupoint health care system with real-time acupoint localization and visualization in augmented reality

Acupressure is a noninvasive method in traditional Chinese medicine to increase blood circulation, relieve discomfort, promote health, and prevent disease. This paper presents a multifunctional acupoint application system that not only provides real-time visualization of acupoints on mobile devices for users’ self-acupressure but also provides health care, education, and entertainment through acupressure. The system allows users to obtain acupoint information to relieve specific discomfort or symptoms. It automatically marks the requested acupoints on a real-time camera stream to guide users to locate acupoints so that even users unfamiliar with acupoints can easily find acupoints for self-acupressure. Three core technologies are used: acupoint localization and knowledge, real-time image recognition, and augmented reality. This paper details the system design and implementation. This system uses Google’s machine learning kit for facial detection to extract the user’s facial features and contours in real time on a mobile device. These contour points are used as landmarks with specific displacements to locate acupoints. As there are many acupoints and various positioning methods, the acupoint location is traditionally described in words through specific feature points and corresponding displacements related to each person’s physiological characteristics. This work proposes a novel acupoint positioning and marking mechanism based on the standard acupoint localization, which formalizes various acupoint positioning methods so that positions can be processed and stored in an acupoint database established in this system. This positioning information allows the proposed mechanism to fulfill real-time visualization of acupoints in augmented reality on the camera stream. Experimental results show that the proposed system locates and marks acupoints with low latency and high accuracy. Moreover, the system also provides recommendations for further medical treatment and a mini-game that tests knowledge of acupoints, which enhances users’ experience of acupoint massage.