CNN-Based drug recognition and braille embosser system for the blind

Visual impairments reduce one's ability to perform daily tasks such as taking medicine. While the sighted can use their vision to effortlessly locate and identify drugs, the blind must rely on external assistance to complement their visual sense. Thus, receiving appropriate aid at the right time is crucial to avoid the misuse of drugs. We conducted interviews regarding medicine intake with 30 partially or completely blinded persons registered at three supporting facilities. Participants reported limitations of their current methods in finding their medication which led to them taking unintentional irregular doses caused by the lack of aid. Based on the results of the interview, we developed a drug recognition model and braille embosser system for Android smartphones. Using a picture of a medicine taken with a built-in camera, the CNN-based recognition model can classify 11 types of medicines with 99.6% accuracy. In addition, a low-cost braille embosser, which can connect to one's smartphone via Bluetooth, can print the classification results as a braille label for future identification without a smartphone. © 2018. The Korean Institute of Information Scientists and Engineers.