EyeSayCorrect: Eye Gaze and Voice Based Hands-free Text Correction for Mobile Devices

被引：14

作者：

Zhao, Maozheng ^{[1
]}

Huang, Henry ^{[2
]}

Li, Zhi ^{[1
]}

Liu, Rui ^{[1
]}

Cui, Wenzhe ^{[1
]}

Toshniwal, Kajal ^{[1
]}

Goel, Ananya ^{[1
]}

Wang, Andrew ^{[3
]}

Zhao, Xia ^{[4
]}

Rashidian, Sina ^{[5
]}

Baig, Furqan ^{[6
]}

Phi, Khiem ^{[1
]}

Zhai, Shumin

Ramakrishnan, I. V. ^{[1
]}

Wang, Fusheng ^{[1
]}

Bi, Xiaojun ^{[1
]}

机构：

[1] SUNY Stony Brook, Dept Comp Sci, Stony Brook, NY 11794 USA

[2] Tappan Zee High Sch, Orangeburg, NY USA

[3] Ward Melville High Sch, East Setauket, NY USA

[4] Stony Brook Med, Stony Brook, NY USA

[5] Verily Life Sci, Cambridge, MA USA

[6] Univ Illinois, Urbana, IL USA

来源：

IUI'22: 27TH INTERNATIONAL CONFERENCE ON INTELLIGENT USER INTERFACES | 2022年

基金：

美国国家科学基金会;

关键词：

multiniodal interaction; eye gaze; text correction; voice input;

D O I：

10.1145/3490099.3511103

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Text correction on mobile devices usually requires precise and repetitive manual control. In this paper, we present EyeSayCorrect, an eye gaze and voice based hands-free text correction method for mobile devices. To correct text with EyeSayCorrect, the user first utilizes the gaze location on the screen to select a word, then speaks the new phrase. EyeSayCorrect would then infer the user's correction intention based on the inputs and the text context. We used a Bayesian approach for determining the selected word given an eye-gaze trajectory. Given each sampling point in an eye-gaze trajectory, the posterior probability of selecting a word is calculated and accumulated. The target word would be selected when its accumulated interest is larger than a threshold. The misspelt words have higher priors. Our user studies showed that using priors for misspelt words reduced the task completion time up to 23.79% and the text selection time up to 40.35%, and EyeSayCorrect is a feasible hands-free text correction method on mobile devices.

引用

页码：470 / 482

页数：13

共 50 条

[1] Hands-free voice activation of personal communication devices
Bou-Ghazale, SE
Asadi, AO
[J]. 2000 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, PROCEEDINGS, VOLS I-VI, 2000, : 1735 - 1738
[2] Gaze-Directed Hands-Free Interface for Mobile Interaction
Park, Gie-seo
Ahn, Jong-gil
Kim, Gerard J.
[J]. HUMAN-COMPUTER INTERACTION: INTERACTION TECHNIQUES AND ENVIRONMENTS, PT II, 2011, 6762 : 304 - 313
[3] Assessing hands-free interactions for VR using eye gaze and electromyography
Pai, Yun Suen
Dingler, Tilman
Kunze, Kai
[J]. VIRTUAL REALITY, 2019, 23 (02) : 119 - 131
[4] Assessing hands-free interactions for VR using eye gaze and electromyography
Yun Suen Pai
Tilman Dingler
Kai Kunze
[J]. Virtual Reality, 2019, 23 : 119 - 131
[5] Voice as a Mouse Click: Usability and Effectiveness of Simplified Hands-Free Gaze-Voice Selection
Zhao, Darisy G.
Karikov, Nikita D.
Melnichuk, Eugeny V.
Velichkovsky, Boris M.
Shishkin, Sergei L.
[J]. APPLIED SCIENCES-BASEL, 2020, 10 (24): : 1 - 12
[6] Hands-Free Web Browsing: Enriching the User Experience with Gaze and Voice Modality
Sengupta, Korok
Ke, Min
Menges, Raphael
Kumar, Chandan
Staab, Steffen
[J]. 2018 ACM SYMPOSIUM ON EYE TRACKING RESEARCH & APPLICATIONS (ETRA 2018), 2018,
[7] Integration of electromyogram and eye gaze tracking inputs for hands-free cursor control
Chin, Craig A.
Barreto, Armando
[J]. Biomedical Sciences Instrumentation, Vol 43, 2007, 43 : 152 - 157
[8] Hands-free Voice Communication with TV
Papp, Istvan I.
Saric, Zoran M.
Teslic, Nikola Dj
[J]. IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, 2011, 57 (02) : 606 - 614
[9] Fast hands-free writing by gaze direction
David J. Ward
David J. C. MacKay
[J]. Nature, 2002, 418 : 838 - 838
[10] Preparation of Music Scores to Enable Hands-free Page Turning Based on Eye-gaze Tracking
Bonnici, Alexandra
Cristina, Stefania
Camilleri, Kenneth P.
[J]. PROCEEDINGS OF THE 2017 ACM SYMPOSIUM ON DOCUMENT ENGINEERING (DOCENG 17), 2017, : 201 - 210

← 1 2 3 4 5 →