DRAGON: A Dialogue-Based Robot for Assistive Navigation With Visual Language Grounding

被引：1

作者：

Liu, Shuijing ^{[1
]}

Hasan, Aamir ^{[1
]}

Hong, Kaiwen ^{[1
]}

Wang, Runxuan ^{[1
]}

Chang, Peixin ^{[1
]}

Mizrachi, Zachary ^{[1
]}

Lin, Justin ^{[1
]}

McPherson, D. Livingston ^{[1
]}

Rogers, Wendy A. ^{[2
]}

Driggs-Campbell, Katherine ^{[1
]}

机构：

[1] Univ Illinois, Dept Elect & Comp Engn, Champaign, IL 61820 USA

[2] Univ Illinois, Dept Appl Hlth Sci, Champaign, IL 61820 USA

来源：

IEEE ROBOTICS AND AUTOMATION LETTERS | 2024年 / 9卷 / 04期

关键词：

Human-centered robotics; natural dialog for HRI; AI-enabled robotics;

D O I：

10.1109/LRA.2024.3362591

中图分类号：

TP24 [机器人技术];

学科分类号：

080202 ; 1405 ;

摘要：

Persons with visual impairments (PwVI) have difficulties understanding and navigating spaces around them. Current wayfinding technologies either focus solely on navigation or provide limited communication about the environment. Motivated by recent advances in visual-language grounding and semantic navigation, we propose DRAGON, a guiding robot powered by a dialogue system and the ability to associate the environment with natural language. By understanding the commands from the user, DRAGON is able to guide the user to the desired landmarks on the map, describe the environment, and answer questions from visual observations. Through effective utilization of dialogue, the robot can ground the user's free-form language to the environment, and give the user semantic information through spoken language. We conduct a user study with blindfolded participants in an everyday indoor environment. Our results demonstrate that DRAGON is able to communicate with the user smoothly, provide a good guiding experience, and connect users with their surrounding environment in an intuitive manner.

引用

页码：3712 / 3719

页数：8

共 50 条

[41] A smart sensor based visual landmarks detection for indoor robot navigation
Arana-Arejolaleiba, N
Lerasle, F
Briot, M
Lemaire, C
Hayet, JB
16TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, VOL II, PROCEEDINGS, 2002, : 848 - 851
[42] Active Vision-Based Robot Localization and Navigation in a Visual Memory
Mariottini, Gian Luca
Roumeliotis, Stergios I.
2011 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2011,
[43] Autonomous Mobile Robot Intrinsic Navigation Based on Visual Topological Map
Luo, Ren C.
Shih, Wei
2018 IEEE 27TH INTERNATIONAL SYMPOSIUM ON INDUSTRIAL ELECTRONICS (ISIE), 2018, : 541 - 546
[44] Scene perception based visual navigation of mobile robot in indoor environment
Ran, T.
Yuan, L.
Zhang, J. B.
ISA TRANSACTIONS, 2021, 109 : 389 - 400
[45] Autonomous positioning of omnidirectional mobile robot based on visual inertial navigation
Li, Peng
Leng, Binghan
Fu, Huixuan
PROCEEDINGS OF THE 39TH CHINESE CONTROL CONFERENCE, 2020, : 3753 - 3758
[46] Robot visual navigation estimation and target localization based on neural network
Zhao Y.
Gupta R.K.
Onyema E.M.
Paladyn, 2022, 13 (01): : 76 - 83
[47] A Fast Floor Segmentation Algorithm for Visual-based Robot Navigation
Geovani Rodriguez-Telles, F.
Abril Torres-Mendez, L.
Martinez-Garcia, Edgar A.
2013 INTERNATIONAL CONFERENCE ON COMPUTER AND ROBOT VISION (CRV), 2013, : 167 - 173
[48] MOTION DESCRIPTION LANGUAGE-BASED TOPOLOGICAL MAPS FOR ROBOT NAVIGATION
Martin, P.
Egerstedt, M.
COMMUNICATIONS IN INFORMATION AND SYSTEMS, 2008, 8 (02) : 171 - 184
[49] Robot Navigation based on Visual Feature Perception and Monte Carlo Sampling
Li, Ying
Sun, Zuolei
Xu, Yafang
Zhang, Bo
2015 27TH CHINESE CONTROL AND DECISION CONFERENCE (CCDC), 2015, : 3237 - 3242
[50] Visual navigation of a mobile robot with laser-based collision avoidance
Cherubini, Andrea
Chaumette, Francois
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH, 2013, 32 (02): : 189 - 205

← 1 2 3 4 5 →