Test and Evaluation of Autonomous Surface Vehicles: A Case Study

被引:0
|
作者
Reitz, Brian C. [1 ]
Wilkerson, Joshua L. [1 ]
机构
[1] Naval Air Warfare Ctr, Weap Div, China Lake, CA 93555 USA
关键词
marine; vehicle; navigation; autonomy; test; evaluation;
D O I
10.1109/plans46316.2020.9110129
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The rigorous test and evaluation of autonomous systems is often a challenging process. Specifically, the transformation of qualitative assessments often used in the development phase to unbiased quantitative assessment can be difficult. Additionally, few standardized metrics exist for the test and evaluation of autonomous systems, as applications/approaches vary so widely that specialized metrics must be developed in the majority of cases. This paper presents a case study for this process, describing a multi-year, iterative test and evaluation effort undertaken by the authors in support of the development of an autonomous surface vehicle system. The case study details the development of quantitative performance metrics (including the rationale behind each one), revisions/additions that were made to the metrics between testing and evaluation events, impacts the metrics had on the autonomy development process observed through successive tests, and lessons learned throughout the process.
引用
收藏
页码:839 / 850
页数:12
相关论文
共 50 条
  • [1] Test and Evaluation of Autonomous Ground Vehicles
    Sun, Yang
    Xiong, Guangming
    Song, Weilong
    Gong, Jianwei
    Chen, Huiyan
    [J]. ADVANCES IN MECHANICAL ENGINEERING, 2014,
  • [2] Integrated Framework for Test and Evaluation of Autonomous Vehicles
    Su Y.
    Wang L.
    [J]. Journal of Shanghai Jiaotong University (Science), 2021, 26 (05) : 699 - 712
  • [3] Systematic Test Case Design for Autonomous Vehicles
    Leilabadi, Shervin Hajinia
    Katzorke, Nils
    Moosmann, Matthias
    Schmidt, Stephan
    [J]. 2020 IEEE 23RD INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2020,
  • [4] Evaluation Method and Test Verification of Road Test Scenes for Autonomous Vehicles
    自动驾驶车辆道路测试场景评价方法与试验验证
    [J]. Wang, Rong (wangrong@cstc.org.cn), 1600, SAE-China (43): : 620 - 628
  • [5] EXPERIMENTAL EVALUATION OF APPROACH BEHAVIOR FOR AUTONOMOUS SURFACE VEHICLES
    Bertaska, Ivan R.
    Alvarez, Jose
    Sinisterra, Armando
    von Ellenrieder, Karl
    Dhanak, Manhar
    Shah, Brual
    Svec, Petr
    Gupta, Satyandra K.
    [J]. ASME 2013 DYNAMIC SYSTEMS AND CONTROL CONFERENCE, VOL 2, 2013,
  • [6] Sensor Evaluation for Autonomous Surface Vehicles in Inland Waterways
    Robinette, Paul
    Sacarny, Michael
    DeFilippo, Michael
    Novitzky, Michael
    Benjamin, Michael R.
    [J]. OCEANS 2019 - MARSEILLE, 2019,
  • [7] A case study of unavoidable accidents of autonomous vehicles
    Sun, Zhiwei
    Lin, Miao
    Chen, Wentao
    Dai, Bing
    Ying, Pengfei
    Zhou, Qing
    [J]. TRAFFIC INJURY PREVENTION, 2024, 25 (01) : 8 - 13
  • [8] Perceptions of Autonomous Vehicles: A Case Study of Jordan
    Abudayyeh, Dana
    Almomani, Malek
    Almomani, Omar
    Alsoud, Hadeel
    Alsalman, Farah
    [J]. WORLD ELECTRIC VEHICLE JOURNAL, 2023, 14 (05):
  • [9] Evaluation of Ocean Currents Observed from Autonomous Surface Vehicles
    Hodges, Benjamin A.
    Grare, Laurent
    Greenwood, Benjamin
    Matsuyoshi, Kayli
    Pizzo, Nick
    Statom, Nicholas M.
    Farrar, J. Thomas
    Lenain, Luc
    [J]. JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY, 2023, 40 (10) : 1121 - 1136
  • [10] Evaluation of a New Carbon Dioxide System for Autonomous Surface Vehicles
    Sabine, Christopher
    Sutton, Adrienne
    McCabe, Kelly
    Lawrence-Slavas, Noah
    Alin, Simone
    Feely, Richard
    Jenkins, Richard
    Maenner, Stacy
    Meinig, Christian
    Thomas, Jesse
    van Ooijen, Erik
    Passmore, Abe
    Tilbrook, Bronte
    [J]. JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY, 2020, 37 (08) : 1305 - 1317