Assessment of Spoken and Multimodal Applications: Lessons Learned from Laboratory and Field Studies

被引:0
|
作者
Turunen, Markku [1 ]
Hakulinen, Jaakko [1 ]
Heimonen, Tomi [1 ]
机构
[1] Univ Tampere, Dept Comp Sci, FIN-33101 Tampere, Finland
关键词
spoken dialogue system; evaluation; service quality; multimodal applications;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we present the key lessons learnt from numerous evaluations conducted to measure the quality of spoken and multimodal applications. The issues we address include the relation of laboratory and field studies, long-term and pilot evaluations, unimodality and multimodality, objective and subjective metrics, and user expectations and experiences. We present concrete case studies to discuss the above issues. For example, there are major differences in evaluating speech-only and multimodal systems. Similarly, there are major differences between laboratory and field studies, which need to be considered in successful evaluations.
引用
收藏
页码:1333 / 1336
页数:4
相关论文
共 50 条
  • [1] FIELD STUDIES - LESSONS LEARNED
    WHORTON, D
    [J]. TERATOGENESIS CARCINOGENESIS AND MUTAGENESIS, 1984, 4 (01): : 25 - 44
  • [2] Lessons Learned from Field and Laboratory Testing of a DBR Project
    Chen, Dar Hao
    Suh, Chul
    Won, Moon
    [J]. JOURNAL OF PERFORMANCE OF CONSTRUCTED FACILITIES, 2009, 23 (03) : 175 - 180
  • [3] Sport as Laboratory: Lessons Learned From Baseline and Postconcussion Assessment Research
    Mainwaring, Lynda
    Comper, Paul
    Hutchison, Michael
    Richards, Doug
    [J]. JOURNAL OF CLINICAL SPORT PSYCHOLOGY, 2012, 6 (04) : 309 - 335
  • [4] Terrestrial field dissipation studies: Best practices and lessons learned from the field
    Case, Timothy
    White, Jeff
    [J]. ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 2016, 252
  • [5] Lessons learned from laboratory lapses.
    Hill, RH
    Green, R
    [J]. ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 1999, 218 : U343 - U343
  • [6] LESSONS LEARNED FROM LABORATORY PRACTICE IN 2020
    Gill, Morayama
    [J]. AMERICAN JOURNAL OF HEMATOLOGY, 2023, 98 : E40 - E40
  • [7] Life cycle sustainability assessment: Lessons learned from case studies
    Alejandrino, Clarisa
    Mercante, Irma
    Bovea, Maria D.
    [J]. ENVIRONMENTAL IMPACT ASSESSMENT REVIEW, 2021, 87
  • [8] Lessons learned from inter-laboratory studies of carbon isotope analysis of honey
    Dunn, Philip J. H.
    Hill, Sarah
    Cowen, Simon
    Goenaga-Infante, Heidi
    Sargent, Mike
    Goren, Ahmet Ceyhan
    Bilsel, Mine
    Simsek, Adnan
    Ogrinc, Nives
    Potocnik, Doris
    Armishaw, Paul
    Hai, Lu
    Konopelko, Leonid
    Chubchenko, Yan
    Chesson, Lesley A.
    van der Peijl, Gerard
    Blaga, Cornelia
    Posey, Robert
    Camin, Federica
    Chernyshev, Anatoly
    Chowdhury, Sadia A.
    [J]. SCIENCE & JUSTICE, 2019, 59 (01) : 9 - 19
  • [9] Challenges in reproducibility of genetic association studies: lessons learned from the obesity field
    A Li
    D Meyre
    [J]. International Journal of Obesity, 2013, 37 : 559 - 567
  • [10] Challenges in reproducibility of genetic association studies: lessons learned from the obesity field
    Li, A.
    Meyre, D.
    [J]. INTERNATIONAL JOURNAL OF OBESITY, 2013, 37 (04) : 559 - 567