Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation

被引：0
|
作者：
Macketanz, Vivien [1 ]
Naderi, Babak [2 ]
Schmidt, Steven [2 ]
Moeller, Sebastian [2 ]
机构：
[1] German Res Ctr AI, Kaiserslautern, Germany
[2] TU Berlin, Qual & Usabil Lab, Berlin, Germany
来源：
PROCEEDINGS OF THE 2ND WORKSHOP ON HUMAN EVALUATION OF NLP SYSTEMS (HUMEVAL 2022) | 2022年
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The quality of machine-generated text is a complex construct consisting of various aspects and dimensions. We present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowd-sourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs. An Exploratory Factor Analysis revealed the underlying perceptual dimensions. As a result, we extracted four factors that operate as relevant dimensions for the Quality of Experience of MT outputs: precision, complexity, grammaticality, and transparency.
引用
收藏
页码：24 / 31
页数：8
相关论文
共 50 条
  • [41] The Limitations of Stylometry for Detecting Machine-Generated Fake News
    Schuster, Tal
    Schuster, Roei
    Shah, Darsh J.
    Barzilay, Regina
    COMPUTATIONAL LINGUISTICS, 2020, 46 (02) : 499 - 510
  • [42] A Polishing Model for Machine-Generated Ancient Chinese Poetry
    Zhe Chen
    Yang Cao
    Neural Processing Letters, 56
  • [43] Evaluation of machine-generated chemical ontologies for molecular information
    Boyer, Stephen
    Griffin, Thomas
    Louie, Eric
    ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 2016, 251
  • [44] Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification
    Mozes, Maximilian
    Bartolo, Max
    Stenetorp, Pontus
    Kleinberg, Bennett
    Griffin, Lewis D.
    2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 8258 - 8270
  • [45] Real or Fake Text?: Investigating Human Ability to Detect Boundaries between Human-Written and Machine-Generated Text
    Dugan, Liam
    Ippolito, Daphne
    Kirubarajan, Arun
    Shi, Sherry
    Callison-Burch, Chris
    THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 11, 2023, : 12763 - 12771
  • [46] COCO: Coherence-Enhanced Machine-Generated Text Detection Under Low Resource With Contrastive Learning
    Liu, Xiaoming
    Zhang, Zhaohan
    Wang, Yichen
    Pu, Hang
    Lan, Yu
    Shen, Chao
    2023 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2023), 2023, : 16167 - 16188
  • [47] Team lm-detector at PAN: Can NLI be an Appropriate Approach to Machine-Generated Text Detection
    Wu, Guojun
    Guan, Qinghao
    CEUR Workshop Proceedings, 2024, 3740 : 2956 - 2962
  • [48] DetectLLM: Leveraging Log-Rank Information for Zero-Shot Detection of Machine-Generated Text
    Su, Jinyan
    Zhuo, Terry Yue
    Wang, Di
    Nakov, Preslav
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EMNLP 2023), 2023, : 12395 - 12412
  • [49] Measuring human readability of machine generated text: Three case studies in speech recognition and machine translation
    Jones, D
    Gibson, E
    Shen, W
    Granoien, N
    Herzog, M
    Reynolds, D
    Weinstein, C
    2005 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOLS 1-5: SPEECH PROCESSING, 2005, : 1009 - 1012
  • [50] Machine Translation of Mathematical Text
    Ohri, Aditya
    Schmah, Tanya
    IEEE ACCESS, 2021, 9 : 38078 - 38086
12345