Constructing Multimodal Corpus of Chinese Vocabulary for Sign Language Linguistics

[Objective] This paper extracts and organizes knowledge from multimodal sign language resources and constructs a corpus for related research. It meets the public’s urgent demands to obtain sign language knowledge. [Context] The new multimodal corpus is suitable for mining sign language knowledge, which addresses low information levels, disordered resource organization, and difficult utilization of sign language knowledge. [Methods] Firstly, we constructed the multi-modal feature annotation system for sign language vocabulary. Secondly, we formulated the feature coding scheme of the vocabulary and implemented multi-level annotation. Finally, we established the graph model for sign language vocabulary and the Neo4j database to store and visualize. [Results] The vocabulary data are from the national sign language vocabulary corpus. Over 10 000 sign language vocabulary multimodal annotation has been completed, and we realized the whole process of constructing a multimodal corpus. [Conclusions] The new corpus increases knowledge retrieval of hand shape, movement, expression, and posture, which greatly improves the usability of the sign language corpus. © 2023 Chinese Academy of Sciences. All rights reserved.