Large Lexicon Project: American Sign Language Video Corpus and Sign Language Indexing/Retrieval Algorithms

Looking up the meaning of an unknown sign is not nearly so straightforward as looking up a word from a written language in a dictionary. This paper describes progress in an ongoing project to build a system that helps users look up the meaning of ASL signs. An important part of the project is building a video database with examples of a large number of signs. So far we have recorded video examples for almost all of the 3,000 signs contained in the Gallaudet dictionary (and some others not listed there). Locations of hands and the face have been manually annotated for a large number of videos. Using this data, we have built an application that lets the user submit a video of a sign as a query, and presents to the user the most similar signs from the system database. System performance has been evaluated in user-independent experiments with a system vocabulary of 921 signs. For 67% of the test signs, the correct sign is included in the 20 most similar signs retrieved by the system.