Challenges and Opportunities in Sociolinguistic Data and Metadata Sharing

Advances in computing technology coupled with recent focus on big data in the social sciences have provided the motivation and some of the infrastructure necessary for sociolinguists to share data among themselves and with researchers in related fields such as human language technologies (HLT). Collaboration among sociolinguists offers the promise to extend current knowledge beyond the community studies that have dominated the field for the past 50 years and focus more on regional and national patterns of variation and change and what they indicate about linguistic theory. Collaboration with HLT developers while relatively new and still uncommon has led to advances both in sociolinguistic methodology and in technologies suited to sociolinguistic research. Before the field can make full use of these advances, however, sociolinguists must confront a number of challenges. Studies that were developed with the intent of describing a single speech community presumably need not ensure, and in many cases have not ensured, consistency with prior work. Given this practice, attempts to compare phenomena across studies must addressmismatches at the levels of data elicitation and selection, coding practice, and the definition of underlying concepts. Adding to the confusion wrought by methodological differences, speech communities differ in ways that the field worker cannot always predict so that different and sometimes unique linguistic and non-linguistic features are found to vary with linguistic structure. This paper underscores the motivation for data sharing by identifying some limitations of comparisons based only on published papers and reviewing advances fueled by data sharing among linguists and between linguists and technology developers. It also documents some of the challenges that hinder data sharing by reviewing work that has build upon available corpora. Finally, it summarizes efforts outside of sociolinguistics that have proposed frameworks for sharing and comparing metadata and categories setting the stage for the papers that follow in these special issues.