Assessing the suitability of an honest rating mechanism for the collaborative creation of structured knowledge

Creating and maintaining semantic structures such as ontologies continues to be an important issue. The approach investigated here is to let members of an online community create structured knowledge collaboratively and to use ratings to evaluate the data created. Obviously, such ratings have to be of high quality. Honest rating mechanisms (HRMs) known from literature are a promising means to gain such high-quality ratings. However, the design of such mechanisms for collaborative knowledge creation and their effectiveness have not been studied so far. To evaluate the effects of an HRM on rating quality in this context, we have conducted several experiments with online communities. We find that an HRM increases rating quality and "punishes" rating errors. We also find that rating-based rewards increase the quality of the structured knowledge created.