INDEXING AS PROBLEM-SOLVING - A COGNITIVE APPROACH TO CONSISTENCY

It is well known that indexers differ significantly in their judgment as to which terms reflect most adequately the content of a given document. For more than three decades, studies on interindexer consistency identified several factors associated with low consistency, but failed to provide a comprehensive model of this phenomenon. Our research program is an attempt to apply theories and methods from cognitive psychology to the study of indexing behavior. From a theoretical standpoint, indexing is considered as a problem solving situation. The research effort then consists in identifying the subject's problem space, that is, the set of declarative and procedural knowledge used to solve the problem, and uncovering the strategic decisions that may lead to differences in chosen keywords. In order to access to these cognitive processes, three kinds of verbal reports are used: 1) thinking aloud protocol collected while the task is in progress, 2) retrospective comments and answers provided by the indexer when confronted with a video recording of the task, 3) indexer's comments on lists of keywords produced by several other indexers for the same document. We will present results of an experiment in which eight indexers (four beginners and four experts) were asked to index the same four documents with two different thesauri. It will be shown that the three kinds of verbal reports provide complementary data on strategic behavior, and that it is of prime importance to consider the indexing task as an ill-defined problem, where the solution is partly defined by the indexer him/herself.