A SENSITIVITY ANALYSIS OF A PROBABILISTIC INFORMATION-RETRIEVAL SYSTEM

Results of a set of exploratory simulations to test the effects of errors in estimation of individual term probabilities on the performance of a probabilistic information retrieval system are presented. Searches were executed with various levels of term error on a test collection of probabilistically indexed information sources. The amount of error in the final probability of relevance used to rank sources introduced by these errors was analytically determined. The first simulation analyzed simulated rankings obtained by simulating probabilities of relevance according to the error distribution; in the second simulation measures of rank correlation between simulated rankings and the system's ranking were calculated; in the third, the effect of the error on retrieval performance using the measure expected search length was determined. It was found that substantial error was introduced into final probabilities of relevance, but that for low levels of term error the impact on ranking and retrieval performance was moderate, while even with high levels the actual ranking performed significantly better than a random ranking of retrieved sources. © 1990 John Wiley & Sons, Inc. Copyright © 1990 John Wiley & Sons, Inc.