Assessing measures of map value for thematic maps with sparse data

The assessment of the accuracy of thematic maps or classified remotely sensed images has been much discussed, but no single approach has emerged as uniformly useful. Various success measures, such as kappa or the overall accuracy, can be derived for a map or classified image but, when addressing different aspects of the map such as the number and taxonomic detail of map classes, these measures may not be completely appropriate. In addition, strong arguments have been made that the use of kappa should be replaced with the use of allocation and quantity disagreement. Also, different measures of map accuracy may be combined into a single measure of composite map value', the form of which may be context specific. To illustrate this, we consider a composite map value metric (V*) that considers the overall predictive accuracy as well as the number, detail and accuracy of map classes. For recently suggested measures of map value, confidence intervals have not been considered. Our aim is to discuss generic methods to derive confidence intervals for assessment metrics with an emphasis on a simply implemented bootstrap procedure adapted for use with sparse confusion matrices with numerous zero entries.