This paper addresses the concept of tourism vulnerability associated with the structure of the destinations' tourism source markets, by using several relevant factors and the construction of synthetic indexes. Following an operable and parsimonious criterion, we select a reduced set of fundamental indicators that have been stressed in the literature, and that are related to dimensions such as the destination proximity, the degree of the market demand concentration, the strength of seasonality, or the economic profile of the tourists. We implement this framework for the fifty Spanish provinces, using hotel travelers as an indicator of the demand for a destination. In this way, we guarantee a wide territorial coverage and heterogeneity of factors. The results obtained withstand various robustness analyses, and indicate that the play of the various factors means that among the most vulnerable provinces there are all kinds, both specialized in sun and beach, and urban, cultural or rural, for example (which indicates that the tourism model does not determine the degree of vulnerability). On the other hand, the vulnerability would be associated with the magnitude of the demand and would have little to do with the competitiveness measures as they have been proposed up to now. Overall, the proposed methodology and empirical results can be easily applied in practice and are of great interest for researchers, managers and policymakers.