Interactive and Multimetric Robustness Tradeoffs in the Colorado River Basin

Policies for environmental systems are being increasingly evaluated on their robustness to deeply uncertain future scenarios. There exist many metrics to quantify robustness, the choice of which depends on stakeholder preferences such as performance thresholds and risk tolerance. Importantly, the policy deemed most robust can vary depending on which metrics are used. Complex systems often result in incomplete or inaccurate understanding of what the performance outcomes of a decision are, meaning that policies selected from initial robustness preferences can result in unexpected trade-offs between performance objectives and exhibit alarmingly poor robustness under alternative metrics. Therefore, we propose a novel, a posteriori robustness framework. Our framework calculates a broad selection of robustness metrics that reflect varying degrees of risk-tolerance and use different methods to summarize performance over future scenarios. Then, we provide stakeholders with background information to interpret what the metrics indicate, rather than propose what metrics are most appropriate based on their initial preferences. We use interactive visualization tools to explore trade-offs between many robustness metrics and performance objectives, iteratively refine what metrics and objectives are important, and remove nonrobust policies. Our dynamic and interactive framework requires an integrated platform, which we create through a web application. We apply our framework to a case study of Lake Mead shortage policies in the Colorado River Basin. Our results show that the selection of Lake Mead policies from predetermined performance requirements and a single type of robustness metric, as done in previous studies, yields solutions with severe but unexpected shortages. Building on these studies, we use our framework to discover these trade-offs, build a multiobjective and multimetric definition of robustness, and choose four robust alternatives.