Evaluating long-term model-based scenarios of the energy system. (November 2020) - Binsted, Matthew; Iyer, Gokul; Cui, Ryna; Khan, Zarrar; Dorheim, Kalyn; Clarke, Leon
Abstract: Energy-economic models are used to provide science-based decision support in many contexts; evaluating scenarios generated by these models is critical for establishing the credibility of the analyses these tools support. We propose a framework for evaluating energy system scenarios consisting of three parts – a qualitative storyline, quantitative metrics, and evaluation criteria. We apply this framework to the reference scenario for GCAM-USA, a version of the Global Change Analysis Model with state-level detail in the United States, focusing on the evolution of the electric power sector. We develop new visual analytic tools to facilitate evaluation of model outcomes in 51 sub-national regions, and demonstrate how scenario performance can be tracked and compared across four quantifications of the GCAM-USA reference scenario. Highlights: We propose an evaluation framework for model-based scenarios of the energy system. Our evaluation framework includes three elements: storyline, metrics, and criteria. Comparison across scenarios is facilitated by well-defined metrics and evaluation criteria. Not all outlier results indicate poor scenario performance.