The fourth radiation transfer model intercomparison (RAMI‐IV): Proficiency testing of canopy reflectance models with ISO‐13528. Issue 13 (1st July 2013)

• Record Type: Journal Article
• Authors: Widlowski, J.‐L.; Pinty, B.; Lopatka, M.; Atzberger, C.; Buzica, D.; Chelle, M.; Disney, M.; Gastellu‐Etchegorry, J‐P.; Gerboles, M.; Gobron, N.; Grau, E.; Huang, H.; Kallel, A.; Kobayashi, H.; Lewis, P. E.; Qin, W.; Schlerf, M.; Stuckens, J.; Xie, D.
• Is Part Of: Journal of geophysical research. Volume 118:Issue 13(2013)
• Page Start: 6869
• Page End: 6890
• Publication Date: 2013-07-01
• Subjects: model comparison -- radiative transfer models -- ISO standards -- BRDF -- vegetation canopies -- validation; Atmospheric physics -- Periodicals; Geophysics -- Periodicals; 551.5
• DOI: 10.1002/jgrd.50497
• Languages: English
• ISSNs: 2169-897X

Abstract

Abstract : [1] The radiation transfer model intercomparison (RAMI) activity aims at assessing the reliability of physics‐based radiative transfer (RT) models under controlled experimental conditions. RAMI focuses on computer simulation models that mimic the interactions of radiation with plant canopies. These models are increasingly used in the development of satellite retrieval algorithms for terrestrial essential climate variables (ECVs). Rather than applying ad hoc performance metrics, RAMI‐IV makes use of existing ISO standards to enhance the rigor of its protocols evaluating the quality of RT models. ISO‐13528 was developed "to determine the performance of individual laboratories for specific tests or measurements." More specifically, it aims to guarantee that measurement results fall within specified tolerance criteria from a known reference. Of particular interest to RAMI is that ISO‐13528 provides guidelines for comparisons where the true value of the target quantity is unknown. In those cases, "truth" must be replaced by a reliable "conventional reference value" to enable absolute performance tests. This contribution will show, for the first time, how the ISO‐13528 standard developed by the chemical and physical measurement communities can be applied to proficiency testing of computer simulation models. Step by step, the pre‐screening of data, the identification of reference solutions, and the choice of proficiency statistics will be discussed and illustrated with simulation results from the RAMI‐IV "abstract canopy" scenarios. Detailed performance statistics of the participating RT models will be provided and the role of the accuracy of the reference solutions as well as the choice of the tolerance criteria will be highlighted. Key Points: ISO‐13528 can be applied to the verification of computer simulation models. Model comparisons require detailed definitions on acceptable bias levels. Operator choices/errors are likely cause for most observed biases in RAMI‐IV.