TY - JOUR

T1 - Fast sensitivity analysis methods for computationally expensive models with multi-dimensional output

AU - Ryan, Edmund

AU - Wild, Oliver

AU - Voulgarakis, Apostolos

AU - Lee, Lindsay

PY - 2018/8/3

Y1 - 2018/8/3

N2 - Global sensitivity analysis (GSA) is a critical approach in identifying which inputs or parameters of a model most affect model output. This determines which inputs to include when performing model calibration or uncertainty analysis. GSA allows quantification of the sensitivity index (SI) of a particular input – the percentage of the total variability in the output attributed to the changes in that input – by averaging over the other inputs rather than fixing them at specific values. Traditional methods of computing the SIs (e.g. Sobol) involve running a model thousands of times, but this may not be feasible for computationally expensive earth system models. GSA methods that use a statistical emulator in place of the expensive model are popular as they require far fewer model runs. Here, we perform an eight-input GSA on two computationally expensive atmospheric chemistry transport models using emulators that were trained with 80 runs of the models. We consider two methods to further reduce the computational cost of GSA: (1) a dimension reduction approach and (2) an emulator-free approach. When the output of a model is multi-dimensional, it is common practice to build a separate emulator for each dimension of the output space. Here, we use principal component analysis (PCA) to reduce the output dimension and build an emulator for each of the transformed outputs. We consider the global distribution of the annual column mean lifetime of atmospheric methane, which requires ~ 2000 emulators without PCA, but only 5–40 emulators with PCA. As an alternative, we apply an emulator-free method using a generalised additive model (GAM) to estimate the SIs using only the training runs. Compared to the emulator-only method, the hybrid PCA-emulator and GAM methods are 6 and 30 times quicker, respectively, at computing the SIs for the ~ 2000 methane lifetime outputs. The SIs computed using the two computationally faster methods are almost identical to those computed using the standard emulator-only method.

AB - Global sensitivity analysis (GSA) is a critical approach in identifying which inputs or parameters of a model most affect model output. This determines which inputs to include when performing model calibration or uncertainty analysis. GSA allows quantification of the sensitivity index (SI) of a particular input – the percentage of the total variability in the output attributed to the changes in that input – by averaging over the other inputs rather than fixing them at specific values. Traditional methods of computing the SIs (e.g. Sobol) involve running a model thousands of times, but this may not be feasible for computationally expensive earth system models. GSA methods that use a statistical emulator in place of the expensive model are popular as they require far fewer model runs. Here, we perform an eight-input GSA on two computationally expensive atmospheric chemistry transport models using emulators that were trained with 80 runs of the models. We consider two methods to further reduce the computational cost of GSA: (1) a dimension reduction approach and (2) an emulator-free approach. When the output of a model is multi-dimensional, it is common practice to build a separate emulator for each dimension of the output space. Here, we use principal component analysis (PCA) to reduce the output dimension and build an emulator for each of the transformed outputs. We consider the global distribution of the annual column mean lifetime of atmospheric methane, which requires ~ 2000 emulators without PCA, but only 5–40 emulators with PCA. As an alternative, we apply an emulator-free method using a generalised additive model (GAM) to estimate the SIs using only the training runs. Compared to the emulator-only method, the hybrid PCA-emulator and GAM methods are 6 and 30 times quicker, respectively, at computing the SIs for the ~ 2000 methane lifetime outputs. The SIs computed using the two computationally faster methods are almost identical to those computed using the standard emulator-only method.

U2 - 10.5194/gmd-11-3131-2018

DO - 10.5194/gmd-11-3131-2018

M3 - Journal article

VL - 11

SP - 3131

EP - 3146

JO - Geoscientific Model Development

JF - Geoscientific Model Development

SN - 1991-959X

ER -