For extreme value modelling based on threshold techniques, a well-documented issue is the sensitivity of inference from the model to the choice of threshold. The threshold above which we assume a non-homogeneous Poisson process, or equivalently generalized Pareto representation, to be a reasonable approximation to the distribution is traditionally selected before analysis and subsequently treated as fixed and known. In doing so, the analyst cannot account for the subjective judgement that has already taken place before formal inference begins. We propose an asymptotically motivated model to account for uncertainty in choice of threshold, under assumptions generated by a penultimate form of extreme value theory. To assess the sensitivity of the conclusions to these assumptions, we additionally present a purely likelihood-based diagnostic for the choice of threshold, developing a non-standard likelihood ratio test which supplements the current suite of tools. We show that the likelihood ratio procedure quantifies evidence derived from traditional threshold diagnostic plots, and that the full model for threshold uncertainty identifies the same features as the diagnostic. We apply our procedures to both simulated data, and a data set of flow rates from the River Nidd.