The aim of this paper is to provide an overview of the possible advantages of adopting a Bayesian approach to nonlinear system identification in structural dynamics. In contrast to identification schemes which estimate maximum likelihood values (or other point estimates) for parameters, the Bayesian scheme discussed here provides information about the complete probability density functions of parameter estimates without adopting restrictive assumptions about their nature. Among other advantages of the Bayesian viewpoint are the abilities to make informed decisions about model selection and also to effectively make predictions over entire classes of models, with each individual model weighted according to its ability to explain the observed data. The approach is illustrated using data from simulated systems, first a Duffing oscillator and then a new application to hysteretic system of the Bouc-Wen type. The modelling and identification of the latter type of system has long presented problems due to the fact that commonly used model structures like the Bouc-Wen model are nonlinear in the parameters, or have unmeasured states, etc. These issues have been dealt with in the past by adopting an optimisation-based approach to the problem; in particular, the differential evolution algorithm has proved very effective. An objective of the current paper is to illustrate how the Bayesian approach provides the same information and more as the optimisation approach; it yields parameter estimates and their associated confidence intervals, but can also provide confidence bounds on model predictions and evidence measures which can be used to select the most appropriate model from a candidate set. A new model selection criterion in this context - the Deviance Information Criterion (DIC) - is presented here.