When using mixture models it may be the case that the modeller has a-priori
beliefs or desires about what the components of the mixture should represent.
For example, if a mixture of normal densities is to be fitted to some data, it
may be desirable for components to focus on capturing differences in location
rather than scale. We introduce a framework called proximity penalty priors
(PPPs) that allows this preference to be made explicit in the prior
information. The approach is scale-free and imposes minimal restrictions on the
posterior; in particular no arbitrary thresholds need to be set. We show the
theoretical validity of the approach, and demonstrate the effects of using PPPs
on posterior distributions with simulated and real data.