This article considers how to implement Markov chain Monte Carlo (MCMC) moves within a particle filter. Previous, similar, attempts have required the complete history ("trajectory") of each particle to be stored. Here, it is shown how certain MCMC moves can be introduced within a particle filter when only summaries of each particles' trajectory are stored. These summaries are based on sufficient statistics. Using this idea, the storage requirement of the particle filter can be substantially reduced, and MCMC moves can be implemented more efficiently. We illustrate how this idea can be used for both the bearingsonly tracking problem and a model of stochastic volatility. We give a detailed comparison of the performance of different particle filters for the bearings-only tracking problem. MCMC, combined with a sensible initialization of the filter and stratified resampling, produces substantial gains in the efficiency of the particle filter.