Functional Magnetic Resonance Imaging (fMRI) is a dynamic four-dimensional imaging modality. However, in almost all fMRI analyses, the time series elements of this data are assumed to be second order stationary. In this paper we examine, using time series spectral methods, whether such stationary assumptions can be made and whether estimates of non-stationarity can be used to gain understanding into fMRI experiments. A non-stationary version of replicated stationary time series analysis is proposed that takes into account the replicated time series that are available from nearby voxels in a region of interest (ROI). These are used to investigate non-stationarities in both the ROI itself and the variations within the ROI. The proposed techniques are applied to simulated data and to an anxiety inducing fMRI experiment.