Measurements are presented of differential dijet cross sections in diffractive photoproduction (Q2<0.01 GeV2) and deep-inelastic scattering processes (DIS, 4<Q2<80 GeV2). The event topology is given by ep→eXY, in which the system X, containing at least two jets, is separated from a leading low-mass baryonic system Y by a large rapidity gap. The dijet cross sections are compared with NLO QCD predictions based on diffractive parton densities previously obtained from a QCD analysis of inclusive diffractive DIS cross sections by H1. In DIS, the dijet data are well described, supporting the validity of QCD factorisation. The diffractive DIS dijet data are more sensitive to the diffractive gluon density at high fractional parton momentum than the measurements of inclusive diffractive DIS. In photoproduction, the predicted dijet cross section has to be multiplied by a factor of approximately 0.5 for both direct and resolved photon interactions to describe the measurements. The ratio of measured dijet cross section to NLO prediction in photoproduction is a factor 0.5±0.1 smaller than the same ratio in DIS. This suppression is the first clear observation of QCD hard scattering factorisation breaking at HERA. The measurements are also compared to the two soft colour neutralisation models SCI and GAL. The SCI model describes diffractive dijet production in DIS but not in photoproduction. The GAL model fails in both kinematic regions.