Measurements are reported of the production of dijet events with a leading neutron in ep interactions at HERA. Differential cross sections for photoproduction and deep inelastic scattering are presented as a function of several kinematic variables. Leading order QCD simulation programs are compared with the measurements. Models in which the real or virtual photon interacts with a parton of an exchanged pion are able to describe the data. Next-to-leading order perturbative QCD calculations based on pion exchange are found to be in good agreement with the measured cross sections. The fraction of leading neutron dijet events with respect to all dijet events is also determined. The dijet events with a leading neutron have a lower fraction of resolved photon processes than do the inclusive dijet data.