This paper reports the first simultaneous measurement of the double differential muon neutrino charged-current cross section on oxygen and carbon without pions in the final state as a function of the outgoing muon kinematics, made at the ND280 off-axis near detector of the T2K experiment. The ratio of the oxygen and carbon cross sections is also provided to help validate various models' ability to extrapolate between carbon and oxygen nuclear targets, as is required in T2K oscillation analyses. The data are taken using a neutrino beam with an energy spectrum peaked at 0.6~GeV and comprises 57.34$\times$10$^{19}$ protons on target. The extracted measurement is compared with the prediction from different Monte Carlo neutrino-nucleus interaction event generators, showing particular model separation for very forward-going muons. Overall, of the models tested, the result is best described using Local Fermi Gas descriptions of the nuclear ground state with RPA suppression.