The average height of the Coulomb-blockade conductance peaks for chaotic elastic scattering is known to increase by a factor of 4/3 upon breaking time-reversal symmetry. We calculate the temperature dependence of this factor in the regime where the inelastic scattering rate �in is greater than the mean tunneling rate �el, which itself is less than the mean level spacing �. Comparison to recent experimental data by Folk et al. (Folk, Patel, Marcus, Duruöz, and Harris, cond-mat/0008052) demonstrates that �in lies below �el and hence also below �, consistent with the low-energy suppression of inelastic electron-electron scattering in quantum dots.