Elastic backscattering of electrons moving along the helical edge is prohibited by time-reversal symmetry. We demonstrate, however, that an ensemble of magnetic impurities may cause time-reversal symmetry-preserving quasielastic backscattering, resulting in interference effects in the conductance. The characteristic energy transferred in a backscattering event is suppressed due to the Ruderman-Kittel-Kasuya-Yosida interaction of localized spins (the suppression is exponential in the total number of magnetic impurities). We predict the statistics of conductance fluctuations to differ from those in the conventional case of a one-dimensional system with quenched disorder.