We report phonon pulse measurements of quasiparticle (qp) self-recombination and trapping by Abrikosov vortice's in Nb and Ta superconducting tunnel junctions. The quasiparticle loss times, determined from the shape of the tunnel junction output pulse, were found to be strongly correlated with flux trapping. The measured results were compared with the model of Golubov and Houwman. It was found that qp self-recombination, as measured from the dependence of responsivity on absorbed phonon energy, was also modified by the presence of traps. A process of phonon-mediated de-trapping is proposed to explain the observed effects.