We have investigated the decay of second-sound acoustic turbulence in superfluid He-4 following removal of the driving force that created it. A periodically driven cylindrical resonator of high quality Q factor was used to create one-dimensional second sound of large wave amplitude. The resultant acoustic turbulence involved an energy cascade towards the higher frequencies (smaller scales) where viscous dissipation occurred. Under some conditions, there was also a co-existing inverse energy cascade in the opposite direction, towards lower frequencies, where dissipation also occurred. We have found that the spectral dynamics that occurs when the periodic drive is switched off exhibits complex and interesting features. We show that the wave decay starts from the high-frequency end of the spectrum in both cases, and we have been able to identify and measure a nonlinear decay time related to the interwave interactions, as distinct from the linear decay process due to dissipation. A windowed Fourier analysis reveals that the occurrence of seemingly chaotic oscillations of the individual harmonic amplitudes were observed during the decay.