In-line terahertz pulsed imaging has been utilized to measure the film coating thickness of individual tablets during the coating process in a production-scale pan coater. A criteria-based waveform selection algorithm (WSA) was developed to select terahertz signals reflected from the surface of coating tablets and determine the coating thickness. Since the WSA uses many criteria thresholds to select terahertz waveforms of sufficiently high quality, it could reject some potential candidate tablet waveforms that are close but do not reach the threshold boundary. On the premise of the availability of large datasets, we aim to improve the efficiency of WSA with machine learning. This article presents a recurrent neural network approach to optimize waveform selection. In comparison with the conventional method of WSA, our approach allows more than double the number of waveforms to be selected while maintaining excellent agreement with offline thickness measurements. Moreover, the processing time of waveform selection decreases so that it can be applied for real-time coating monitoring in the pharmaceutical industry, which leads to more advancements in quality control for pharmaceutical film coating.