Current fault diagnosis methods for rotor-bearing system are mostly based on analyzing the vibration signals collected at steady rotating speeds. In those methods, the data collected under one operating condition cannot be accurately used for diagnosis under a different condition. Moreover, in vibration monitoring, installing the necessary sensors will affect the equipment structure and hence the vibration response itself. The present paper proposes a new method based on two-stage parameter transfer and infrared thermal images for fault diagnosis of rotor-bearing system under variable rotating speeds. The method of parameter transfer enables the use of data (or parameters) acquired under one operating condition (called the source domain) to be extended for use in a different operating condition (called the target domain). First, scaled exponential linear unit (SELU) and modified stochastic gradient descent (MSGD) are used to construct an enhanced convolutional neural network (ECNN). Second, a stacked convolutional auto-encoder (CAE) trained based on unlabeled source-domain thermal images is employed to initialize a source-domain ECNN. Third, model parameters from the pre-trained source-domain ECNN are transferred to the target-domain ECNN to adapt to the characteristics of the target domain. The collected thermal images for a rotor-bearing system under variable speeds are used to test the transfer diagnosis performance of the proposed method. The experimental results demonstrate the performance improvement and the advantages of the proposed method.