Data-driven intelligent fault diagnosis methods generally require a large amount of labeled data and considerable time to train network models. However, obtaining sufficient labeled data in practical industrial scenarios has always been a challenge, which hinders the practical application of data-driven methods. A digital twin (DT) model of rolling bearings can generate labeled training dataset for various bearing faults, supplementing the limited measured data. This paper proposes a novel DT-assisted approach to address the issue of limited measured data for bearing fault diagnosis. First, a dynamic model of bearing with damages is introduced to generate simulated bearing acceleration vibration signals. A DT model is constructed in Simulink, where the model parameters are updated based on the actual system behavior. Second, the structural parameters of the DT model are adaptively updated using least squares method with the measured data. Third, a Vision Transformer (ViT) -based network, integrated with short-time Fourier transform, is developed to achieve accurate fault diagnosis. By applying short-time Fourier transform at the input end of the ViT network, the model effectively extracts additional information from the vibration signals. Pre-training the network with an extensive dataset from miscellaneous tasks enables the acquisition of pre-trained weights, which are subsequently transferred to the bearing fault diagnosis task. Experiments results verify that the proposed approach can achieve higher diagnostic accuracy and better stability.