Deep neural networks (DNNs) have been widely used for intelligent fault diagnosis under the closed world assumption that any testing data is within classes of the training data. However, in reality, out-of-distribution (OOD) cases such as new fault conditions can happen after the original trained model is deployed. Most of the current DNNs are deterministic which can misclassify with high confidence in the open-world scenario. This overconfident behavior would not guarantee the reliability and robustness of fault diagnosis results in practice. Therefore, trustworthy intelligent fault diagnosis with uncertainty estimation is crucial for real applications. In this paper, we develop a novel convolutional neural network integrating evidence theory to achieve fault classifications with prediction uncertainty estimation. The estimated prediction uncertainty can identify potential OOD samples. This approach allows a minimal modification of the state-of-the-art DNN model by using a risk-calibrated evidential loss function and Dirichlet distribution that replaces the classification probabilities. The experimental results show that the proposed approach can not only achieve accurate classification of known classes but also detect unknown classes effectively. The proposed method shows significant potential in detecting OOD patterns and provides trustworthy fault diagnosis in open and non-stationary environments.