In the fifth generation and beyond (B5G), delay constraints emerge as a topic of particular interest, e.g. for ultra-reliable low latency communications (URLLC) such as autonomous vehicles and enhanced reality. In this paper, we study the performance of a two-user uplink NOMA network under statistical quality of service (QoS) delay constraints, captured through each user's effective capacity (EC). We propose novel closed-form expressions for the EC of the NOMA users and show that in the high signal to noise ratio (SNR) region, the "strong" NOMA user has a limited EC, assuming the same delay constraint as the "weak" user. We demonstrate that for the weak user, OMA achieves higher EC than NOMA at small values of the transmit SNR, while NOMA outperforms OMA in terms of EC at high SNRs. On the other hand, for the strong user the opposite is true, i.e., NOMA achieves higher EC than OMA at small SNRs, while OMA becomes more beneficial at high SNRs. This result raises the question of introducing "adaptive" OMA / NOMA policies, based jointly on the users' delay constraints as well as on the available transmit power.