The combination of Non-Orthogonal Multiple Access (NOMA) and Transmit Antenna Selection (TAS) techniques has recently attracted significant attention due to the low cost, low complexity and high diversity gains. Meanwhile, Random Linear Coding (RLC) is considered to be a promising technique for achieving high reliability and low latency in multicast communications. In this paper, we consider a downlink system with a multi-antenna base station and two multicast groups of single-antenna users, where one group can afford to be served opportunistically, while the other group consists of comparatively low power devices with limited processing capabilities that have strict Quality of Service (QoS) requirements. In order to boost reliability and satisfy the QoS requirements of the multicast groups, we propose a cross-layer framework including NOMA based TAS at the physical layer and RLC at the application layer. In particular, two low complexity TAS protocols for NOMA are studied in order to exploit the diversity gain and meet the QoS requirements. In addition, RLC analysis aims to facilitate heterogeneous users, such that, sliding window based sparse RLC is employed for computational restricted users, and conventional RLC is considered for others. Theoretical expressions that characterize the performance of the proposed framework are derived and verified through simulation results.