In this thesis we apply results from multivariate probability, random matrix theory (RMT) and free probability theory (FPT) to analyse the theoretical performance limits of future-generation wireless communication systems which implement multiple-antenna technologies. Motivated by the capacity targets for fifth generation wireless communications, our work focuses on quantifying the performance of these systems in terms of several relevant metrics, including ergodic rate and capacity, secrecy rate and capacity, asymptotic capacity, outage probability, secrecy outage probability and diversity order.
Initially, we investigate the secrecy performance of a wirelessly powered, wiretap channel which incorporates a relatively small number of transmit antennas in a multiple-input single-output scenario. We consider two different transmission protocols which utilise physical layer security. Using traditional multivariate probability techniques we compute closed-form expressions for the outage probability and secrecy outage probability of the system under both protocols, based on the statistical properties of the channel. We use these expressions to compute approximations of the connection outage probability, secrecy outage probability and diversity orders in the high signal-to-noise ratio (SNR) regime which enables us to find candidates for the optimal time-switching ratio and power allocation coefficients. We show that it is possible to achieve a positive secrecy throughput, even in the case where the destination is further away from the source than the eavesdropper, for both protocols and compare their relative merits.
We then progress to considering small-scale multiple-input multiple-output (MIMO) channels, which can be modelled as random matrices. We consider a relay system that enables communication between a remote source and destination in the presence of an eavesdropper and describe a decode-and-forward (DF) protocol which uses physical layer security techniques. A new result on the joint probability density function of the largest eigenvalues of the channel matrix is derived using results from RMT. The result enables us to compute the legitimate outage probability and diversity order of the proposed protocol and to quantify the effect of increasing the number of relays and antennas of the system.
Next, we consider much larger-scale massive MIMO arrays, for which analysis using finite results becomes impractical. First we investigate the ergodic capacity of a massive MIMO, non-orthogonal multiple access system with unlimited numbers of antennas. Employing asymptotic results from RMT, we provide closed-form solutions for the asymptotic capacities of this scenario. This enables us to derive the optimal power allocation coefficients for the system. We demonstrate that our approach has low computational complexity and provides results much closer to optimality when compared with existing, suboptimal methods, particularly for the case where nodes are equipped with very large antenna arrays.
Finally, we analyse the ergodic capacity of a single-hop, massive MIMO, multi-relay system having more complex properties, by applying results in FPT. Our method allows for an arbitrary number of relays, arbitrarily large antenna arrays and also asymmetric characteristics between channels, which is a situation that cannot typically be analysed using traditional RMT methods. We compute the asymptotic capacity across the system for the case when the relays employ a DF protocol and no direct link exists between the endpoints. We are able to calculate the overall capacity, to a high degree of accuracy, for systems incorporating channels greater than $128\times 128$ in dimension for which existing methods fail due to excessive computational demands. Finally, the comparative computational complexities of the methods are analysed and we see the advantages of applying the FPT method.