ATLAS measurements of two-particle correlations are presented for √s =5.02 and 13 TeV pp collisions and for √sNN =5.02 TeV p+Pb collisions at the LHC. The correlation functions are measured as a function of relative azimuthal angle Δϕ, and pseudorapidity separation Δη, using charged particles detected within the pseudorapidity interval |η|<2.5. Azimuthal modulation in the long-range component of the correlation function, with |Δη|>2, is studied using a template fitting procedure to remove a “back-to-back” contribution to the correlation function that primarily arises from hard-scattering processes. In addition to the elliptic, cos(2Δϕ), modulation observed in a previous measurement, the pp correlation functions exhibit significant cos(3Δϕ) and cos(4Δϕ) modulation. The Fourier coefficients vn,n associated with the cos(nΔϕ) modulation of the correlation functions for n=2–4 are measured as a function of charged-particle multiplicity and charged-particle transverse momentum. The Fourier coefficients are observed to be compatible with cos(nϕ) modulation of per-event single-particle azimuthal angle distributions. The single-particle Fourier coefficients vn are measured as a function of charged-particle multiplicity, and charged-particle transverse momentum for n=2–4. The integrated luminosities used in this analysis are, 64 nb−1 for the √s =13 TeV pp data, 170 nb−1 for the √s =5.02 TeV pp data, and 28 nb−1 for the √sNN =5.02 TeV p+Pb data.