We demonstrate the presence of an optical phase transition with frustration-induced spontaneous symmetry breaking in a triangular planar atomic array due to cooperative light-mediated interactions. We show how the array geometry of triangle unit cells at low light intensities leads to degenerate collective radiative excitations forming nearly flat bands. We drive degenerate pairs of collective excitations to be equally populated in both the case of the atomic polarization being in the lattice plane and the case of the atomic polarization being perpendicular to it. At higher intensities, above specific threshold values, this symmetry in the populations is spontaneously broken. We also develop an effective few-mode model that provides semianalytic descriptions of the symmetry-breaking threshold and infinite-lattice limit phase transition. Surprisingly, we find how excitations due to dipolar interactions correspond to optical analogs of those found in frustrated magnets and superfluids, with closely related symmetry-breaking mechanisms despite the significant physical differences between these systems, opening the potential for simulating even quantum magnetism. Transmitted light through the array conveys information about symmetry breaking in the hysteresis behavior of the spectrum. Moreover, in a Mott-insulator state, the atomic positions are subject to zero-point quantum fluctuations. Interpreting each stochastic realization as a light-induced quantum measurement of the atomic position configuration, we find how strong nonlinearities and even weak position uncertainties lead to considerable measurement-induced symmetry breaking, while ensemble averaging over many realizations restores the original symmetry and the unbroken state. Larger position uncertainty results in the formation of domains of different broken symmetries.