Deep neural networks have been very successful as highly accurate wave function Ansätze for variational Monte Carlo calculations of molecular ground states. We present an extension of one such Ansatz, FermiNet, to calculations of the ground states of periodic Hamiltonians, and study the homogeneous electron gas. FermiNet calculations of the ground-state energies of small electron gas systems are in excellent agreement with previous initiator full configuration interaction quantum Monte Carlo and diffusion Monte Carlo calculations. We investigate the spin-polarized homogeneous electron gas and demonstrate that the same neural network architecture is capable of accurately representing both the delocalized Fermi liquid state and the localized Wigner crystal state. The network converges on the translationally invariant ground state at high density and spontaneously breaks the symmetry to produce the crystalline ground state at low density, despite being given no a priori knowledge that a phase transition exists.