A search is presented for the production of new heavy quarks that decay to a Z boson and a third-generation Standard Model quark. In the case of a new charge +2/3 quark (T ), the decay targeted is T → Zt, while the decay targeted for a new charge −1/3 quark (B) is B → Zb. The search is performed with a dataset corresponding to 20.3 fb−1 of pp collisions at s√=8 TeV recorded in 2012 with the ATLAS detector at the CERN Large Hadron Collider. Selected events contain a high transverse momentum Z boson candidate reconstructed from a pair of oppositely charged same-flavor leptons (electrons or muons), and are analyzed in two channels defined by the absence or presence of a third lepton. Hadronic jets, in particular those with properties consistent with the decay of a b-hadron, are also required to be present in selected events. Different requirements are made on the jet activity in the event in order to enhance the sensitivity to either heavy quark pair production mediated by the strong interaction, or single production mediated by the electroweak interaction. No significant excess of events above the Standard Model expectation is observed, and lower limits are derived on the mass of vector-like T and B quarks under various branching ratio hypotheses, as well as upper limits on the magnitude of electroweak coupling parameters.