For many of the envisioned optoelectronic applications of graphene, it is crucial to understand the subpicosecond carrier dynamics immediately following photoexcitation and the effect of photoexcitation on the electrical conductivity—the photoconductivity. Whereas these topics have been studied using various ultrafast experiments and theoretical approaches, controversial and incomplete explanations concerning the sign of the photoconductivity, the occurrence and significance of the creation of additional electron-hole pairs, and, in particular, how the relevant processes depend on Fermi energy have been put forward. We present a unified and intuitive physical picture of the ultrafast carrier dynamics and the photoconductivity, combining optical pump–terahertz probe measurements on a gate-tunable graphene device, with numerical calculations using the Boltzmann equation. We distinguish two types of ultrafast photo-induced carrier heating processes: At low (equilibrium) Fermi energy (EF ≲ 0.1 eV for our experiments), broadening of the carrier distribution involves interband transitions (interband heating). At higher Fermi energy (EF ≳ 0.15 eV), broadening of the carrier distribution involves intraband transitions (intraband heating). Under certain conditions, additional electron-hole pairs can be created [carrier multiplication (CM)] for low EF, and hot carriers (hot-CM) for higher EF. The resultant photoconductivity is positive (negative) for low (high) EF, which in our physical picture, is explained using solely electronic effects: It follows from the effect of the heated carrier distributions on the screening of impurities, consistent with the DC conductivity being mostly due to impurity scattering. The importance of these insights is highlighted by a discussion of the implications for graphene photodetector applications.