Stochastic inflation can resolve strong inflationary perturbations, which seed primordial black holes. I present a fast and accurate way to compute these perturbations in typical black hole producing single-field models, treating the short-wavelength Fourier modes beyond the de Sitter approximation. The squeezing and freezing of the modes reduces the problem to one dimension, and the resulting new form of the stochastic equations, dubbed 'constrained stochastic inflation,' can be solved efficiently with semi-analytical techniques and numerical importance sampling. In an example case, the perturbation distribution is resolved in seconds deep into its non-Gaussian tail, a speed-up of factor 109 compared to a previous study. Along the way, I comment on the role of the momentum constraint in stochastic inflation.