We present new direct constraints on a general $Wtb$ interaction using data corresponding to an integrated luminosity of 5.4 fb$^{-1}$ collected by the D0 detector at the Tevatron $p\bar{p}$ collider. The standard model provides a purely left-handed vector coupling at the $Wtb$ vertex, while the most general, lowest dimension Lagrangian allows right-handed vector and left- or right-handed tensor couplings as well. We obtain precise limits on these anomalous couplings by comparing the data to the expectations from different assumptions on the $Wtb$ coupling.