We extend the notion of ascent and descent for an operator acting on a vector space to sets of operators. If the ascent and descent of a set are both finite then they must be equal and give rise to a canonical decomposition of the space. Algebras of operators, unions of sets and closures of sets are treated. As an application we construct a Browder joint spectrum for commuting tuples of bounded operators which is compact-valued and has the projection property.