Gaze interaction paradigms rely on the user needing to look at objects in the interface to select them or trigger actions. ”Not looking” is an atypical and unexpected interaction to perform, but the eye-tracker can sense it. We illustrate the use of ”not looking” as an interaction dynamic with examples of gaze-enabled games. We created a framework containing a spectrum of five discrete categories for this unexpected use of gaze sensing. For each category, we analyse games that use gaze interaction and make the user look away from the game action up to the extent they close their eyes. The framework is described based on whether specific game events mean the player might not; cannot; should not; must not; or does not look. Finally, we discuss the outcomes of using unexpected gaze interactions and the potential of the proposed framework as a new approach to guide the design of sensing-based interfaces.