This paper looks at judgments of guilt in the face of alleged wrong-doing, be it in public or in private discourse. Its concern is not the truth of such judgments, although the complexity and contestability of such claims will be stressed. The topic, instead, is what sort of activities we are engaged in, when we make our judgments on others' conduct. To examine judging as an activity it focuses on a series of problems that can occur when we blame others. On analysis, we see that these problems take the form of performative contradictions, so that the ostensible purposes of assigning guilt to others are undermined. There is clear evidence from social psychology that blame is especially frequently and inappropriately attributed to individuals in modern Western societies. On the other hand, it has often been observed how suspicious we are about the activity of judging – thus a widespread perception that a refusal to judge is somehow virtuous. My suggestion is that the sheer difficulty of attributions of responsibility, in the face of a complex and often arbitrary moral reality, frequently defeats us. This leads to a characteristic set of distortions when we blame, so that it is no surprise that we have become suspicious of all blaming activities. Yet, the paper argues, these problems need not arise when we hold others responsible. This paper therefore investigates what, exactly, can be questionable about attempts to assign guilt, and the structural logic that lies behind these problems – what will be called, adapting a term from social psychology, a belief in a just world. Such a belief takes for granted what needs to be worked for through human activity, and therefore tends to be counter-productive in dealing with misdeeds and adverse outcomes.