The outcome of a weak quantum measurement conditioned to a subsequent postselection (a weak value protocol) can assume peculiar values. These results cannot be explained in terms of conditional probabilistic outcomes of projective measurements. However, a classical model has been recently put forward that can reproduce peculiar expectation values, reminiscent of weak values. This led the authors of that work to claim that weak values have an entirely classical explanation. Here we discuss what is quantum about weak values with the help of a simple model based on basic quantum mechanics. We first demonstrate how a classical theory can indeed give rise to non-trivial conditional values, and explain what features of weak values are genuinely quantum. We finally use our model to outline some main issues under current research.