Through the Safety Tech Challenge Fund, the UK Government awarded funding to five projects to prototype innovative, automated technologies to help keep children safe in E2EE environments, such as online messaging platforms, while ensuring user privacy is respected.

To enable academic scrutiny, the UK National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online (REPHRAIN) was requested by the Department for Culture Media and Sport to act as an independent, external evaluator to each of these five projects. This is the first independent, public evaluation of automated industry tools developed for online child protection.

The report presents:

The finalised version of the evaluation criteria, following public consultation. Aside from performance these also include criteria focusing on human rights impact, security, explainability, transparency, fairness, accountability, etc. The criteria are intended to highlight the trade-offs that are faced when selecting different approaches for online child protection purposes in the context of E2EE environments. Additionally, they can be used as a guidance by the safety tech industry to help build public trust in their systems, to positively influence AI technology developments for online child protection, and to ensure all users benefit from these solutions.
A case study in which these criteria were implemented as part of a formal evaluation of each Proof-of-Concept (PoC) tool funded by the Safety Tech Challenge Fund. We describe the presence or absence of different measures that assure compliance with each criterion and provide guidance where possible when certain criteria were not met.
A discussion on how future research can support a framework for evaluating CSAM detection and prevention tools.