On-device approaches have a contested historical past, and any deployment have to be genuinely privacy-preserving by design. However expertise corporations should weigh the objection towards the harms that proceed to happen. A security by design method is required.
On-device security measures have been demonstrated at scale with Apple’s on-device nudity detection for photos despatched or obtained by way of Messages, AirDrop and FaceTime. A 2025 examine demonstrated high-accuracy grooming detection utilizing Meta’s AI mannequin designed particularly for on-device deployment on cell phones.
Lately, each Apple and Google have began to take measures in direction of app store-based age verification in some jurisdictions.
The very best-profile real-world deployment of those is Apple enabling device-level privacy-preserving age verification within the UK.
Social media and personal messaging corporations, together with working system distributors (Microsoft, Apple, and Google), all have a job to play in guaranteeing dangerous content material is detected, whether or not or not end-to-end encryption is used. Progress has been gradual. However we, as a neighborhood, have to demand extra from these corporations.
Joel Scanlan is Adjunct Affiliate Professor on the Faculty of Legislation and Educational Co-Lead at CSAM Deterrence Centre, College of Tasmania. This text first appeared in The Dialog.
