The Framing Problem: “Safety” as a Cryptographic Requirement
When a government or platform proposes scanning private messages for child sexual abuse material (CSAM), the technical framing is often deceptively simple: “We just want to stop bad content.” In practice, this translates into a requirement that an end-to-end encrypted system must also allow a third party—Apple’s proposed neuralMatch, Microsoft’s PhotoDNA on-device, or a server-side hash-matching relay—to inspect plaintext or near-plaintext before it reaches the recipient. The entity doing the inspection is rarely a neutral auditor; it is the platform itself, acting under legal compulsion or a Terms of Service clause. The core cryptographic tension is that any mechanism capable of reliably identifying prohibited content within an encrypted payload introduces a confidentiality downgrade that is indistinguishable, at the protocol level, from a general-purpose backdoor. This article examines the exact technical mechanisms proposed, the adversarial models they must survive, and the verification gap that makes them dangerous even when the designers are well-intentioned.
How Client-Side Scanning Architectures Actually Work
To move beyond rhetoric, we need to examine the three dominant architectural patterns for content scanning in end-to-end encrypted (E2EE) systems. Each pattern makes different assumptions about the adversary’s capabilities and the user’s threat model.
1. On-Device Perceptual Hashing
This is the model Apple proposed (and later suspended) for iCloud Photos. Before an image is uploaded to iCloud, the device derives a NeuralHash—a perceptual hash that maps visually similar images to identical or near-identical hash values. The hash is compared against a database of known CSAM hashes, which is stored on-device as a blinded set using threshold secret sharing. If a match occurs, a safety voucher is generated: the device encrypts the match result and a low-resolution version of the image under a key held by Apple, but only after a threshold number of matches is reached. The voucher is uploaded alongside the encrypted photo, and Apple can decrypt it only when the account exceeds the threshold.
Adversary capability required: The attacker must either (a) compromise the blinded hash database distribution mechanism to inject non-CSAM hashes, (b) reverse-engineer the NeuralHash model to find universal collisions, or (c) coerce Apple into lowering the match threshold to 1 and altering the hash set. All three are within the capabilities of a well-resourced state actor. The database is signed by a non-governmental organization, but the verification path relies on the operating system’s trust anchor—which the platform controls.
2. Server-Side Encrypted Matching
Some proposals keep scanning on the server by having the client encrypt the message with a special “scanning key” in addition to the recipient’s public key. The server holds the corresponding private key and can decrypt a derivative of the ciphertext to perform matching. This is sometimes called “triple-encryption” or “escrowed matching.” The cryptographic primitive is often a variant of identity-based encryption or attribute-based encryption where the scanning key is an attribute.
Adversary capability: The server holds a decryption-capable key. A compromise of the server infrastructure, a legal order compelling key disclosure, or a malicious insider all convert the system into one with no effective encryption against the scanning entity. The distinction between “scanning for CSAM” and “reading everything” is a policy control, not a cryptographic one.
3. Message Franking and Reporting
Used in Facebook Messenger’s “secret conversations” reporting flow, message franking allows a recipient to report an abusive message and provide cryptographic proof to the platform that the reported message was genuinely sent by the purported sender. The platform never scans proactively; it only verifies after a report. This is the least invasive model because it preserves confidentiality until a recipient—a human—initiates the process. The cryptographic primitive is a franking tag: a MAC generated by the sender that the recipient can forward to the platform along with the plaintext, allowing the platform to verify authenticity without being able to forge tags.
Adversary capability: The platform cannot unilaterally decrypt messages. However, a compromised or coerced recipient can retroactively report any message. The system shifts the trust assumption from the platform to the communication partner, which is a more acceptable trade-off in many threat models.
The Verification Gap: Why Audits Cannot Close the Loop
Proponents of client-side scanning often invoke “verifiable transparency”—the idea that the hash database, the scanning logic, and the reporting mechanism can all be audited by third parties. In theory, this prevents a platform from secretly expanding the scope of scanning. In practice, the verification chain has a fatal gap: the user cannot verify what their own device is doing in real time.
Consider the NeuralHash deployment on iOS. Apple published the hash database and the model weights. A determined researcher could, in principle, confirm that the published model matches the published hashes. But the user has no way to confirm that the running model on their device is the published one. The operating system controls the execution environment. A targeted update—or a legal order compelling one—could replace the model with an arbitrary classifier, and the user would see no visible change. The device’s secure enclave can attest to the integrity of the software, but the attestation is verified by Apple’s servers, not by the user. This is a trusted computing problem, not a transparency problem.
To make this concrete: an adversary with lawful access capabilities can compel the platform to push a configuration change that (a) lowers the match threshold to 1, (b) replaces the CSAM hash set with hashes of protest imagery or dissident content, and (c) disables the user-facing notification. The platform’s public transparency report would show no change because the configuration is ephemeral and targeted. The user’s device would show no change because the scanning occurs below the user-interface layer. The auditor would see no change because the published artifacts remain unchanged. This is not a hypothetical; it is the documented behavior of mobile device management frameworks repurposed for surveillance.
Cryptographic Properties That Scanning Violates
To understand why even “privacy-preserving” scanning is architecturally dangerous, we must examine which formal security properties are sacrificed. End-to-end encryption typically provides three guarantees:
- Confidentiality: Only the intended recipients can read the plaintext.
- Integrity: Messages cannot be modified in transit without detection.
- Authenticity: The recipient can verify the sender’s identity.
Client-side scanning violates confidentiality by design: a third party gains access to plaintext-derived information without the sender’s consent. But it also introduces a more subtle violation: participant deniability. In a properly designed E2EE protocol like Signal’s X3DH with double ratchet, neither party can cryptographically prove to a third party what the other said. Scanning systems that generate vouchers or franking tags create exactly such proof. The sender can no longer plausibly deny having sent a particular message, because the platform now holds a cryptographic artifact binding the sender’s identity to the content. This transforms a private conversation into a signed statement, with profound implications for journalists, whistleblowers, and anyone operating under a regime that criminalizes speech.
Practical Impact: Who Actually Loses Protection?
The stated target of these systems is child safety, and the moral weight of that goal is not in dispute. But the technical reality is that scanning systems are trivially evaded by sophisticated offenders. Anyone with basic operational security can compress, re-encode, or encrypt content before it reaches the scanning layer. The scanning system primarily affects users who are not evading detection—ordinary people whose private photos, medical images, or intimate conversations now pass through a classifier that may generate false positives and route their content to human reviewers.
False positives are not a theoretical concern. Perceptual hashing systems have a measurable false-positive rate. When Microsoft deployed PhotoDNA, independent researchers demonstrated collisions with benign images. When Apple published its NeuralHash collision test, researchers found universal adversarial perturbations that could cause arbitrary images to match known CSAM hashes. A false positive in this context means an innocent user’s private photo is flagged, encrypted under a platform-held key, and potentially reviewed by a human moderator. The user may never know this occurred. The harm is not hypothetical; it is a direct consequence of the base rate problem: even a tiny false-positive rate applied to billions of daily uploads produces a large absolute number of erroneous flags.
Constructive Alternatives That Preserve Encryption
Rejecting client-side scanning does not mean rejecting child safety efforts. Several approaches address the problem without breaking encryption’s security properties:
User-Initiated Reporting with Cryptographic Binding
Signal’s sealed sender and profile-key-based reporting allow a recipient to report abusive messages while preserving sender anonymity against the platform. The platform receives a report that is cryptographically bound to the conversation but cannot determine the sender’s identity unless the sender has opted into discoverability. This model respects participant deniability while enabling enforcement against abusive accounts. The trade-off is that it requires a human to initiate the report, meaning it does not catch content where both parties are complicit. For CSAM distribution networks, however, complicit parties are the norm, and traditional investigative techniques—infiltrating distribution rings, following financial trails, and using open-source intelligence—remain more effective than mass scanning.
Metadata Analysis Without Content Inspection
Encrypted messaging systems inevitably leak metadata: who is talking to whom, when, how often, and from which IP addresses. While metadata can be sensitive, it can also be analyzed in aggregate to identify networks exhibiting patterns consistent with CSAM distribution—sudden influxes of new contacts, high message volumes to unknown recipients, or connections to known bad actors identified through other means. This analysis can be performed on the server side without accessing plaintext, and it can generate leads for human investigators without compromising the confidentiality of all users. The key is that metadata analysis is targeted and auditable: the criteria for flagging an account can be published, and the number of accounts flagged can be reported transparently.
Open-Source Client Builds with Reproducible Verification
If a platform insists on deploying on-device scanning, the minimum acceptable safeguard is reproducible builds. The user must be able to compile the client from publicly available source code and verify, through a deterministic build process, that the binary running on their device matches the source. This does not eliminate the risk of a targeted update, but it makes such an update detectable by the security research community. When combined with a user-visible indicator that scanning is active—and a user-controllable toggle to disable it—the trust model shifts from “trust the platform” to “trust, but verify.” Currently, no major platform offers reproducible builds for their E2EE clients with integrated scanning.
FAQ
Can’t homomorphic encryption solve this? Scan the encrypted data without decrypting it?
Homomorphic encryption allows computation on ciphertexts, but the output of that computation is itself encrypted. To determine whether an image matches a CSAM hash, someone must decrypt the result. If the platform holds the decryption key, the confidentiality guarantee is lost. If the user holds the key and must consent to decryption, the system cannot perform covert scanning. There are proposals using multi-party computation where the user and platform jointly compute the match, but these require the user to actively participate in scanning their own content—a non-starter for surveillance use cases. Homomorphic encryption also imposes a computational overhead of several orders of magnitude, making it impractical for real-time message delivery.
What about the argument that “if you have nothing to hide, you have nothing to fear”?
This argument misunderstands the function of encryption in a digital society. Encryption protects not only content but also context. A photo that is innocent in one context—a parent photographing their child at bath time, a doctor documenting a skin condition—becomes suspicious when viewed by a stranger without context. The scanning system strips context and applies a binary classifier. The “nothing to hide” framing assumes a static, benevolent legal environment. Laws change. Governments change. The database of hashes can be updated to include content that is legal today but criminalized tomorrow. Encryption that is weakened for one purpose is weakened for all purposes.
How can I verify whether my messaging app is scanning my content?
For closed-source apps, you cannot definitively verify the absence of scanning. For open-source apps, you can inspect the code, but you must also verify that the binary you are running matches the source—which requires reproducible builds. On mobile platforms, you can use network analysis tools to monitor outbound traffic for unexpected connections to content-classification services. On desktop, you can use application firewalls to block connections to known scanning endpoints. However, these are detective controls, not preventive ones. The most reliable method is to use messaging systems that have publicly and credibly committed to never implementing client-side scanning, and that have a protocol design that makes such scanning architecturally incompatible with their encryption guarantees.
What is the difference between client-side scanning and server-side scanning for encrypted messages?
Client-side scanning performs content analysis on the user’s device before encryption, meaning the plaintext or a derivative is accessible to the scanning module. Server-side scanning requires the server to decrypt the content, which means the server holds (or can obtain) the decryption keys. Both models break end-to-end encryption’s confidentiality guarantee, but they differ in where the trust boundary is violated. Client-side scanning places the violation on the user’s device, where it is harder to audit. Server-side scanning places it on the platform’s infrastructure, where it is more visible but also more centralized. The franking model avoids both by requiring a human recipient to initiate the reporting process.
Where This Leaves the Ecosystem
The debate over client-side scanning is not a debate between privacy and safety. It is a debate about whether we are willing to accept that the same mechanism that scans for CSAM today will, with absolute certainty, be used to scan for other content tomorrow. The technical community has a responsibility to explain this not as a slippery-slope fallacy but as a direct consequence of how these systems are architected. A backdoor is not defined by the intent of its designers; it is defined by the capabilities it provides to those who control it. Any system that allows a third party to access plaintext-derived information without the sender’s knowledge or consent is a backdoor, regardless of the nobility of its stated purpose.
The constructive path forward involves investing in investigative techniques that do not require mass surveillance, building systems with verifiable privacy properties, and accepting that some problems cannot be solved by technology without creating larger problems in the process. The next article in this series will examine the specific cryptographic primitives used in Apple’s NeuralHash and the practical collision attacks that researchers have demonstrated against them.