AI Face-Swap Fraud Is Rampant: How Should Platform Liability Be Determined? A 2025 Lawyer’s Interpretation of Three Major Compliance Obligations

AI face-swapping and AI voice cloning are turning fraud from skilled work into an assembly line. In the past, fraud by impersonating an acquaintance required a criminal group, script training, and multiple layers of transfers; now, a ten-second voice sample and a photo from a social media feed are enough to synthesize a convincing video call in minutes. The cost of fraud has plummeted from tens of thousands of yuan to a few hundred yuan, or even zero. When victims come knocking, platforms often deflect by citing technological neutrality or users' own carelessness. How, under the law, should this account be settled?

I. The Industrialization of AI Fraud: Legal Characterization Behind Low Costs

From a criminal law perspective, the core charge for using AI face-swapping to commit fraud remains fraud under Article 266 of the Criminal Law. However, the involvement of technological means has significantly expanded the boundaries for determining joint crime and the crime of aiding information network criminal activities. For example, providing AI face-swapping software, selling voice cloning models, or training deepfake models on behalf of others may constitute accomplice liability for fraud if the person knows others will use them for fraud; if neutral technology is merely provided but is extensively used for crime, it may fall under Article 287-2 of the Criminal Law, the crime of aiding information network criminal activities.

In 2024, the Supreme People's Court, the Supreme People's Procuratorate, and the Ministry of Public Security expressly provided in the Opinions on Several Issues Concerning the Application of Law in Handling Criminal Cases Such as Telecom and Network Fraud (II) that providing fraud groups with technical support, advertising promotion, payment settlement, and other assistance, once a certain amount is reached, is sufficient to constitute

For ordinary individuals, this means that once defrauded, when reporting to the police they cannot focus only on the person who directly scammed them out of money. Third parties that provide AI tools, traffic-diversion channels, or payment-receiving accounts may all become targets of legal liability. In legal practice, we often advise victims to do three things immediately: preserve chat records and transfer receipts, request that the platform disclose the other party’s registration information, and submit leads to the public security authorities regarding the offense of assisting information network criminal activities.

II. Platform Liability: The Safe Harbor Is Not a Universal Shield

Platforms often invoke the notice-and-takedown rule under Article 1195 of the Civil Code, arguing that they bear only a post hoc deletion obligation. But what makes AI fraud special is that the fraudulent conduct is often completed within minutes; by the time the victim notifies the platform, the money has long since been transferred away. At that point, speaking of notice-and-takedown again offers almost no meaningful remedy to the victim.

Article 21 of the Anti-Telecom and Online Fraud Law provides that telecom business operators and internet service providers shall monitor and identify abnormal accounts and abnormal communication activities involved in fraud, and take measures such as re-verification, restricting functions, and suspending services. Article 38 further clarifies that failure to perform the above obligations and thereby causing harm to others shall give rise to civil liability in accordance with the Civil Code and other laws. This means that platforms’ ex ante monitoring obligation has risen from a “moral advocacy” to a statutory obligation.

In judicial practice, courts are tightening their determinations of platform liability. In 2024, in a batch of AI face-swapping fraud cases, the Hangzhou Internet Court held that if a platform already has the ability to identify AI-synthesized content but fails to set warnings or blocks for high-risk scenarios (such as video verification before large transfers), it may be found to have failed to fulfill its duty of reasonable care and be required to bear supplementary liability for compensation in a corresponding proportion. The proportion is usually between 10% and 30%, depending on the platform’s technical capabilities and degree of fault.

III. Compliance Obligations Checklist: Three Things Platforms Must Do

Taking into account the Anti-Telecommunications and Internet Fraud Law, the Personal Information Protection Law, and the Provisions on the Administration of Deep Synthesis of Internet Information Services, lawyers recommend that platforms implement at least three compliance measures.

First, labeling and detection of deep synthesis content. Article 17 of the Administrative Provisions on Deep Synthesis in Internet Information Services requires that deep synthesis service providers shall add labels to generated content that do not affect its use. Platforms, as dissemination channels, should deploy AI-synthesized content detection capabilities to label or block suspected face-swapped videos and cloned voices.

Second, proactive intervention in high-risk scenarios. For scenarios involving transfers, identity verification, customer service communications, and the like, platforms should set up "AI-synthesis risk alerts"; for example, a pop-up in the video call interface should read, "The other party's image may have been processed by AI; please verify their identity through other channels." This is not only a compliance requirement, but also a concrete manifestation in cyberspace of the safety protection obligation under Article 1198 of the Civil Code.

Third, cooperation with investigations and data retention. Platforms shall, in accordance with Article 34 of the Anti-Telecom and Online Fraud Law, retain the relevant logs for no less than six months and promptly provide them when public security organs conduct investigations. In practice, in many cases missing platform logs have led to the interruption of investigative leads, and platforms may consequently face administrative penalties or even be sued by victims seeking to hold them liable for fault.

IV. Protection of Victims’ Rights: Three Paths and Two Key Actions

If you or a family member fall victim to AI fraud, do not just call the bank’s customer service first. The correct steps are: immediately call 110 or 96110 and ask the public security authorities to initiate an emergency stop-payment; at the same time, file a complaint through the platform’s official channels, requesting that the accounts involved be frozen and evidence be preserved.

There are three avenues for safeguarding rights. Filing a criminal report is the main line, seeking to recover the money through the recovery of stolen assets and compensation for losses; filing a civil lawsuit against the platform, asserting under Article 38 of the Anti-Telecommunications and Online Fraud Law that it failed to fulfill its monitoring obligations; and filing an administrative complaint, reporting the platform’s compliance loopholes to the cyberspace and telecommunications authorities to push it to rectify them.

Two key actions determine success or failure: first, preserving evidence, including the other party's account information, chat records, transfer receipts, and samples of AI-synthesized content, preferably through notarization or blockchain-based evidence preservation; second, racing against time, as the 30 minutes after being scammed is the golden window for stopping payment, while after more than two hours the funds have often already been transferred through multiple layers.

AI has lowered the cost of fraud, but it should not lower the level of legal protection for victims. Platforms cannot enjoy the benefits of technology while shifting all the risks onto users. For ordinary people, remember one bottom line: any video call involving a transfer may be AI-generated; hang up and verify through another channel.

Guangdong Zhiming Law Firm has long focused on cybercrime and platform compliance. It can provide victims with full-process legal services, including criminal reporting, civil claims, and evidence preservation, and can also provide Internet companies with plans for building compliance systems against telecom and online fraud. Technology moves fast; the law cannot lag half a beat.