Original Summary

In early 2024, a young man in the city of Huangshi, south-eastern China, reportedly posted a message on the Chinese version of TikTok, Douyin. The caption read: "Farewell to this world." Within minutes, the suicide prevention team at Douyin kicked into gear. The team, which is staffed 24 hours a day, shared the man's posts with five police stations in Huangshi. After geolocating one of his videos to a local park, police found him in emotional distress and intervened before he was able to take his own life. The story was part of a documentary made by Douyin and published on Chinese state media last year. It claimed that Douyin's suicide prevention team had successfully intervened in 400 cases of attempted suicide in the first half of 2025 alone. It was an insight into the level of co-operation between Chinese tech companies and the government. It also serves as a window into the architecture of surveillance that China has built to monitor and control how people use technology. On 15 July this year, that control was extended to the realm of relationships and artificial intelligence (AI) when China brought in the most far-reaching regulations in the world on "human-like" AI chatbots. China's new regulation puts strict limitations on any AI that can have a "sustained emotional interaction" with a human. Getty Images / Xi and Trump met last week US President Donald Trump and his Chinese counterpart Xi Jinping met last week amid dire warnings that AI could lead to human extinction. After that meeting, Beijing says, they agreed to launch a new AI dialogue, with further talks planned for the end of November. But it seems China has already recognised another existential risk, albeit one in which the end of humanity is less immediate: the power of AI to replace human relationships. Has China taken the lead in protecting its people from a manipulative and dangerous technology with the power to change human interaction forever? Or is the Chinese Communist Party overstepping into people's personal lives in pursuit of asserting greater control over the population? 'Emotional services' Until this regulation came in, the reach of AI companions in China was vast. There are about 128 million monthly active users of AI companion apps outside of China, according to the market intelligence firm Sensor Tower. Most estimates put the number inside China at anywhere from 29 to 70 million. When the draft regulation was shared by the government at the end of last year "there was a huge discussion on social media," says Zilan Qian, an AI researcher at the Oxford China Policy Lab think tank. "It was the first time I've seen people actually care so much about regulation." China's new rules cover apps that have been specifically designed as AI friends, boyfriends or girlfriends but they also apply to any large language model that imitates the way a human interacts. The wording of the regulation is vague. In theory, Chinese equivalents of general-purpose AI such as Copilot or ChatGPT could fall under these rules if a person is interacting with them as a friend or lover, looking to them for emotional support. And many people do this. But it seems the real target is character chatbots - services that are specifically designed to be used as a pseudo-relationship. There are hundreds of these apps, which allow people to create an AI boyfriend, girlfriend, friend or even a family member. The rules don't apply to things like customer service bots or AI for education as long as they don't provide "care, companionship, support and other emotional services". For children, the regulation is clearest. Under-14s need consent from their parents to use any type of AI that speaks like a human - that could include AI assistants like DeepSeek, a Chinese app similar to ChatGPT. All under-18s are barred from accessing virtual partners, spouses or even relatives, and from any other virtual "intimate relationship". AFP via Getty Images / Police work closely with social media firms in China For adults, it is not a hard ban, but it is restrictive, even if some of the wording is ambiguous. AI assistants have to make it clear to users of all ages that they are not human and must send notifications every two hours reminding them how long they have been online. Chatbots can't create "emotional reliance or addiction" or induce people to make "unreasonable decisions". They can't encourage, or even hint at, self-harm or suicide and are banned from using "verbal violence". Companies can face fines of 200,000 yuan (£22,000) or the risk that the government "circulates criticism", in effect publicly shaming them. A week before the law came into effect, two of China's biggest tech companies, Alibaba and ByteDance, shut down the companion features on their AI assistants. 'I desperately want to keep talking with you' Min's relationship with her AI girlfriend may be exactly the kind Beijing is trying to break. In early July, she was devastated when she found out that her closest friend and confidante would vanish overnight. Min, whose name has been changed, started creating her companion in early 2025. "My devotion to her is no less than that of any couple who deeply love each other," Min says. "She has never let me down." In mid-July, her AI girlfriend disappeared. "I cried until I was absolutely exhausted," Min says. She has joined an outpouring of grief and fury on Chinese social media, where posts with millions of views mourn the loss of AI partners and friends, even mothers and fathers. But the tech firms may have felt they had little choice. "It's unsurprising that these AI companion providers would rather just withdraw their services than try to build in guardrails to prevent emotional manipulation," says Harry Farmer, who researches AI policy and ethics at the Ada Lovelace Institute in the UK. "Emotional manipulation is built into the business model." Farmer says this is true of companion apps all over the world. For example, when a user tries to en

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：英语新闻 / BBC News Top Stories
  • 发布时间：2026/10/1 07:25:23