OpenAI bans suspected Chinese accounts using ChatGPT to plan surveillance

OpenAI has banned ChatGPT accounts believed to be linked to Chinese government entities attempting to use AI models to surveil individuals and social media accounts.

“What we saw and banned in those cases was typically threat actors asking ChatGPT to help put together plans or documentation for AI-powered tools, but not then to implement them,” Ben Nimmo, principal investigator on OpenAI’s Intelligence and Investigations team, told reporters.

One now-banned user, suspected to be using a VPN to access the AI service from China, asked ChatGPT to design promotional materials and project plans for a social media listening tool, described as a “probe,” that could scan X, Facebook, Instagram, Reddit, TikTok, and YouTube for what the user described as extremist speech, and ethnic, religious, and political content.

Since the company started producing threat reports in February 2024, OpenAI said it has banned more than 40 networks that violated its usage policies.

To read the complete article see: The Register