Crime & Justice

OpenAI confirms use of ChatGPT by Chinese-linked cyberattack campaign

Beijing used artificial intelligence to meddle in Japanese politics and to torment Chinese dissidents overseas.

Illustration showing a smartphone displaying a ChatGPT interface against a backdrop of China's flag and digital code, symbolizing artificial-intelligence-enabled influence operations and coordinated online networks. [Focus]
Illustration showing a smartphone displaying a ChatGPT interface against a backdrop of China's flag and digital code, symbolizing artificial-intelligence-enabled influence operations and coordinated online networks. [Focus]

By Jia Feimao |

A Chinese-linked influence operation used generative artificial intelligence (AI) tools, including OpenAI's ChatGPT, to support online harassment and disinformation about critics of Beijing at home and abroad, according to a new security report from the US tech firm OpenAI.

The activity sheds light on how bad actors increasingly are weaving AI tools into information operations, cybercrime and political influence campaigns across the Indo-Pacific and beyond.

OpenAI investigators uncovered accounts tied to Chinese actors who used ChatGPT to help document and coordinate elements of covert online campaigns, the report, released February 25, said.

Targeting Japan's Takaichi and Chinese dissidents

One case involved a plan to smear Japanese Prime Minister Sanae Takaichi through coordinated online messaging, while other operations victimized Chinese dissidents overseas.

Japanese Prime Minister Sanae Takaichi (R) waves during a campaign speech ahead of the House of Representatives election in Tokyo on February 7. Investigations later found thousands of online accounts spreading disinformation about her government during the election period. [Philip Fong/AFP]
Japanese Prime Minister Sanae Takaichi (R) waves during a campaign speech ahead of the House of Representatives election in Tokyo on February 7. Investigations later found thousands of online accounts spreading disinformation about her government during the election period. [Philip Fong/AFP]

OpenAI said it ultimately banned the accounts after detecting the activity, which investigators described as a sustained influence operation carried out across numerous online platforms.

"This effort appears to be large-scale, resource-intensive, and sustained, engaging at least hundreds of staff, thousands of fake accounts across scores of platforms, and the use of locally-deployed AI models, especially Chinese ones," the report said.

The findings illustrate how authoritarian governments may be incorporating AI into existing surveillance and information control systems.

"This is what Chinese modern transnational repression looks like," said OpenAI principal investigator Ben Nimmo, according to CNN.

"It's not just digital. It's not just about trolling. It's industrialized. It's about trying to hit critics of the CCP [Chinese Communist Party] with everything, everywhere, all at once," Nimmo said ahead of the report's release.

In a notable case last year, a Chinese operative used ChatGPT to document a covert campaign against the then-incoming Japanese Prime Minister Sanae Takaichi.

The operator sought a multistep plan to discredit Takaichi by amplifying public anger over US tariffs. While ChatGPT refused the request, investigators found that similar attack hashtags later appeared on a Japanese artist forum. The discovery suggests that the campaign used other tools.

Although the chatbot rejected the initial prompts, the user later used it to edit progress reports, accidentally exposing a broader strategy of going after political opponents and dissidents globally.

The user's behavior "revealed a well-resourced, meticulously orchestrated strategy for covert IO [influence operations] against domestic and foreign adversaries," the report said.

Beijing's tactics against dissidents overseas included impersonating US immigration officials and forging court documents in an effort to engineer the removal of a dissident's social media account. Investigators identified a campaign to spread "fake news" of a dissident's death with a phony obituary and photos of a tombstone.

Meddling in Japanese election

The findings coincided with separate investigations in Japan of coordinated online influence activity around the country's snap election earlier this year.

Thousands of social media accounts appeared to promote disinformation about Takaichi's government ahead of the election, Japan Nexus Intelligence Inc., a cybersecurity analytics firm, found, according to Yomiuri Shimbun. The campaign appeared designed to intensify social divisions and weaken Japan's international standing.

The analysis suggested about 1,000 accounts generated original posts while another 2,000 accounts amplified them through reposting.

Some messages contained telltale artifacts of translation or unusual Japanese phrasing. Images circulating in the campaign appeared to originate from Chinese state media or AI-generated content, according to the analysis.

Dozens of X accounts linked to Chinese disinformation operations circulated corruption allegations and portrayed Takaichi as a warmonger in the run-up to the vote, researchers at the Washington-based Foundation for Defense of Democracies (FDD) said.

"China has the political will to interfere in Japanese elections and internal affairs," Maria Riofrio, a researcher at the foundation, told Reuters.

At least 327 networked accounts

The accounts formed part of a broader network of at least 327 social media accounts that had been active since December or earlier in hounding Beijing's critics or promoting pro-China narratives, said Riofrio. The network additionally targeted human rights organizations and sought to influence political discourse in Japan, the United States, the Philippines and parts of Latin America.

A spokesperson for the Chinese embassy in Washington rejected the analysis, calling the FDD's conclusions "groundless."

AI is becoming a new tool of repression, a Chinese dissident influencer whom the report cited as a target of Beijing's operations posted on X in February.

The activist, whose X account bears the alias "Teacher Li Is Not Your Teacher," urged social media platforms to detect coordinated state-backed harassment campaigns rather than leaving victims to fend for themselves.

Do you like this article?

Policy Link