Researchers from Binghamton University have developed an artificial intelligence (AI) tool that can detect fake, generative AI-created scientific reports.

The team has created a machine-learning algorithm dubbed xFakeSci that can reportedly detect up to 94% of fake scientific papers — almost twice as successfully as more common data-mining techniques — created by generative AI programs like ChatGPT.

To develop the tool, the Binghamton researchers, in collaboration with a team from Hefei University of Technology in China, created 50 fake articles for each of three specific medical topics — including Alzheimer's, cancer and depression. Those fake articles were then compared to the same number of authentic articles on the same topics.

xFakeSci was programmed to analyze two features associated with how the papers were written. The first feature involved the number of bigrams, which are two words that often appear in combination — including "climate change," "clinical trials" or "biomedical literature,” for example — the team explained, Meanwhile, the second feature involved how those bigrams were linked to other words and concepts within the text.

"The first striking thing was that the number of bigrams were very few in the fake world, but in the real world, the bigrams were much more rich. Also, in the fake world, despite the fact that there were very few bigrams, they were so connected to everything else," the researchers concluded.

Further, the team theorized that the writing styles differ because human researchers do not have the same intentions as AIs prompted to create a report on a given topic.

"Because ChatGPT is still limited in its knowledge, it tries to convince you by using the most significant words," the researchers explained. "It is not the job of a scientist to make a convincing argument to you. A real research paper reports honestly about what happened during an experiment and the method used. ChatGPT is about depth on a single point, while real science is about breadth."

An article detailing xFakeSci, “Detection of ChatGPT fake science with the xFakeSci learning algorithm,” appears in the journal Scientific Reports.