When Put to the Test, Are We Any Good at Spotting AI Fakes?

But not all AI-generated photos are so obviously fake. Sometimes careful observation can reveal a clue that something looks off, but other times, AI-generated photos are indistinguishable from real photographs. So, how often and in what contexts can people tell the difference between real and fake photos?

To find out, Kellogg assistant professor of management and organizations and deepfake expert Matthew Groh and his team conducted a large experiment involving more than 50,000 participants, the results of which they presented at the 2025 CHI Conference on Human Factors in Computing Systems.

He and his Northwestern colleagues—PhD student Negar Kamali, research assistant Karyn Nakamura, postdoctoral researcher Aakriti Kumar, and professor Jessica Hullman—along with Angelos Chatzimparmpas of Utrecht University found that (beyond the accuracy expected in random guessing) people were able to distinguish between real and fake images in about five out of every six images they saw.

People’s accuracy varied widely depending on the complexity of a photo, the kinds of distortions it contained, and the amount of time participants spent looking at it.

These findings motivated the researchers to create a taxonomy that characterizes the different kinds of artifacts that appear in AI-generated images, the ultimate goal being to help boost people’s ability to distinguish real from fake images, especially in light of ever-improving image-generation platforms.

“We as humans have always been concerned about fakes from a philosophical standpoint,” Groh says, “but many of these images are so obviously fake. We wanted to see how big of a problem it is right now and to try to create clarity for people.”

Why it’s important for humans to spot fakes

Though software capable of manipulating photos has been around for decades, since 2022, new text-to-image tools have emerged that can turn users’ text prompts into images using diffusion models. Examples of such tools include Stable Diffusion, Firefly, Midjourney, Reve, Recraft, Imagen, Ideogram, and GPT-4o.

Understanding how well humans spot these deepfakes is important, Groh notes, since machine-learning models themselves often can’t identify fakes, especially after the images have been cropped, compressed, and changed in other subtle ways.

For the study, the team created a dataset of 149 real photographs curated from the internet and 450 images generated using AI text-to-image tools including Midjourney, Firefly, and Stable Diffusion. Both the real and AI-generated images depicted similar scenarios, and the AI-generated images were selected from a larger dataset of 3,000 AI-generated images based on images that looked most photorealistic to the team.

The team then set up an online experiment, where participants viewed a random arrangement of these images and indicated whether they thought each was real or fake. After they viewed five images, participants were then randomized into one of five groups that viewed each of the remaining images for a certain amount of time: one second, five seconds, ten seconds, twenty seconds, and unlimited time.

“We were curious about how opinions changed after you’ve looked at something for more than a couple seconds,” Groh says. “If you’re scrolling through your social-media feeds, you might not be able to spot a fake, but if you can shine a spotlight of visual attention on an image to make sense of all the relationships within the image, you may be much better at finding fakes.”

More than 50,000 participants took the test, contributing nearly 750,000 observations. Participants could view and categorize as many images as they liked. Most saw at least seven images, but some only saw one image, and one participant saw more than 500.

The longer you look

Overall, participants correctly identified AI-generated images 76 percent of the time and real photographs 74 percent of the time. “That’s very much in line with what other experiments have found,” says Groh. “It’s halfway between random guessing and perfect identification.”

But accuracy among participants varied widely; some only identified AI images correctly 32 percent of the time, while others were accurate 100 percent of the time.

Accuracy increased with more-complex images, like photos of groups—because there was a bigger chance of the AI platform getting something wrong, like weird-looking hair or inconsistent lighting. “Specifically, AI-generated simple portraits were harder to detect than group photos,” says Kamali.

Perhaps unsurprisingly, longer viewing times also increased accuracy. But the magnitude of the increase was impressive: with one second of display time, participants were accurate 72 percent of the time for AI images; at five seconds, accuracy increased to 77 percent, and at ten seconds, to 80 percent.

“That’s a big jump in accuracy,” Groh says. “If you just take a few more seconds to look at an image, you will be much better at determining if it is AI-generated. It’s a simple intervention that anyone can use.”

Of note, no real photo was identified with more than 91 percent accuracy, while some fake images were nearly unanimously identified as fake. “Real images are harder to say for sure that they are real, but some fakes are just so obvious,” Groh says.

Knowing where to look

Furthermore, Groh and his colleagues created a taxonomy of common issues associated with AI-generated images, from functional implausibilities like a woman holding a sandwich sideways to stylistic artifacts like waxy skin. (The taxonomy provides five telltale signs that an image is AI-generated.)

In general, participants were the least accurate at categorizing AI images that had functional implausibilities.

In contrast, “anatomical errors like unrealistic body proportions were easiest to spot,” Kamali says.

“These categories give people a way to talk about [AI images],” adds Groh. “So often, people are just looking at hands to find extra fingers. But there are these other possibilities of what can go wrong. Now, people can look at the hands, then look to see if the photo looks waxy, or if the shadows are wrong, and keep going down the list.”

A toolset for digital literacy

The team ran a second experiment with a second batch of images generated from the same prompts as the first dataset but without any human curation. And this time, participants were much more accurate at categorizing the images.

“People have a lot of anxiety that there’s going to be a deluge of disinformation from these AI-diffusion models,” Groh says. “But most of the images that are produced by AI today are not photorealistic. I don’t see a deluge of photorealistic images happening in 2025. A deluge of AI slop, however, is a different matter.”

The team plans to use this data and their new taxonomy to create interventions that can help people get better at distinguishing real from fake photos.

“We want to guide people where to look, especially if they only have a few seconds,” Groh says. “That way, you can engage their attention to help them make more-informed decisions so they don’t fall for deepfake images.”

Their taxonomy is relevant to videos as well. In many ways, fake videos are easier to spot; there are more opportunities for contradictions. But as technology improves, viewers will need to pay even closer attention to spot these implausibilities.

“We want people to have a useful framework to make informed choices about the media that comes at us,” Groh says.