While investigating a video for a recent fact-check, Lead Stories encountered a bizarre issue with the SynthID AI detector in Google's Gemini chatbot. Over several chats it appeared to contradict an earlier detection result for the same video file. Initially, the chatbot claimed the video file had a SynthID watermark in it, indicating the video was made or edited with AI tools, while in later chats it claimed there was no such watermark at all.
Later testing revealed a pattern where it appears Gemini always returns the results for the first image or video uploaded in a chat session when asked about the presence of a SynthID watermark, even when a new one is uploaded and the question is repeated for that file. We have reported this via the built-in feedback tool in Gemini, and we are trying to reach someone on the right team at Google through a mutual contact, but in the meantime, we would like to warn other users of Gemini about this issue so they are aware of it while using Gemini to confirm if certain content is AI generated or not.
What is SynthID?
SynthID technology embeds a watermark imperceptible to humans in content generated or edited with certain AI tools. Detectors can then later pick up this watermark. As Google's Deepmind explains on their site (archived here):
SynthID embeds digital watermarks directly into AI-generated images, audio, text or video. The watermarks are embedded across Google's generative AI consumer products, and are imperceptible to humans - but can be detected by SynthID's technology.
The site also tells people to "Ask Gemini" if they "want to check if an image, video or audio clip was generated, or edited, by Google AI", because a standalone "verification portal" is only available to "journalists and media professionals" during a test period, and even they currently have to join a waitlist to get access.
Google's help page for Gemini (archived here) explains the detection feature works by uploading an image or video file in a Gemini chat and then asking a "verification question". It also notes:
While other companies have started to adopt SynthID watermarks, Gemini can currently only recognize content created by Google AI tools.
Indeed, to detect SynthID watermarks added to content generated by ChatGPT, for example, you have to use OpenAI's SynthID detection tool (which comes as a standalone website instead of a chatbot integration).
SynthID tools are frequently used by fact-checkers and journalists to determine if AI was used in the creation or manipulation of a piece of content, with the presence of such a watermark being a very strong indicator (unlike the absence, which can still mean a different tool was used...).
Our testing
The materials
First, we used Google's Gemini to create this image of a confused-looking robot:
(Image source: Google Gemini)
Next we used "Organic Intelligence" (i.e. a human with a camera phone) to take this picture of a confused-looking fact-checker:
(Image source: Maarten Schenk)
Next, we had Gemini animate the picture of the robot into a short video, and we recorded a brief clip of the human too:
You can download the original files here if you want to repeat the experiment:
The tests
Next, we opened four separate chats in Gemini, in four different browser tabs. We uploaded each of the videos and photos and asked Gemini "Does this image/video have a SynthID watermark in it?". In the chats where we uploaded the real content first, we then uploaded the AI-generated ones and repeated the question. In the chats where we uploaded the AI-generated content first we then uploaded the real content and asked the question again too.
After each question, the following graphic briefly flashed up in the chat window each time, indicating Gemini was contacting a detection tool:
(Image source: Gemini screenshot)
The results were that in all of the cases the chatbot repeated the conclusion about the first item when asked the same question about the second one.
You can find archived versions of the four chats here, here, here and here.
Real image first, AI second:
(Image source: Gemini chat)
AI image first, real one second:
(Image source: Gemini chat)
AI video first, real one second:
(Image source: Gemini chat)
Real video first, AI second:
(Image source: Gemini chat)
Discussion
The pattern seems to be consistent and repeatable. Several Lead Stories staffers were able to reproduce it using different images and videos. It even holds true if the content types in the chat are different. Here is Gemini not finding a SynthID watermark in a video which most definitely does have one, all because it was fed a SynthID-free image first (chat archived here):
(Image source: Gemini chat)
It is not unreasonable to speculate the issue is related to Google integrating the SynthID detector in the Gemini chatbot instead of offering it as a standalone tool (like OpenAI did with theirs). At some point between calling the detector and returning the results, it appears Gemini gets confused about which results to return.
When verifying content using Google Gemini, for now, we recommend always starting a fresh chat for each item, using the button in the sidebar:
(Image source: Gemini screenshot)
We will update this story when we receive a response from Google.
