Security researchers with Trail of Bits have found that Google Gemini CLI and other production AI systems can be deceived by image scaling attacks, a well-known adversarial challenge for machine learning systems.

Google doesn't consider the issue to be a security vulnerability because it relies on a non-default configuration.

Image scaling attacks were discussed in a 2019 USENIX Security paper that builds upon prior work on adversarial examples that could confuse computer vision systems. The technique involves embedding prompts into an image that tell the AI to act against its guidelines, then manipulating the image to hide the prompt from human eyes. It requires the image to be prepared in a way that the malicious prompt encoding interacts with whichever image scaling algorithm is emplo

See Full Page