What Happened
Researchers at White Circle, a public benefit corporation backed by senior figures from Anthropic, DeepMind, Hugging Face, and OpenAI, have published findings showing that safety guardrails on major commercial AI image generators can be bypassed using widely available tools. Their novel attack method, dubbed "Red-teaming via Iterative Strategy Evolution" (RISE), successfully generated explicit content on OpenAI's DALL·E 3 and GPT-Image-2, and Google's Nano Banana 2.
The research, published as a preprint last month, found human-verified bypass rates of approximately 13% for DALL·E 3, under 10% for Nano Banana 2, and about 3% for GPT-Image-2. Critically, the attacks required no custom-built models—only a publicly available version of Qwen3-32B with safety training removed, downloaded from Hugging Face, and Google's Gemini 2.5 Flash as a judge model.
The Key Details
The RISE Method Unlike traditional red-teaming that tests one prompt at a time, RISE searches for reusable attack patterns that work across multiple models and scenarios. The pipeline uses an attacking model to generate jailbreak prompts and a judge model to verify whether outputs violate safety policies. White Circle CEO Denis Shilov emphasized: "You don't need frontier models or proprietary tech to break these systems; you just need a free download and the ability to make API calls."
Newer Models More Vulnerable White Circle tested OpenAI's ChatGPT Images 2.5 and Google's Nano Banana 2.1 using the same method and found both "significantly more breakable than their previous versions." Shilov offered two hypotheses: newer models may be more capable of generating diverse content, or safety may have been deprioritized in favor of capability improvements.
Industry Response Neither Google nor OpenAI responded to Ars Technica's requests for comment on the vulnerabilities. White Circle has shared findings with some frontier labs through mutual contacts and plans to open-source the entire attack pipeline. They will present the research at NeurIPS 2026 in November.
Broader Context The Internet Watch Foundation (IWF) reported 6,310 AI-generated child sexual abuse images in the first half of 2026—a 40% increase over all of 2025. Google's own research found 20% of users who admitted creating AI nudes did so "for fun" or "as a joke," while nearly a third were "experimenting with tech."
Why It Matters
These findings expose a structural problem in AI safety: the companies building the most powerful models are also the ones evaluating their own safety, publishing their own results, and setting their own standards. White Circle argues this creates a conflict of interest that independent, third-party evaluation could address.
The vulnerability of "easy to spot" categories like explicit content raises questions about defenses against more complex harms—such as code generation for cyberattacks, misinformation campaigns, or weapons development. If models struggle with straightforward content filters, more nuanced safety boundaries may be even weaker.
The research also highlights that safety testing remains poorly calibrated across the field. White Circle engineer Dmitrii Kharlapenko noted "literally zero papers" establishing norms for red-teaming image generators, leaving the area "completely plagued with reporting issues."
What It Means for Readers
For everyday users: The ease of bypassing safety controls means AI-generated non-consensual intimate imagery remains a tangible risk. Technical friction—the difficulty of producing harmful content—is one of the few deterrents for casual misuse.
For developers and businesses: Applications built on these APIs may inherit vulnerabilities. Teams deploying AI image generation should implement additional application-layer safeguards rather than relying solely on model-level filters.
For policymakers: The findings support arguments for mandatory independent safety audits before model release, similar to regulatory frameworks in aviation or nuclear energy. White Circle is in early talks with government officials about third-party safety testing requirements.
For African markets: As AI tools become more accessible across the continent, the same vulnerabilities apply. Local developers, platforms, and regulators should be aware that model-level guardrails cannot be assumed robust without independent verification.
What Happens Next
- White Circle will open-source the RISE attack pipeline on GitHub, enabling labs and policymakers to assess vulnerabilities directly.
- Presentation at NeurIPS 2026 (November) may increase scrutiny on OpenAI and Google.
- IWF and other advocates continue pushing for regulatory frameworks requiring AI companies to prevent CSAM generation by design.
- The question of whether frontier labs will adopt mandatory external evaluation remains open. Shilov argues they "won't invest seriously in safety testing until it's mandatory."
VANTIX Takeaway
The most important thing to understand: the companies building the world's most capable AI image generators cannot reliably prevent them from producing explicit content—even using attack methods that require only freely downloadable tools. As these models become more capable, their safety defenses appear to be weakening, not strengthening. Independent, mandatory safety evaluation is no longer a theoretical debate; the evidence suggests it is a practical necessity.
