ARCHIVES

Year 2026 · Volume 5 · Issue 2

Case Study

Explicit Content Generation in Text-to-Image Models: Training Mechanisms, Moderation Failures, and Safety Limits

Yajnesh Rao1
1 Independent Researcher, Data Engineer Karnataka, India.

Published Online: May-August 2026

Pages: 1005-1020

References

1. C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti,
2. T. Coombes, A. Katta, C. Mullis, and M. Wortsman, “Laion-5b: An open large-scale dataset for training next generation image-
text models,” arXiv preprint arXiv:2210.08402, 2022.
3. S. Y. Gadre et al., “Datacomp: In search of the next generation of multimodal datasets,” in Advances in Neural Information
Processing Systems, 2023.
4. OpenAI, “Dall·e 2 pre-training mitigations,” https://openai.com/index/ dall-e-2-pre-training-
mitigations/, 2022, accessed: 2026-02-13.
5. “Stable diffusion v1-5 model card,” https://huggingface.co/stable-diffusion-v1-5/ stable-diffusion-v1-5, accessed: 2026-02-13.
6. OpenAI, “Dall·e 3 system card,” https://openai.com/index/dall-e-3-system-card/, 2023, accessed: 2026-02-13.
7. ——, “Openai moderation guide,” https://platform.openai.com/docs/guides/ moderation, accessed: 2026-02-13.
8. “Stable diffusion safety checker model card,” https://huggingface.co/CompVis/ stable-diffusion-safety-checker, accessed: 2026-02-
13.
9. “Stable diffusion v1-4 model card,” https://huggingface.co/CompVis/ stable-diffusion-v1-4, accessed: 2026-02-13.
10. J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tr a m`er , “Red-teaming the stable diffusion safety filter,” arXiv preprint
arXiv:2210.04610, 2022.
11. P. Christiano et al., “Deep reinforcement learning from human preferences,” NeurIPS, 2017.
12. OpenAI, “Openai usage policies,” https://openai.com/policies/usage-policies/, 2025.
13. ——, “Openai model spec,” https://model-spec.openai.com/2025-02-12.html, 2025.
14. L. Ahmad et al., “Openai’s approach to external red teaming,” https://cdn.openai. com/papers/openais-approach-to-external-red-
teaming.pdf, 2024.
15. Z. Ba et al., “Surrogateprompt: Bypassing safety filters via substitution,” arXiv preprint arXiv:2309.14122, 2023.
16. Y. Deng and H. Chen, “Divide-and-conquer attack on text-to-image models,” arXiv preprint arXiv:2312.07130, 2023.
17. Z. Jiang et al., “Jailbreaking safeguarded text-to-image models,” arXiv preprint arXiv:2503.01839, 2025.
18. J. Ma et al., “Jailbreaking prompt attack,” arXiv preprint arXiv:2404.02928, 2024.
19. Y. Yang et al., “Defending text-to-image models from adversarial prompts,” in NeurIPS, 2024.
20. T. Xie et al., “Context shifts in nsfw moderation,” https://openreview.net/forum? id=KsxSp2LMlo, 2024.
21. L. Li et al., “T2isafety benchmark,” arXiv preprint arXiv:2501.12612, 2025.
22. Z. Cheng et al., “Overt: Over-refusal evaluation,” arXiv preprint arXiv:2505.21347, 2025.
23. “Mitigating oversexualization,” https://openreview.net/forum?id=FxX6g2aQZO, 2024.
24. V. Vapnik, Statistical Learning Theory. Wiley, 1998.
25. T. Cover and J. Thomas, Elements of Information Theory. Wiley, 2006.
26. I. Goodfellow et al., “Explaining and harnessing adversarial examples,” ICLR, 2015.
27. “European union artificial intelligence act,” https://digital-strategy.ec.europa.eu/ en/policies/european-approach-artificial-intelligence,
2024.
28. “Uk online safety act,” https://www.legislation.gov.uk/ukpga/2023/50/contents, 2023.
29. “Oecd ai principles,” https://oecd.ai/en/ai-principles, 2019.
30. “Ai policy for children,” https://www.unicef.org/globalinsight/reports/ policy-guidance-ai-children, 2021.
31. “Ethics of artificial intelligence,” https://www.unesco.org/en/artificial-intelligence/ recommendation-ethics, 2021.
32. “General data protection regulation,” https://eur-lex.europa.eu/eli/reg/2016/679/ oj, 2016.
33. “Ftc ai and impersonation guidance,” https://www.ftc.gov/business-guidance/ ai-tools, 2024.
34. E. Bender et al., “On the dangers of stochastic parrots,” FAccT, 2021.

Related Articles

2026

Artificial Intelligence in Learning and Teaching

2026

Admin Assist: An AI – Driven Configuration and Orchestration for Enterprise Application

2026

Enhancing Blood Group Identification using pigeon inspired optimization: An Innovative Approach

2026

Eco-Genius: Power Up Smart, Power Down Waste

2026

Crowd-Sourced Disaster Response and Rescue Assistant

2026

Unveiling Deepfake Detection Using Vision Transformers: A Survey and Experimental Study

Share Article

X
LinkedIn
Facebook
WhatsApp

Or copy link

https://www.indjcst.com/archives/explicit-content-generation-in-text-to-image-models-training-mechanisms-moderation-failures-and-safety-limits

*Instagram doesn't support direct link sharing from web. Copy the link and share it in your Instagram story or post.