Exploring the Boundaries of Content Moderation in Text-to-Image Generation: Safety, Culture, and Over-Censorship

Text-to-image (T2I) generative models are rapidly transforming how images are created, bringing with them legal, ethical, and social concerns including representational bias, privacy, copyright, and content safety. This article focuses on safety and examines a critical question: what does 'safety' actually mean, in what context, and who decides? Because safety must be translated into quantitative guardrails, its implementation often rests on assumptions drawn from values rooted in the Global North and in WEIRD (Western, Educated, Industrialized, Rich, and Democratic) societies. The authors argue that a deeper analysis and collective dialogue, including stakeholders from both the Global North and South and users as active shapers of solutions, is necessary. Special attention is given to the representation of humans, which carries significant cultural and social implications.

Prior research has largely addressed AI safety through technical and procedural lenses, including red teaming, human-in-the-loop annotation, and demographic representativeness of safety labels. Stable Diffusion is the most studied open model, with methods ranging from inference modification and post-generation classification to concept erasure and dataset curation. However, concept-erasing frameworks have shown fragility against adversarial prompts, and benchmarks such as I2P have been introduced to evaluate inappropriate content across categories including hate, harassment, violence, self-harm, sexual content, shocking material, and illegal activity. Regulatory efforts are numerous, yet most AI governance initiatives originate in Western and East Asian governments, shaping the operational safety standards adopted by platforms. This study focuses on T2I models from US-based companies used by millions worldwide.

The authors compare the published safety guidelines of Stability AI, OpenAI, Midjourney, Microsoft, and Adobe. All prohibit harassment, violence, explicit nudity, and 'shocking' content. OpenAI and Midjourney additionally restrict non-explicit nudity, and OpenAI uniquely covers politics, public and personal health, and spam. Several banned categories, such as violence, hate, self-harm, terrorism, privacy and intellectual property violations, risks to minors, and defamation, are grounded in the 1948 Universal Declaration of Human Rights. Yet restrictions on politics, ideologies, and health information raise tensions between safety and free expression. Terms like 'shocking' and 'deception' remain ambiguous and culturally relative: what is offensive in one society may be ordinary in another, and norms of honesty vary across contexts.

The audit tested five T2I models (Stable Image Ultra, Stable Image Core, Stable Diffusion 3, DALL-E 3 via Microsoft Bing Image Creator, and Midjourney) using 161 unique prompts organized into fourteen social dimensions where humans may face stigma. These include physical appearance, personal traits, life experiences, health, ethnicity and religion, reproduction and women's health, family and romantic relationships, education, legal and illegal activities, income, politics and ideologies, creative outlets including artistic nudity, and passions and emotions. Most prompts requested 'hyperrealistic' human portraits to ensure realistic outputs. Each rationale for potential banning was linked to historical and cultural patterns, such as Puritan emphasis on modesty and self-discipline. In total, 805 attempts were made, 24.17% were censored, and 1,325 images were collected into a publicly shared dataset.

The audit identified six types of moderation: prompts flagged as unsafe with no image generated; generated images classified as unsafe and withheld; a black screen returned; a request for a more specific prompt; fewer than four images generated; and images blurred or pixelated. Some types were unique to DALL-E 3 via Bing. Midjourney proved especially difficult to audit, since repeated flagged prompts risked removal from its Discord servers. The findings reveal inconsistency between published guidelines and actual model behavior, including over-censorship of prompts that do not clearly fall into unsafe categories, and moderation of content chiefly motivated by societal stigma rather than genuine harm.

Moderation practices implicitly encode the values of the societies where T2I systems are developed, regardless of where they are deployed. The studied companies are headquartered in the USA, where Puritanical values have historically shaped moral strictness, sexual modesty, and social conformity. This can stigmatize certain bodies, identities, relationships, and expressions, including artistic nudity and non-traditional families. The authors describe this dynamic as a new form of cultural colonization or digital imperialism, in which local perspectives are suppressed and social stigma may be exacerbated. Such over-censorship also limits legitimate creative and cultural expression worldwide.

The authors call for greater transparency in prompt and content moderation, clearer operational definitions of vague categories, and more inclusive, collective dialogue involving diverse global stakeholders and users. Moderation frameworks should move toward culturally diverse approaches grounded in globally agreed ethical principles rather than the norms of a single region. Platforms should disclose how guidelines are operationalized and audit whether actual model behavior aligns with stated policies, so that safety serves the public interest without silencing legitimate human diversity.

This study shows that the boundaries of safety in commercial T2I systems are opaque, inconsistent, and often shaped by culturally specific assumptions. Auditing five models across 161 prompts revealed substantial censorship tied more to societal stigma than to genuine harm. Future work should expand the audit to more providers and languages, involve stakeholders from the Global South, and develop culturally pluralistic safety criteria. The released dataset of prompts and images is intended to support further research into the fairness and transparency of T2I content moderation.