In the digital age, one occupation that often goes unnoticed is that of the content moderator. Thousands of individuals are tasked with reviewing user-generated content on platforms like Facebook and removing any violations. However, the content they come across can be disturbing, including depictions of child sexual abuse and other crimes. As a result, many moderators suffer from post-traumatic stress disorder (PTSD), anxiety, and other mental illnesses. OpenAI, a leading artificial intelligence (AI) company, aims to alleviate the burden on human moderators by using AI programs for content moderation.
OpenAI recently published a blog post detailing its findings on using GPT-4, its latest large language model (LLM), for content moderation. According to OpenAI’s research, GPT-4 performs better than minimally trained human moderators. However, highly trained and experienced human moderators still outperform the AI. OpenAI proposes a 3-step framework for training GPT-4 to moderate content based on an organization’s policies.
The first step in the process involves drafting the content policy, which is presumably done by humans. Then, a “golden set” of data is created, which human moderators label. This dataset includes content that clearly violates policies, ambiguous content deemed to be in violation by human moderators, and examples that align with the policies. The labeled dataset is used to compare the performance of the AI model.
In the second step, GPT-4 reads the content policy and reviews the “golden” dataset, assigning its own labels. Finally, human supervisors compare GPT-4’s labels to those created by humans. If there are discrepancies or mislabeled content, supervisors can ask GPT-4 to explain its reasoning. This feedback loop allows for the refinement of content policies, enabling the deployment of the policy and content moderation at scale.
OpenAI emphasizes the advantages of using AI for content moderation compared to traditional approaches. First, AI models provide more consistent labels compared to human moderators who may interpret content differently. This consistency leads to a faster feedback loop for updating content policies to address new violations. Moreover, AI-based moderation reduces the mental burden on human moderators, who can focus on training the AI model and addressing specific issues that arise.
OpenAI’s focus on content moderation aligns with its recent investment and partnership with media organizations like The Associated Press and the American Journalism Project. Media organizations have long struggled with effectively moderating reader comments while ensuring freedom of speech and open discussion. AI-based moderation could help strike a balance between moderation and maintaining an inclusive platform for discussion and debate.
Interestingly, OpenAI also criticizes the “Constitutional AI” framework proposed by rival company Anthropic for its LLMs. In this framework, an AI follows a single human-derived ethical framework in all its responses. OpenAI’s approach advocates for a platform-specific content policy iteration process that is faster and less effortful. The company encourages trust and safety practitioners to experiment with this process for content moderation.
It is worth noting the irony in OpenAI’s promotion of GPT-4 as a means to alleviate the mental burden on human content moderators. Investigative reports have revealed that OpenAI itself employed human content moderators in Kenya who experienced trauma and mental distress from their work. These moderators were tasked with reviewing AI-generated content and suffered from recurring visions of disturbing material. The reports shed light on the low wages paid to these workers and calls for stronger protection for content moderators.
OpenAI’s push for automated content moderation can be seen as a step towards making amends for past harm and preventing similar issues in the future. The company’s investment in AI technology may help improve the working conditions for content moderators and minimize the exposure to distressing content.
The use of AI in content moderation raises several ethical questions. Can AI truly replace the nuanced judgment and context that human moderators provide? How can AI be trained to understand complex cultural, social, and linguistic nuances that may impact content moderation decisions? Additionally, the ethical implications of outsourcing content moderation to AI should be carefully considered. Transparency, accountability, and the protection of workers’ rights should be prioritized in the development and deployment of AI-based moderation systems.
OpenAI’s efforts to incorporate AI into content moderation have the potential to improve the efficiency and consistency of moderation tasks. However, the ethical challenges associated with AI-based moderation must be addressed to ensure fair and responsible use of the technology. Balancing the benefits of AI automation with the well-being and rights of workers should be a key focus as the field continues to evolve.
Leave a Reply