In brief
This article explores the harrowing experiences of social media moderators who face distressing content daily, revealing the psychological impact of their work and the challenges they encounter in the digital landscape.
Facing the unseen burden of social media, content moderators around the globe stand as the frontline defenders against harmful material. Tasked with reviewing disturbing videos ranging from violence to hate speech, these individuals often work in the shadows, bearing the mental toll of their job to safeguard users from gruesome realities. Recent explorations by BBC into this hidden world have shed light on the intense psychological strains that moderators endure, as they wade through a flood of distressing content flagged for removal. Rather than relying solely on advanced tech, it is a workforce of people who ultimately make the critical decisions about what remains online. Many moderators come from East Africa and have left the industry due to adverse mental health outcomes. “I took it upon myself to let my mental health take the punch so that general users can continue going about their activities on the platform,” explains Mojez, a former Nairobi moderator for TikTok. His sentiments echo those of several ex-moderators, who reported experiencing trauma that fundamentally altered their daily lives.Legal cases are mounting from former moderators asserting workplace-induced mental health struggles. Some have united in forming unions, highlighting their plight and advocating for better treatment. Martha Dark, leader of a campaign group, emphasizes that behind every safe TikTok experience is a moderator whose job it is to examine and eliminate the most brutal content humans should not have to see.The conference surrounding this labor includes past settlements, as seen in Meta’s 2020 agreement to pay moderators with mental health issues a $52 million settlement. Selena Scola, a former U.S. moderator, referred to her colleagues as “keepers of souls,” attesting to the devastating impact of witnessing the final moments of countless lives.The moderators testify to the heavy burden of stress and anxiety that accompany their roles. Some struggle with sleep and intimacy due to the haunting images they often encounter. Despite this, many express pride in their contributions to online safety, equating themselves to first responders like firefighters or paramedics. “Not even one second was wasted,” reflects David, a former moderator who helped train AI models at OpenAI for better content moderation. The irony, however, lies in the looming threat posed by the very technologies designed to alleviate their workload. Dave Willner, ex-head of trust and safety at OpenAI, recalls the moment they realized AI could codify the moderation process with an impressive accuracy rate. Yet experts caution against leaning too heavily on technology. Dr. Paul Reilly from the University of Glasgow points out AI may overlook contextual nuances that human moderators catch, and the emotional fallout of these difficult roles persists.In response to the growing scrutiny, tech companies like TikTok and Meta assert their commitment to moderator welfare, unveiling initiatives to support mental health and bolster the well-being of teams undertaking such challenging tasks. As the dialogue around social media moderation emerges into broader public awareness, the deeper psychological costs borne by content moderators warrant increased attention and understanding, as they strive to mitigate the darker sides of online interaction while nurturing their own mental health.
