Unpaid social media moderators perform labor worth $3.4 million a year on Reddit alone

The social networking platform Reddit relies on volunteer moderators to prevent the site from being overrun by problematic content—including hate speech—and ensure that it remains appealing for users. Though uncompensated, this labor is highly valuable to the company: According to a pair of new studies led by Northwestern University computer scientists, it’s worth at minimum $3.4 million per year, which is equivalent to 2.8% of Reddit’s 2019 revenue.

Social media companies and tech platforms have gained an increasingly outsized role in shaping how information is consumed and shared as they’ve sought to expand their market shares, said lead study author Hanlin Li, a Ph.D. student at Northwestern. Putting a dollar value on the contributions of volunteer moderators for the first time could give ordinary users leverage with big tech when it comes to content moderation.

“Big tech companies have introduced some harmful practices, facilitating the spread of misinformation and failing to adequately support volunteers battling the proliferation of harmful content online,” Li said. “Broadly, our research is about thinking through how we can redistribute the decision-making power in the tech industry to include users. This is one step toward that.”

On Reddit, users are organized into communities called “subreddits” that share and discuss content related to topic areas of interest, from humor to news, art, video games and memes.

The subreddits are monitored by volunteers whose work is often framed as a labor of love, something they do because they care about and are interested in maintaining vibrant online communities, which Reddit hosts and makes available to users for free, in what might seem like a fair trade-off.

But the studies raise an important question: Are volunteer moderators and other users giving tech companies a “free ride” by providing their time, data and energy? After all, Reddit and Facebook—which also hosts groups organized around shared interests—are for-profit companies, with revenue models based on selling advertising to users.

Volunteer moderators, on the other hand, do not know how much collective work it takes to keep those online communities viable and safe, and how much that work is worth to the company. Would being armed with that knowledge make a difference in negotiations between users and tech companies?

“Putting a price tag on the labor that people—in this case, content moderators on Reddit—have subsidized is leverage those moderators could wield when asking platforms for better resources and tools to help them monitor more effectively,” Li said.

Li co-authored both papers with her adviser Brent Hecht, an associate professor in Northwestern’s School of Communication, and Stevie Chancellor, an assistant professor at the University of Minnesota.

Hecht leads the People, Space and Algorithms (PSA) Research Group at Northwestern. The group’s overall mission is to “identify and address societal problems that are created or exacerbated by advances in computer science.”

Li said a key part of the PSA Group’s work involves re-framing user contributions to sites like Google, Facebook, Twitter and Reddit as “work”—not passive participation in online space—because companies use data and time that users provide to generate profit: to train their algorithms, better target advertising, recruit new users and ultimately earn more revenue.

This reframing led them to coin the term “data labor subsidy” when placing a dollar value on the contributions of tech platform users. Both recent studies of Reddit were published as peer-reviewed papers at the International Conference on Web and Social Media, a top computational social science conference where Li will present them formally in June.

In the first study, “Measuring the Monetary Value of Online Volunteer Work,” the authors worked with moderators on dozens of subreddits and calculated the time it took the moderators to perform a variety of moderation actions such as approving posts, removing comments and banning users based on the timestamps between when different clicking actions occurred and calculating the average time spent performing each action.

Li notes that their research, particularly in the second paper, “All That’s Happening Behind the Scenes: A Data-Driven Analysis of Volunteer Moderator Labor on Reddit,” highlights that moderator activities aren’t exclusively related to routine actions like hiding comments. They also involve more nuanced curation of content, such as tagging to facilitate user searches.

The research team incorporated both publicly available data and data supplied by a subset of moderators whom they asked to participate (those asked were randomly chosen, although they do not constitute a perfectly random sample) in both studies.

In the first, they extrapolated from the time these moderators spent moderating daily to determine the approximate amount of collective time spent moderating by all active Reddit moderators on a given day (currently, there are about 21,500 such moderators). In the second study, they described the activities of those moderators in detail.

They found that collectively, the whole Reddit moderator population spends 466 hours per day performing moderating actions on the platform at a minimum. This is a lower bound estimate because the study could not account for every single action that might reasonably be construed as moderating. Using the median hourly U.S. rate for similar paid services gleaned from the freelancing platform UpWork ($20/hour), they calculated the $3.4 million per year figure.

For the time being, Reddit provides only limited tools—such as keyword filters—to moderators tasked with maintaining groups with thousands and sometimes millions of users, prompting moderators to feel overwhelmed and frequently ask the company for more tools, according to Li.

“Volunteers moderators are in such a disadvantaged position when it comes to asking for support,” she said. But, she added, the studies now provide evidence that volunteers have performed real work for the company that they can bring to the bargaining table.

“Our motivation is to introduce needed information into this kind of transaction to ensure that it’s a fair value exchange,” she said.