I’m writing a book in November, and it’s about a content moderator. Turns out that there’s not a lot competition, which is good and bad. Good because it’s easier to stand out, and bad because if you do the thing that is already done in one of the only two other novels taking this on then you’re a total hack.

And so I will now share with you the fruits of my homework.

Non-Fiction

Behind the Screen: Content Moderation in the Shadows of Social Media (Sarah T. Robertson, 2019)
Behind the Screen is the text on the subject of content moderation, an unflinching look at the realities of content moderation at home and abroad. Sarah T. Robertson has been working on this subject for ~15 years now, coined the term “Commercial Content Moderation,” and beyond this appears in many other documentaries/articles/podcasts to talk about the work.

There’s a lot to talk about! Here, she introduces us to different types of moderators, from Silicon Valley all the way to Manila. And she discusses the differences between the moderators and their expectations; I was shocked to learn that the largest international “call center” was not in fact India but the Philippines, because their culture has been more forcibly westernized. But also the threat of Indian replacement results in ever-worsening rates from the companies looking to skimp any and everywhere.

(Capitalism bad and all that.)

There’s a general acceptance among all of these works that Commercial Content Moderation is necessary. If we’re going to accept that anyone can upload text and images and video, people will upload heinous shit and someone will need to determine whether that heinous shit is acceptable. But it doesn’t need to happen like it happens. The people who work in Silicon Valley offices should be allowed to climb the rock wall with the “real” employees, and the people in Manila should frankly be the best-paid people in their buildings.

They’re forced to face the worst of humanity, and the disrespect that their employers show for them and their struggles is utterly infuriating. There are lasting effects, and Behind the Screen does an excellent job helping us see what those are.

The Cleaners (Hans Block and Moritz Riesewieck, 2018)
The Cleaners makes a great companion to Behind the Screen, as it is less concerned with the moderators (“cleaners”) themselves than the impact of their decisions and also of the people who make the decisions that the moderators are then expected to implement.

Moderators talk about their work means to them (why many see their jobs as genuinely important (I would agree)), but then we see them make choices and then dig a bit deeper. A moderator sees a painting of Donald Trump with a small penis and chooses to delete it; we cut to artist Illma Gore, who painted it, talking about her art and the choice to make it and the impact that its rejection by social platforms had.

Not every subject is as interesting — it’s weird that we spend any time with a right-wing activist even if he is depicted as the total fucking loser he is — but many are, and it’s a really effective way to help contextualize what is actually happening when someone does content moderation. What happens when someone’s political expression is censored.

But also what happens when violent rhetoric is not.

We see a Rohingya woman brutalized in Myanmar talk about her unfathomably awful experiences, interspersed with people talking about what Facebook has and has not chosen to do. Their algorithms, by prioritizing engagement and outrage, ensured that hate would spread and in doing so they facilitated a genocide.

Policies are not created in a vacuum, and moderation is not handled in a vacuum. These decision have real effects on real people. And it’s critical to never lose sight of that fact.

Fiction

American Sweatshop (Uta Briesewitz, 2025)
So this novel I’m writing will actually be the second time I’ve taken on content moderation as a subject; as discussed in my my review of the new Faces of Death, I had a short script about content moderation that was really about corporate exploitation and dehumanization. And when I first heard about American Sweatshop, I thought, “Ah, someone’s done it.”

Nope. (ConSan holds up super well.)

American Sweatshop actually has quite a lot in common with Faces of Death: its protagonist, Daisy, is a content moderator who sees a particularly horrible video that might be fake but also seems super real and decides that she needs to do something about it. Like, both of them even have scenes where the women post videos to Reddit asking for help identifying the source.

(Maybe I should update the “Twin Films” Wikipedia page… can I cite this newsletter as justification for updating the Twin Films Wikipedia page? These are totally Twin Films.)

The “thriller” elements are really where American Sweatshop falters, because it’s frankly just not that interesting. She can stake out a man’s house and then put violent pornography in his face, and it just doesn’t feel like anything. She can pretend like she wants to be in violent pornography in order to get access to the place where the video was shot and then pepper spray the dude who seemingly shot it. She can even beat a bartender with his belt when he tries to hit on her.

These sort of heightened moments clash with the more grounded emotional stuff that we’re supposed to be feeling. So I was always a little bit… confused?

The latter event almost works, when she is faced with a video of herself doing it in a moderation queue and has to reckon with what’s become of her, but the “lesson” you assume she learns seems to be entirely unlearned for the genuinely stupid finale that instead of actually paying off this whole plot ends with her literally turning to the camera like “Man, once this movie’s over I’ma do something so cool but you won’t get to see it sucks to be you dude.”

And that sucks!

Moderation (Elaine Castillo, 2025)
When I started looking for “Novels about content moderators,” this was seemingly it: the only thing I saw on Reddit threads, Goodreads lists, even a chatbot’s output. When I saw that it was about “The world’s best content moderator,” I rolled my eyes pretty hard but ya know what, if Girlie can work a decade on CSAM without losing their humanity, she probably does deserve some kind of commendation.

I like Castillo’s writing. I like the perspective she brings to the story as a Filipino writer who was probably introduced to this work by other people in her community doing it, since (as noted previously), the largest subset of “Commercial Content Moderators” are in the Philippines. But where I think Castillo is very good at depicting, like, today, I think she’s less good at imagining tomorrow. Because it turns out that Moderation is a near-future story, but not in ways that I really recognize.

I thought that the book came out in 2022, so I wasn’t surprised that “Twitter” is still referred to as such, or that there is no mention literally at all of chatbots… but there are actual self-driving Teslas and a PlayStation 7 (which would put us solidly in the mid-2030s). I had assumed that her employer, Reeden, was a Facebook stand-in but then she explicitly laughs off Meta’s attempt at the Metaverse, when it turns out that Reeden is going hardcore into an actually flawless full-body virtual environment that Girlie is going to be helping to moderate… an interesting idea with frankly boring execution, because by the time we actually get into this (impossible) digital world, the book has shifted over to mostly being about Girlie falling for her hot boss, William. And while I think the romance works fine in a vacuum, it’s so much less interesting than basically every other part of the story, and the big romantic climax (lol) is utter nonsense.

Still, I mostly enjoyed it. Not so much the next one:

We Had to Remove This Post (Hanna Bervoets via Emma Rault, 2021)
Turned out that all those searches had missed one. We Had to Remove This Post is a svelte little thing (the CloudLibrary eBook is ~70 pages) that people in general don’t like very much, and I get it!

This one is much more focused on the act of moderation and the effects it has on the people who perform it: the end even includes a list of sources that looks quite similar to the one I’ve compiled (plus an untranslated Danish book from 2019, The Back End of Facebook: 8 Months in Hell… good name!). It presents as a letter to a lawyer who is turning content moderator experiences into a class action suit against her former employer, as she explains why she does not want to take part in said suit. He will understand by the end why she does not want to be involved.

This would be a fine conceit if it made the story feel more grounded, but it kinda does the opposite. And maybe that's on purpose: we’re setting up a story to a specific audience and so we know we can't fully trust it, but it really feels like a way to get up the scale of the problem in the real world in a way that has nothing to do with the story.

Which is something that I have been thinking about a lot. The competing urges to “teach your audience about this horrible job that exists” and also “tell a compelling story.” Moderation was ultimately not that interested in the former, while We Had to Remove This Post didn't focus enough on the latter.

And as a result, the revelation in its final pages that the dissolution of Kayleigh’s relationship was not because Sigrid is mean but because she is afraid — just doesn't land. It's a rugpull that required more setup and payoff… and the way it's presented is also confusing because again this is a letter to a lawyer?

I dunno. It’s not like there’s nothing of interest here, and the short length makes it less frustrating than it would be if it were 300+ pages like Moderation, but unfortunately rather than reading the book itself I’d recommend you read its inspirations (aka the things in the Non-Fictions section).