AI-Generated Book Scandals and the Question of Quality

From a Substack called 72 Degrees North: Why Literary Agents and Publishers Can’t Detect AI-Generated Writing, but Readers Can

My personal hypothesis has been: At least when auction fever hits, acquisitions editors do not read the books they acquire. They read a couple of paragraphs and make the call without reading beyond that. Then I asked: But what about agents? Because surely agents read the thing? The linked blog post goes much, much further and asserts that no, they don’t, generally speaking.

Who wrote this essay? Whose Substack is this? This is Michael Church, who evidently, says Google, “[F]requently critiques [the traditional publishing industry’s]structural flaws, the commercialization of modern novels, and the “zero-sum mentality” that he argues exploits or shuts out independent authors.” Thus we see that this essay is likely to be critical in its take on the phenomenon of book deals that start out great and then abruptly go south due to accusations of AI use. So it is; very critical indeed.

Literary agents and acquisitions editors, when they read at all, read differently than you and me. They haven’t read for pleasure for decades, and it’s probable they no longer can, and it’s not their fault. … agents and editors associate books and reading with the office, and they read in one of two modes:

  • They see dollar signs. The book’s commercial hooks are so brazen, the title sells itself. The author did his MFA under a professor with friends on every award jury in the nation. The topic is a reliable culture-war ragebait issue, like college admissions or casual sex.
  • They don’t see dollar signs. There’s no one-sentence pitch for the book. The protagonist has the same name as the publicity director’s more-successful and already-married younger sister, so the book will be punished in its promotion budget. The author’s follower count is under 20,000.

When these people see dollar signs, they read (skim) to confirm a decision to publish, and hope to hell nothing surprises them. When they don’t see dollar signs, they get out as efficiently as they can: close the book, send a form letter, set a filter to block that email address in perpetuity.

What do you think? I think this is (a) harsh, but (b) fairly plausible, but (c) probably overstated and overly harsh, so I wind up going in a circle and not putting a whole lot of credence in this opinion. But I think it’s not wholly wrong, either, because the thing is, SOMETHING caused the agent in the Jerry Felade case to take on that book, take it to auction, and present it as though it was a human-written book, when Jane Friedman reported that other agents thought it was clearly and obviously generated from the first paragraphs. And I don’t know which is true, but my point is, opinions about the opening paragraphs looking generated ran all the way from “Yes, obviously” to “no, I trust the author.”

So … how DO experienced readers, jaded or otherwise, interested in quality of the prose or otherwise, tell the difference, or else fail to tell the difference?

Here’s a study that indicates that experience reading fiction doesn’t lead to better ability to identify generated fiction, BUT experience looking at GENERATED prose DOES improve this ability. This strikes me as highly reasonable.

Same thing here, basically: People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text.

It’s your experience looking at prose you KNOW is generated that trains your eye to recognize generated prose. That’s just like all my posts about, eg, where does this story stop being real and other such posts. Plus asking ChatGPT to narrow down options when I was buying a car and going absolutely bonkers from all the stupid new technology in current models. …

Digression: the single thing I like best about my new car … this may seem trivial … and I do mean other than not sounding like it is about to explode … but the backup camera is humongously better than the one in my previous car, and the rearview mirrors are also better, and I have to say, I do appreciate this. I literally could not tell where the back of my previous car exactly was. Short people of the world, you are all nodding, I bet, because neither the backup camera nor the rearview mirrors necessarily shows you the ground directly behind your car if you are short (or at least, my 2014 Chevy Equinox did not). My new car draws a little outline and lays it over the ground behind my car — this is shown in the touchscreen display — so I can tell EXACTLY where my car is relative to everything behind me, including low curbs that were completely invisible with my last car. Great job, major gold star to whoever came up with this. Excellent use of touchscreen technology, and I do not mind music also being embedded in the touchscreen. End of digression.

… And whatever else I’ve been doing lately. Oh, investigating AI detectors using generated essays, humanized essays, all sorts of things. I looked at a lot of generated pose while doing that. Also, I’ve been trying to come up with prompts that allow students to use AI chatbots in legitimate ways, given that I know for certain some of my students are using those chatbots and do not expect they are going to stop. …

Second digression: “Chat, look at this essay. Turn this essay into a list of individual sentences and identify and explain all usage errors for each sentence, one at a time. Do not rephrase or suggest ways to rewrite any of the sentences. Your sole job is to identify and explain errors.” I came up with this prompt to try to help a student with DIRE usage problems in EVERY sentence, except the sentences he generated. I went through four sentences saying, essentially, “Let’s recite the normal capitalization rules. Where does this sentence end? Then you need a period at that spot. But don’t put a space BEFORE the period, EVER. Where is the verb. Okay, what IS a verb?” On and on, for each and every sentence, except the ones that were generated, and this student could not tell how glaringly obvious those generated sentences were compared to his own sentences, but it was like setting a dish containing a raw egg next to a platter holding a chocolate cake and saying, “Which of these looks like a cake?” No teacher on the planet could possibly have failed to notice the difference or would have believed the student wrote both types of sentences himself. So, I developed that prompt so that an infinitely patient chatbot could go through every single sentence for this student, without rephrasing anything, because that should help prevent this student flunking every essay for either usage errors or generated prose, whichever. And he was definitely, for sure, using a chatbot, so it did not seem likely he was going to stop, so finding a prompt that would allow some sort of useful and legitimate use seemed like a better plan than pretending he was going to just give up using a chatbot and somehow learn how to do basic third-grade capitalization on his own. End of digression.

What was my point? Oh, right: I have by now looked at a lot of generated prose, mostly but not exclusively ChatGPT prose, and I’ve looked at it very carefully. I therefore think that failure to look at generated prose may cause a severe problem for agents and editors who hate hate hate AI chatbots and refuse to look at generated prose. This could explain why some agents and editors can’t tell when they are looking at a screamingly obvious piece of generated prose, even when it is exactly like looking at a raw egg next to a cake.

AND, this hypothesis also suggests that the linked Substack post is wrong: Not all readers can recognize AI-generated prose either. A lot probably cannot do so. The readers who can recognize generated prose are, most reliably, the readers who have been looking at generated prose, and I bet it does not matter a lot whether those readers are agents, editors, or other people. So this is my tentative conclusion. And I hereby suggest that agents and editors who have never looked at a lot of generated prose had better (a) pass the manuscript to someone who has, or else (b) use an aggressive detector as part of their evaluation and acquisitions process, or at the very least (c) stop being all, “Well, the author is personable, so I’m sure it’s fine,” when making acquisitions decisions.

My current hypothesis is two -part:

1) Amazingly enough, some agents are just way too trusting of authors who say they aren’t using AI “for anything but proofreading, cross my heart.” Those agents are putting themselves at risk by taking on clients who generated a substantial portion of the words in their books.

2) Apparently some or many acquisitions editors are also just assuming agents are weeding out generated novels, but this is an unsafe assumption, so all or nearly all acquisitions editors are surely going to run everything through an aggressive detector before they go to an exciting auction to bid on something they haven’t read. It doesn’t matter whether detectors are wholly reliable. It does matter that detectors are sitting there, one tool to use to help avoid extremely embarrassing scandals, so editors are going to use them. Any who don’t will be at increased risk of being nailed in the next few scandals, so then they probably will.

And this theory does not mean I think the publishers involved in these scandals have behaved well, because I think they have behaved shamefully. But I also think they had better start taking more care proactively and then they can stand behind their authors with (I sincerely hope) dramatically more gumption in the face of accusations.

But on top of that, the original linked post that got me started here asserts that:

Traditionally published novels are not written so much as they are engineered. Write the book you care about, and no agent will touch it. Engineer it, and it’s a possibility. Choose your commercial hooks first. Make sure they’re still in vogue. Then come up with a plot. Add characters. Write words—they don’t have to be great, or even good; they just need to be there. Slush tells need to be avoided not because readers care about them, but because their existence signifies that a writer has not fully absorbed the social protocols and message language on which publishing runs. … This process results in books—of human or machine authorship—that invariably do well in traditional publishing, if their authors can get them read, but that readers find stilted and bland. A generation decides, based on what traditional publishing is willing to provide, that novels just are not for them.

And I think this is overstated and overly harsh, meaning not exactly true. But I also think this isn’t entirely, one hundred percent false either. When I’ve looked at Ray Rhamey’s first-page columns at Writer Unboxed, such as this one, I have been STUNNED at how bland and boring and unappealing the first pages of bestselling novels often are. I mean, not all of them. But a lot of them. This is all making me think of Erik Hoel’s post How the MFA swallowed literature, which includes this:

Workshop-trained writers are often, not always, but often, intrinsically defensive. This single fact explains almost all defining features of contemporary literature. What you’re looking at on the shelf are not so much books as battlements. … Consider the minimalism of many current novels, their brevity—all to shrink the attack surface. Oh, the prose is always well-polished, with the occasional pleasing turn of phrase, but never distinctive, never flowery nor reaching. 

I think this is a contrary take on why modern novels might be same-y, bland, undistinguished, generic, whatever term one cares to apply. If you perceive modern fiction this way, then the author of the first post might say, “Look! The gatekeepers are using secret handshakes to decide what to publish; they don’t actually read the novels they bring out, they don’t care if the novels are any good!” And Hoel might say, “Current authors are formed by MFA programs that teach them to stay bland and safe lest they ever write anything that might be attacked by anybody.”

And for all I know, both might be at least partly right. I think it seems quite plausible that the contemporary publishing system selects for exactly those qualities that make generated prose seem acceptable to agents and editors. That’s superficially clever sentences that don’t produce any particular depth or coherence — which is possibly what you also get when you write a novel while trying to avoid risk, distinctiveness, and vulnerability to criticism, as Erik Hoel is arguing has become typical for MFA writers. Suppose agents and editors have become accustomed to reading fiction that is superficially clever and polished, but also safe, predictable, and conventional. Suppose writers — particularly authors in the Literary genre — have been trained to produce fiction that fits those expectations. Then the problem isn’t just that agents and editors can’t recognize generated prose, it’s also that they’ve been trained to think that generated prose looks publishable and that authors have been trained to write prose that overlaps with generated prose.

And … I realize I keep going farther afield, but all this reminds me of the Booker Prize judge who complained fairly bitterly about the terrible quality of most of the nominees in 2025.

Five judges spent the past year dutifully reading 153 works of fiction to select the most “Booker book,” per the Booker Prize Foundation’s livestreamed announcement of the shortlist in September. We know this number because the judges invoked it, repeatedly. (Their workload was commensurate with previous years, save for the apple polishers of 2022 and their 169-book total.) Was it such a burden? Were the books really so terrible this year? Chair Roddy Doyle thought so. He lamented that only thirty-one of the submissions “were worth discussing.” 

And I don’t know what the judges exactly thought was not great about the other nominees, but “safe, bland, defensive, superficially clever but boring” is one possibility. Another is “weird metaphors and strangely clever sentences that don’t actually make sense in context,” which means “looks generated once you look more carefully.” Or maybe they meant something else entirely. Boring, awkward prose, who knows. I’m just saying that all three of these wildly different perspectives are saying, fundamentally, that modern fiction, especially modern Literary fiction, is disappointing.

And I just want to note that one way to acquire a novel that does not look generated is to acquire novels that DO NOT LOOK GENERATED, meaning not generic, not bland, not same-y, not defensive, not superficially clever but actually engineered to tick off boxes during the acquisitions process by agents and editors who neither read nor care about books … whatever is true or sort of true or partly true about these novels that are at the center of various scandals. Because a publishing culture that values distinctive, original, ambitious fiction might be less hospitable to at least certain kinds of generic AI-generated fiction, AND less hospitable to less-great fiction produced by defensive workshop attendees, AND less hospitable to less great novels across the board. Meanwhile, as a side note to the overall project of encouraging better authors to write better novels, probably agents and editors should take the time to look at more generated prose, because raw eggs and cake are not hard to tell apart at a glance, once you’ve looked at a reasonable number of both eggs and cakes.

Please Feel Free to Share:

Facebooktwitterredditpinterestlinkedintumblrmail

10 thoughts on “AI-Generated Book Scandals and the Question of Quality”

  1. I was wondering if this is the same Michael Church I was thinking of. And, it is. He’s a serial complainer, and a perfect example of the Dunning-Kruger effect.

    Got fired from Google.

    Was banned from editing Wikipedia:
    https://en.wikipedia.org/wiki/User:Mike_Church~enwiki and
    https://en.wikipedia.org/wiki/User:Isomorphic/Minions_of_the_Church

    Got banned from Quora: https://www.quora.com/topic/Michael-O-Church-Quora-user

    And now I guess he’s pontificating on Substack. About things he, as usual, knows less about than he claims.

  2. Quite a history, Amit. And really, not altogether surprising. I don’t know what it takes to get banned from Wikipedia, but I don’t think it’s that easy to get banned from Quora — except by spamming.

  3. What I do know is that except for a few writers I buy on sight, I haven’t picked up and finished a recently written novel in years. Whatever the acquisitions people are choosing new writers on, isn’t interesting to me. I have better luck with much older books.

    I do like “Flog a Pro”, which occasionally gets me to try a new to me writer (but see above). This month’s and last month’s don’t, and I think both books are by the same writer, McFadden. The comments can have some insights on what works and what doesn’t which also interests me. And every once in a while someone there points out what everyone else responding misses. I recall about a year ago, the extract was from a fantasy, and no one liked it. One commenter analyzed how it was signaling exactly what the subgenre was, so people who liked it could find it. I still wasn’t interested in the book, but that analysis of what it was doing interested me.

  4. McFadden’s popcorn thrillers look superficially good on the first page, but I never have any interest in actually turning the page. They’re not really my type of thing, and the smoothness of the prose does look superficial to me.

  5. It’s true. The more you look at generated prose, the better you are at spotting it. I’ve also seen my share of people, often in the creative fields, who hate hate hate AI chatbots and refuse to look at generated prose–intentionally, anyway. They’re often the first to say there’s no way to distinguish AI writing and that it’s inevitable it will only get better.

    Having subjected to myself to an overdose of AI-generated prose, I’m inclined to disagree, especially when it’s left unedited (or poorly). You won’t see me going up to a florist and telling them there’s no aesthetically pleasing way to arrange flowers, despite never having ever done it myself.

    Not the best analogy? Probably not. There’s nothing pretty about AI prose, and you’re not going to cheer anyone up if you present them with Elara Voss’s Aethelgard Adventure. Maybe one of these days, I will figure out a better comparison. I know it isn’t fun, but if we want to say we stand against AI-generated writing, the first step is knowing how it reads. Sun Tzu knows–if he was against AI, I’m sure he would’ve told us all to read lots of AI prose, much to the chagrin of many.

    One resource I’ve found useful for studying AI fiction is the data from the StoryScope study. I went and converted the data files into text, and pick a piece at random to read once in a while as a reminder of where we are in AI’s writing quality in 2026. How anyone can find it tolerable is beyond me, but I was also once unfamiliar with the cadence and its “shape”, and maybe life would be better if I still was. Happy to share that if you’re curious.

    This is all to say I agree with everything you’ve said. I think if we all collectively increased our standards, AI-generated novels would have a very hard time skirting by as it does now. Some of us (not me) are more afraid of what could be than what is, and miss that trying to avoid AI writing is like trying to park your car with no rearview camera and poor spatial reasoning, and hoping–just hoping, that you don’t back into a rate increase in your car insurance.

  6. M. J, I would be very interested indeed! I went and looked at Story Scope, which evidently is looking at story structure rather than style and assessing AI generated stories according to structural characteristics rather than stylistic characteristics, and that’s fascinating.

    Maybe … maybe it’s like cheap grocery store chocolate chip cookies, against fabulous chocolate chip cookies. There are lots of ways for cookies to be fabulous, but all fabulous cookies are immediately distinguishable from crappy grocery store cookies, and it’s not at all difficult to tell which are which. And then one can add, “But many people do buy awful cheap grocery store cookies, so they either find them acceptable or don’t know there’s any difference or actually like them.” And in the same way, the least discriminating readers may find generated stories acceptable — especially if they have never encountered good writing. Could one extend the analogy and say, “If people refuse ever to try cheap grocery store cookies, they will never, ever believe how obvious they are. When someone brings those cookies to a potluck, they may try one and think it’s bad, but they won’t realize the badness is because it is a cheap grocery store cookie and the badness is extremely characteristic of that whole category.”

  7. Here are the StoryScope stories:https://drive.google.com/file/d/1r3Iw0Mr8gj_QH3rQNxWLwD8mQTuH_DBh/view?usp=sharing

    It’s broken down into ten .doc files because it slows down too much to load as one. The formatting is very poor because the file it was stored in stripped it all, but I tried to make it easy to tell where one starts and ends. I still think it’s an excellent resource/reference for anyone curious about what AI can produce at what could be considered an above average quality from its usual. If you ever end up doing an analysis of it, I’d be interested in reading it. :) If there is an issue with the file, I could reupload it but as individual docs instead of a zip.

    I like the cookie analogy (and even food in general would work here) a lot. I’ll be sure to use it–we can probably consider writers the bakers, where knowing your good and bad baked goods are more important.

    It makes me think of the sausage analogy I’ve seen here and there, where folks who use AI argue that a sausage is a sausage–why would you care how it’s made? Which I found one of the sillier arguments. Some of us care whether the ingredients are good quality, or how the animals were treated, or if the casing has traces of asbestos in it, or if the sausage handler wore no gloves and has no hygiene standards. Some cookie eaters may consume it mindlessly, some may avoid any with aspartame or textured soy protein fillers. And many of us want nothing to do with the ones manufactured without any type of flour while masquerading as being made by a passionate baker.

  8. Thank you very much, M.J! I’ve got the files, I’ll look at them, I appreciate that you’re sharing them because I bet it was a certain amount of trouble to get those.

    I think cookies are a better choice here than sausages, because the quality difference is both stark and familiar. I won’t say a sausage is a sausage, but I WILL say a cookie is most definitely not a cookie, and almost everyone is aware of that. To extend the metaphor (beyond plausible reason), AI prose is like cheap almond flavoring, which is the dominant flavor in a whole lot of cheap grocery store baked goods. The almond flavoring is like a sledgehammer that smashes into your taste buds: THIS IS A GROCERY STORE COOKIE AND YOU MUST AGREE BECAUSE IT TASTES OF ALMOND FLAVORING ABOVE ALL.

    I think people have been trained to associate ALMOND FLAVORING !!! as “cookie flavor” and accept that they are eating a cookie because it tastes of almond flavoring. I wonder, in 10 years, 20 years, whether (some) readers will be so accustomed to AI-generated prose that they just think of it as “prose,” while other (more discriminating) readers shake their heads and say, “But can’t you tell how bad it is? Isn’t it obvious? How can this not be obvious?”

  9. That makes many false assumptions though. Aside from the famous example of Watson for a character who does not start out with an assumption and has no reason to question it, it leaves no room for the character who starts out with a certain belief about himself or the world, and ends the story with that belief not rejected but refined. For examples see Tristan from FORTRESS (and other CJC characters). Edward Elric from Fullmetal Alchemist.

  10. Glad to be able to share it, Rachel! It was indeed a bit of a hassle, but nothing terrible, and now I can offer others easy access in the name of education or curiosity.

    There are certainly a lot of cookies these days with cheap almond flavoring. Sometimes, there are thousands of people talking about how delicious they are on websites like CookieEats, while others are throwing their hands up, going “Do you not taste how disgusting this is! These are not from the bakery, they’re from the grocery store, but taste like a gas station’s!” And some will say, “I did think it tasted off for some reason!”, while others will go, “These cookies are perfectly fine, there’s no almond flavoring.”

    I think AI prose is a bit of a separation process from the wheat from the chaff, from what I’ve seen. With the mindless consumption culture corporations have been pushing aggressively, some will continue to take what is served. Others will grow increasingly sick of it. I’m curious what will happen when the AI bubble pops–not looking forward to the massive fallout–and if that will result in a reduction of the endless slop we’re seeing when companies can’t afford to subsidize it anymore.

    That’s also not taking into consideration all the watermarking being added to AI content lately. That will help the less discerning who want to eat healthier cookies, but don’t know how to identify them. SynthID for example, will put a lot of people using ChatGPT or Gemini to generate images in a bind (putting aside how obvious its outputs are) when anyone can plug a book cover or marketing image into the watermark detector and prove beyond a doubt AI had its motor oily roboprints all over it.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top