Which is real?

One of these excerpts shows an actual excerpt from a real novel. The other is a fake excerpt generated by Chat GPT.

Which is real?

Excerpt 1:

In the ancient realm of Hokkaro, Empress Yui, revered as The Phoenix Empress, Daughter of Heaven, the Light of Hokkaro, and the Celestial Flame, ascends the grand throne with a regal grace that commands the attention of all who gaze upon her. The courtiers, draped in silken robes, bow in unison as the empress moves through the ornate corridors of the palace.

The grandeur of the Celestial Palace mirrors the magnificence of Yui herself, whose presence radiates an ethereal warmth. Her ebony hair, adorned with golden phoenix feathers, cascades down her back like a river of darkness, contrasting with the resplendent golden robes that drape elegantly over her shoulders. The delicate diadem on her brow glows softly, a testament to the divine lineage she carries.

Empress Yui’s eyes, pools of wisdom and kindness, survey the realm with a discerning gaze. Her subjects whisper of her innate connection to the heavens, a belief reinforced by the legend of her birth—an event accompanied by the flight of a fiery phoenix across the celestial canvas.

In the court, where politics and alliances are as intricate as the patterns woven into Hokkaro’s legendary tapestries, Empress Yui navigates with diplomacy as her compass. Her decisions are tempered with compassion, for she sees the struggles and dreams of her people as intimately connected to the fate of Hokkaro itself.

Excerpt 2:

Empress Yui wrestles with her broken zither. She’d rather deal with the tiger again. Or the demons. Or her uncle. Anything short of going north, anything short of war. But a snapped string? One cannot reason with a snapped string, nor can one chop it in half and be rid of the problem.

When she stops to think on it—chopping things in half is part of why she’s alone with the stupid instrument to begin with. Did she not say she’d stop dueling? What was she thinking, accepting Rayama-tun’s challenge? He is only a boy.

And now he will be the boy who dueled One-Stroke Shizuka, the boy whose sword she cut in half before he managed to draw it. That story will haunt him for the rest of his life.

The Phoenix Empress, Daughter of Heaven, the Light of Hokkaro, Celestial Flame—no, she is alone, let her wear her own name—O Shizuka pinches her scarred nose. When was the last day she behaved the way an Empress should?


Let’s try that again. Which is real?

Excerpt 1:

The wind roared in my ears as I leaned forward on the sleek sportbike, my fingers wrapped tightly around the handles. The asphalt blurred beneath me, and the adrenaline surged through my veins. I was JD Marshall, and racing against Viv Nixon was a challenge I couldn’t resist.

Viv, with her fiery red hair and a reputation for tearing up the track, was the only rider who could match my intensity. The rivalry between us had grown with each race, a friendly competition that pushed us to our limits. Today was no different – the sun hung low in the sky, casting long shadows across the winding road as we revved our engines, waiting for the signal to start.

The air crackled with anticipation as the flag dropped, and we shot forward like arrows released from a bow. The sportbikes beneath us roared in unison, engines blending into a symphony of power. The road twisted and turned, demanding precision and skill. Viv pulled ahead, her bike hugging the curves with an effortless grace that always left me in awe.

But I wasn’t one to back down easily. I leaned into the next turn, feeling the G-force pull at me. The thrill of the race was intoxicating, a dance between man and machine, where split-second decisions could make or break you. Viv and I weaved through traffic, each of us seeking the perfect line, the optimal speed.

Excerpt 2:

My red sportbike breached a thick wall of opaque heat radiating off the pavement. To my left, the pristine blue waves of the Pacific Ocean. To my right, towering walls of gray-brown rock and boulder. Behind me, closer than ever before: Vivien Nixon, a yellow projectile hurtling forward at almost impossible velocity.

We’d raced each other in these canyons hundreds, if not thousands, of times. I had every curve, every line, every crevice memorized—and used them to my advantage. Even the seemingly insignificant angles of shadow and light were weapons at my disposal.

Our machines equal, only strategy and technique separated us—and perhaps the intangible will to win.

Entering a straightaway, Viv made her move. In my rearview I could see her foot stabbing downward, downshifting into third. The sound of five thousand RPMs rattled inside my helmet as I watched her yellow streak blast by me.

Instinctually, I matched her technique: Downshift. Accelerate. Overtake. She was not going to beat me.

In an instant, the road narrowed, and we were even. A blur of yellow and red intertwined.

And then came the curve. Our two bikes, cornering at breakneck speed, inches apart along the cliff’s edge.


One More Pair: Which is real?

Excerpt 1:

The pulsating rhythm of the music throbbed through the crowded room as I navigated through a sea of bodies. The air was charged with excitement, and the dance floor beckoned with its promise of fleeting connections. Amidst the swirl of laughter and movement, he appeared—a boy with hopeful eyes and an outstretched hand.

“Hey, would you like to dance?” he asked, a shy smile playing on his lips. The sincerity in his voice was unmistakable, and for a moment, I hesitated. But as I met his gaze, I knew I had to be honest.

“Thank you, but I’m going to sit this one out,” I replied, my voice gentle yet firm. His hopeful expression faltered, and I felt a pang of guilt. “I appreciate the offer, though.”

He nodded, a mix of disappointment and understanding in his eyes. “Okay, maybe next time.”

I offered a small smile, hoping to convey that it wasn’t a rejection of him as a person, just a choice in that moment. “Definitely, maybe next time.”

As I walked away, I couldn’t shake the feeling of having disappointed someone. The music continued to pulse, the dance floor alive with the ebb and flow of people lost in the rhythm. I found a quiet corner, watching the kaleidoscope of emotions on the dance floor, grateful for the courage it took to be honest about what I wanted.

Excerpt 2:

I didn’t even notice the young groomsman from earlier approaching until he was right beside me.

“Want to dance?”

I turned, taking him in: the short hair, those blue eyes, the black bowtie I’d picked up earlier loose, but not undone, around his neck. “No, thanks.”

He looked surprised and—I realized, horrified—embarrassed. “Oh. Okay.”

“I’m working,” I said quickly, stepping over his last syllables. Now we were both blushing. “For the wedding planner. So I can’t—”

“Oh, right.” His face relaxed. “I didn’t realize—”

“I know, it’s fine.” I looked at the floor, tucking a piece of hair behind my ear. “Thank you anyway.”

He smiled then, and there was something about the way it changed his face, taking it from cute to outright charming, that suddenly made me wish I could say yes. To a boy, and a dance, and to also having that chance, one night, to be away from everything. We stood there a minute, until the bridesmaids nearby opened up their circle, whooping, and pulled him in. I walked over to the table I’d seen earlier, collecting the glasses and putting them on a nearby tray. Underneath one of the chairs was another penny card, face down, and I picked it up, rubbing my finger over the coin. When I looked back at the dance floor, the boy and the bridesmaids were gone.


Oh, actually, let’s do one more!

First Excerpt:

A sudden tremor rumbled through the stone floor, sending vibrations up through the spines of the dusty tomes. The library fell into an uneasy silence as students and scholars alike exchanged puzzled glances. I felt an instinctive pull, a magical resonance that drew me toward the towering bookshelves.

Turning the corner, my eyes widened at the scene that unfolded before me. A vast window, long obscured by ancient tapestries, now revealed the sprawling landscape beyond the library. And there, against the backdrop of the distant mountains, soared a creature of mythical proportions—a dragon.

Its scales shimmered with an otherworldly iridescence, catching the sunlight and reflecting a kaleidoscope of colors. Wings that spanned the horizon beat with a majestic rhythm, casting shadows over the land. I stood frozen, spellbound by the sheer magnificence of the creature that had graced our world.

Gasps echoed through the library as other students joined me at the window, their eyes widening in awe and trepidation. The dragon circled in the sky, its movements graceful and deliberate. It seemed to sense our collective gaze, and for a moment, our eyes locked.

In that instant, a surge of ancient magic passed between us, a silent understanding that transcended words. The dragon wasn’t a mere creature; it was a guardian of the arcane, a living embodiment of the enchantments that wove through the fabric of our world.

Second Excerpt:

The crisp air of the mountainous realm of Draconia sent shivers down my spine as I ascended the stone steps leading to the fabled Dragon’s Peak, on my excursion to observe and document the legendary dragons said to inhabit the high reaches of these peaks.

As I reached the summit, the panoramic view took my breath away. The mountain range stretched far and wide, and the azure sky above seemed to touch the snow-capped peaks. A sense of awe and trepidation filled me as I realized the gravity of my quest.

Suddenly, the air stirred with a powerful presence. I turned, and there, against the canvas of the cerulean sky, a dragon unfolded its colossal wings. Its scales gleamed like burnished gold, and eyes that held the wisdom of ages met mine. My heart raced, a mix of fear and fascination coursing through me.

The dragon descended with an elegance that defied its massive form, landing before me with a ground-shaking thud. I could feel the warmth radiating from its scaled body, a stark contrast to the crisp mountain air.

“Lyra of the Arcane Academy,” a resonant voice echoed in my mind, as the dragon inclined its head. “Welcome, seeker of knowledge.”



Will the Real Book Please Stand Up



In the first pair, Excerpt 2 was from The Tiger’s Daughter by K. Arsenault Rivera. The first excerpt was generated by asking for a third person present tense story about Empress Yui, The Phoenix Empress, Daughter of Heaven, the Light of Hokkaro, Celestial Flame. I felt I had to tell ChatGPT to match person and tense in order to be fair. That didn’t make a lot of difference in this case. It’s screamingly obvious which is fake, isn’t it?

In the second pair, Excerpt 2 was the beginning of Devastation Class, a YA space opera by  Glen Zipper and Elaine Mongeon. Excerpt one was generated by ChatGPT in answer to the prompt “first person story about JD Marshall, and Viv Nixon racing sportbikes.” This time, I actually don’t think it’s nearly as obvious which one is fake. It is possibly relevant that I don’t think the excerpt from the real book is especially interesting or particularly well-written. However, you all picked the first one as fake, right? Do you know why, or was that a general feeling?

In the third pair, Excerpt 2 was still the real one. This time it was Once and For All, a YA contemporary romance by Sarah Dessen. I think it was pretty obvious that the first one was fake. Yes? No?

In the final pair, they were both fake. I realize that’s not exactly fair, but did anybody guess that? I think ChatGPT was doing far better with first person than third person. It seems to want to write story summary rather than story if permitted to write in third person. That’s much less true for first person.

Suppose you were reading slush at a publishing house. Would you have immediately pegged the fakes as fake? All of them? Would you have immediately rejected them as pretty terrible, fake or not? I think all of the fake ones are using a lot of clichés — but so are both real YA excerpts. Not as much, but there are definitely phrases I’d peg as clichéd. Does that mean I’d reject them immediately as fake? Or just as not very good? Or would I turn the page?

I would not turn the page for either of the real YA novels, probably.

I think that The Tiger’s Daughter stands way, way out as genuine. If I’d been handed all eight excerpts and told only one was real, I’d have picked this one. I would, however, have been mildly surprised if someone told me the excerpt from Devastation Class was fake. What makes it look real? Downshift. Accelerate. Overtake. This set of three single-word sentences does not, to me, look like something ChatGPT would do.

Did you notice that ALL the fakes have nothing but complete sentences? That’s not true of ANY of the real novels. I’m not particularly impressed by Once and For All, but it’s got much more interesting sentence structure than the fake. I wonder whether a detector could sort out real from fake based on nothing but this one characteristic, at least for fiction?

ChatGPT is also pretty bad at creating a sense of place. Devastation Class shows this pretty well. The setting of the real book is much more interesting and also the author doesn’t seem to forget that the race is happening in a setting. The fake doesn’t really have a setting at all. There’s a sun in the sky and a racetrack, but that’s all we see of the setting in this excerpt.

What does this mean?

Well, I don’t know what it means. BUT, I think if AI-generated books start to replace some real books, it’s going to be pretty bad books that bite the dust first. Any novel that creates a sense of place and has anything remotely interesting about the prose style is going to remain way better than text-generated fake books until the generators learn to quit writing in complete sentences all the time, and also start to remember that the story is happening somewhere. All this is besides having a coherent plot and compelling characters, which is also not going to happen for a good while, probably.

I will leave you with an excerpt that is wildly unlike anything a text generator is going to produce probably ever:

I was minding my own business in beautiful downside Wanderweb, having just managed to mislay my cargo for the right price. My nighttime man had talked me into bootlegging again, and damsilly stuff it was, too – either maintenance manuals or philosophy texts. I never did figure out which, even with sixty hours time in Firecat between Coldwater and Wanderweb to stare at them and Paladin to read them to me.

So I was making my way around Wanderweb, free, female and a damn sight over the age of reason, when I saw this greenie right in front of me in the street.

He was definitely a toff, and no stardancer – you never saw such clothes outside of a hollycast. He was lit up like Dream Street at night and wearing enough heat to stock an Imperial Armory besides. And this being scenic Wanderweb, land of enchantment, there was six of K’Jarn’s werewolves and K’Jarn facing him. I was of the opinion – then – that he couldn’t do for them before they opened him up, so, fancy-free, I opened my mouth and said: “Good morning, thou nobly-born K’Jarn. Airt hiert out to do wetwork these days or just to roll glitterborn for kicks, hey?”

K’Jarn looked up from pricing Tiggy Stardust’s clothes and said, “N’portada je, S’Cyr. Purdu.”

K’Jarn and me has known each other ever since I started running cargos into Wanderweb Free Port and he started trying to boost them. For once I should of took his advice. But, hell, it was seven-on-one and I’ve never liked K’Jarn.

“Like Imperial Mercy I will. Yon babby’s my long-lost lover and my maiden aunt and I’m taking him home to Mother any day now. Fade.”

He might have, except that just then one of K’Jarn’s wingmen got restless and took a swipe at the glitterborn with a vibroblade. Tiggy Stardust moved faster than anything human and swiped back and I burned K’Jarn before he could mix in. K’Jarn dropped his blaster, not having a hand to hold it with anymore, and left on urgent business. So did everybody else.

Business as usual in wondertown and not enough fuss for the CityGuard to show up. Except for the deader Tiggy made and another I didn’t have time to get fancy with, me and him was alone and he wasn’t moving.

I went to see if there was anything left to salvage. He snaked around and then it was me down and staring up at an inert-blade knife as long as my thigh while he choodled at me unfriendly-like.

I can get along in flash, cant, and Trade, but I couldn’t make head nor hind out of his parley, and I thought at first I’d hit my head too hard. But then I knew that what I had actually gone and done was the stupidest thing of my whole entire life. I’d rescued a hellflower.

This is one of the most wonderful novel openings in the history of SFF. Recognize it? It’s Butterfly and Hellflower by Eluki bes Shahar, not available in ebook form, but the paper omnibus edition I linked to is available right now and cheap right at the moment and if you don’t have this in your library, you should grab it while you can. If you’re interested in prose style, that goes triple.

UPDATE: Thanks to Sarah Wynde, because after she said the ebooks existed, I tracked them down WHICH WAS NOT EASY, AMAZON, what is with that? And here is the link to the series page for the ebooks.

Please Feel Free to Share:


12 thoughts on “Which is real?”

  1. Wow! I don’t have anything to say about ChatGPT, but I’m in awe of that opening to Butterfly and Hellflower. I’d love to read it, but I’d need it in ebook form.

    It sort of reminds me of the parts of Iain M. Banks’ Feersum Endjinn that are told from Bascule’s point of view.

  2. Butterfly and Hellflower are available as ebooks, just not in the omnibus edition. $4.99 each, so the omnibus is cheaper, (by a lot, currently it’s $6.25, I think) but of course, the author will get no money from the purchase of a used book, so… I prefer ebooks anyway, because that way I never have to made hard decisions about which books I am keeping when I move or when my bookshelf space is overflowing.

    As for the excerpts, I’m going to say that I got them all right, because the last two were both so bad that I skipped over them after the first few lines. I’ll sometimes use Chat when I’m writing to tell me what NOT to do. Like, I’ll ask it for two descriptions of a building and then I will describe a building that is as wildly different from those two descriptions as possible. It can be helpful when I’m stuck to just give me ideas for what I most don’t want to do next. I paid the $20/month to explore the new Creative Writing Coach feature, and it’s just as bad. Or good, if good = “don’t do this, do something else.”

  3. Sarah, I literally had to google “kindle books eluki bes shahar” to find the ebooks, even though they are on Amazon. They aren’t listed under Eluki bes Shahar’s author page AND they don’t turn up when you search the Kindle store and I honestly have no idea why it’s so difficult to link both names when an author uses two different names, but here is the link to the series page for the ebooks. And I’m updating the post, too.

  4. The first pair are easily the most distinct.

    The second pair, I figured the first one was fake when it clumsily went from in media res first paragraph to the start of the race in the second. Then I got to the second selection and it switched from racing along the Pacific Coast highway to racing through “canyons” in the second paragraph, and that set me back much the same way. But it still felt more like a real, if not particularly expert, writer.

    By the third pair, I already felt as though I was getting a feel for the text generator’s style, although not enough for it to be as obvious that the final pair are both fakes as it probably should have been. And I don’t if I could articulate it: I didn’t notice the full-sentences thing until you pointed it out.

    It does seem like someone who reads a lot of mediocre prose should pretty quickly get a feel for ChatGPT’s particular style of mediocrity.

    I haven’t read Hellflower for a lot of years, but I recognized the opening enough that “Yon babby’s my long-lost lover and my maiden aunt, and I’m taking him home to Mother any day now. Fade.” came as a confirmation.

  5. My office is currently doing a trial of using ChatGPT for legal research. When I first heard this, I thought our boss was…not technically sound, but eventually found that what she wants is something that handily cuts and pastes a useful summary of quotes from our reference library so she doesn’t have to do it manually. I don’t think ChatGPT will quite get her there, but it’s an interesting experiment.

    I could immediately see the difference for the first quotes, found the second set harder to pick, and then my attention wandered and I skipped to the end. :D But am actually commenting to mention that eluki bes shahar also published as Rosemary Edghill, and I rather liked her Bast series. Unfortunately she then got drawn into being a collaborater for a bunch of other writers, and I rarely pick those kind of collaborations up.

    I still have my Butterfly and Hellflower paperbacks – very fun space opera (with a commitment to evolved language).

  6. Andrea K., this immediately reminded me of a Legal Eagle video about using ChatGPT for legal research: https://youtu.be/oqSYljRYDEM?feature=shared, how to ruim your legal career using ChatGPT.

    ChatGPT *makes up* precedents and cases, including real-looking citations that don’t exist, or don’t say what it says they do.

    You not only have to check everything it writes, there’s a big chance of missing something important in what it didn’t write.

    When it was rolled out, there were tests done by scientist, showing it made up very real-looking ‘scientific’ articles full of bogus results and false citations, which could only be uncovered by following all its citations to the sources and reading those – when it quoted a real source, it often stated the opposite of what the results were in the real article.

    It is just a predictive text generator, trained on the internet – if there are a lot of wrong texts out there, e.g. conspiracy theories, on those subjects there is a fairly large chance of it following one of those false trails in its text generation. And as it’s doing so in the format you specified, it’ll use legal-looking language and add in real-looking citations, as it follows the format you’ve requested for a legal brief or whatever.

    Even trying to use it to get specific quotes from a specific law library, it’s a text generator: it will still make things up.

  7. Hanneke, absolutely, and doctors using text generators for diagnoses or treatments are definitely going to treat patients incorrectly. Personally, I would hold the doctor responsible for any treatment mistakes, nail at least one doctor to the wall in a highly publicized manslaughter trial, and that should help push younger, less experienced doctors away from depending on a completely undependable, untrustworthy too.

    And yes, I expect Andrea K’s boss to find out that this kind of labor-saving technique does not save labor at all because checking the information is almost certainly going to be more time consuming than having someone with a brain do the research in the first place.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top