The Worst Use Case for Generative AI is Writing
Time for a victory lap.
I’m just going to go ahead and engage in a little self-congratulation, a wee bit of valedictory preening over not just being correct, but being publicly, clearly, 100% on the money before lots of other people came around to my way of thinking.
From the first appearance of ChatGPT in November of 2022 I said that whatever the applications of the technology may be, deploying generative AI tools for “writing” is the worst possible use.
I have been arguing, again, from the beginning, that writing is an inherently human activity and that the capacity of LLMs to simulate writing through automated text generation did not change this irrefutable fact. When people wrote about how they loved using ChatGPT to write a first draft or as a brainstorming partner, I continued to insist that by doing so they are changing writing into not writing, but something else, and these changes would be demonstrably bad both short and long term.
I put all these beliefs and others on the record in the form of a book, written 100% by me, except for the chapter where I experimented with letting ChatGPT write an example of my Chicago Tribune column in order to show the differences between writing and automated text generation.
That book was published in February 2025. It was correct on these core points then. It is correct on these core points now and while it is frustrating that it is taking so long for others to come to this realization, there have been a handful of recent data points that have been very encouraging on this front.
One of these data points is an op-ed by Rebecca Winthrop published this week in the New York Times which looks at a couple of studies examining student writing, both of which find that AI-assisted text production literally narrows the range of ideas explored and makes the user less creative. As the co-author of The Disengaged Teen: Helping Kids Learn Better, Feel Better, and Live Better, Winthrop gets it. She’s spent years pushing against the transactional model of schooling.
She describes a world where AI-assistance sands away the interesting parts of our humanity, starting in our earliest years, ultimately leaving us incapable of even recognizing a genuinely original expression.
Yech.
The next data point comes from a more surprising place, the newsletter of Ethan Mollick, author of Co-Intelligence: Living and Working with AI, someone who I both cite as one of my most important guides to AI in More Than Words, and sometimes label (with a mix of envy and derision) “the AI-whisperer for the consultant class.”
Mollick has had enough with AI-generated text showing up, well…everywhere, and has some good insights on why this is bad beyond our surface-level annoyance.
It isn’t just the sameness of the AI writing, though that eventually gets to be tedious enough that I find myself skipping writing on even interesting topics if my internal “AI detector” goes off. It is also that badly prompted AI writing produces very little meaning per word, taking you in intellectual circles instead. We are trained to read well-crafted sentences and intellectual sounding texts as the result of effortful human work and thus pay attention to these AI written comments when we see them. But there is often no human meaning there, these posts are just meaning-shaped attention vampires that take mental effort to decode and give you no equivalent understanding in return.
“Meaning-shaped attention vampires” is gonna stick with me. In a way, AI-generated text falls into the same trap as the artifacts my students would produce when they had been trained to write to rubrics and standardized assessments like the 5-paragraph essay before ChatGPT showed up.
I called these things “writing-related simulations” in order to distinguish them from the genuine artifact. When students combined the writing-related simulation with the “academic pseudo-academic B.S.” via injudicious use of the Microsoft Word thesaurus function, you’d get something very much like the LLM outputs of today. When I would ask students why they would give into these impulses to pull out a ten-dollar word they didn’t fully know the meaning of, they said it was a good technique to “sound smart” which is what passed for learning in the context of their schooling.
LLMs always sound smart, even when they say nothing.
It drove me nuts in the early 2000’s when I first started seeing writing-related simulations chock with pseudo-academic B.S., and I fought against it throughout my entire career by insisting to students that I was interested in their thoughts, and that writing is an inherently communicative act where the communication must come from a unique intelligence.
They didn’t always believe me because school is school, but I did my best to hold the line.
Mollick does not throw LLMs as writing assistance entirely under the bus (emphasis mine)…
This is not a condemnation of using AI to help with writing in any way. I think AI can be a fantastic tool for good writers (I have AI check all of my writing and roleplay different reader perspectives to see if I missed something important). For those who struggle with communication, AI can help get their ideas across better, and writing may not be thinking for everyone. Plus, a little bit of effort can make AI writing less cliche, more personal, and more worth using (in moderation). So, this is instead a condemnation of using AI as a default, or, even worse, without thinking at all. Balancing using AI with our own mental abilities is going to be a defining challenge of the coming years.
..but I will. LLMs can be helpful for automated or automation-assisted text generation, which has some real utility in the world, but when it comes to writing they have vanishingly little to add. Mollick’s best argument is that if you really really know how to write you may be able to use the technology in a way that mitigates the potential downsides, but even his one specific example “role-play different reader perspectives” is kind of bullshitty. Audience awareness is a key skill/habit-of-mind of writers, but the notion that this is something you need a non-reading chatbot to do rather than continuing to hone your own editorial senses is wrong. Maybe it makes Mollick feel better to check in with Claude, but it wasn’t necessary before the tech existed and it isn’t necessary now.
When it comes to writing, this balancing Mollick speaks of is all downside. Fortunately, you can avoid falling off the wrong side of that balance simply by writing.
Thankfully, the perfect example for what happens when you slide to the bottom of the AI-assistance slope has arrived in the form of this dope.
Meet, Steven Rosenbaum, author of The Future of Truth: How AI Shapes Reality who was outed via a New York Times investigation as having included a number of AI-fabricated quotations in his book about AI and truth.
Normally I try to refrain from calling people dopes unless they are obviously, irredeemably dopey, but rather than slinking off into ignominy, Rosenbaum has been talking to curious reporters, with each engagement dopier than the next.
Kate Knibbs of Wired reached out to Rosenbaum, and well, the article headline says it all:
Rosenbaum seems unable to discern what is his ass, and what is a hole in the ground. After some general incoherence about his method of AI use, Knibbs probes deeper.
He doubled down on his personal commitment to AI, noting that he still uses it every day. “If the only way for me to not end up with a mistake ever again is to literally stop using AI, that’s just not realistic. If the answer is to stop writing, that’s not out of the realm of possibility.”
I asked him whether he would rather stop writing than stop using AI in his writing process. “Yeah,” he answered.
Rosenbaum vacillated between acknowledging that AI use could cause problems (“I do not understand why it’s my job as an author to play whack-a-mole with a multibillion-dollar company who puts hallucinations into their feed as a business practice”) and repeatedly insisting that AI is indispensable, calling it the best writing partner he has ever had.
Rosenbaum is the end state of the people who presumably once knew how to write that allow this technology to invade their process. He uses AI every day and would even stop writing if he couldn’t use it. This is a fully grown human who wrote a book before LLMs existed who, by his own admission, cannot function without them.
Even Winthrop, in a comment at the New York Times, remarked how working on a document on a plane while cut off from accessing the AI model she uses revealed that she had lost some measure of her “patience” for the struggle of writing because she had been habituated to “collaborating” with AI. I reality, it was “a total crutch.”
The idea that we should inject this technology into classrooms for students to figure out how to co-create with AI while they’re in the early stages of their development as writers and thinkers seems like a rather dubious proposition, no?
What’s exciting about these sorts of reflections showing up in the work of prominent people and in opinion-shaping spaces is that we have an opportunity to stop going down a damaging path before we get all that far.
I am thinking of the damage done via the incredibly rapid and thorough embrace of the “grit narrative” around education, positing that if we could just teach kids resiliency they could overcome whatever obstacles they found in their way - poverty, hunger, etc.
To be fair, this was not the intended message of Angela Duckworth, the progenitor of the concept, but the way the concept was put into practice was hugely counterproductive. Duckworth herself started refuting some of these practices fairly early on, but it’s not until a decade later that we get a new book from Duckworth saying what seemed obvious to others, including me (warning: more self-congratulation coming).
We can nip the push to teach students AI-integrated document production in the bud and instead focus on what we know how to teach…writing.
However, the same structural factors of insufficient human-based resources, a transactional system, and a crisis around student engagement that has been going on so long we must now see it as something other than a crisis, are still in place. These barriers are significant, but they are not impossible to overcome if we orient ourselves towards a different way of viewing the work of teaching writing and learning to write.
We must frame writing as an inherently human act of communication between unique intelligences.
Writing in school must put engagement at the center of every writing experience in order to attack the demand-side problem of students turning to AI to satisfy the transaction of schooling, rather than doing the work of learning.
We must shift our assessment from judging products to valuing process and individual student growth in the building of their writing practices, the skills, knowledge, attitudes, and habits-of-mind of writers.
Resources must be put into the labor of teaching and learning to write so that the scale of interaction is appropriate to our goals of helping individuals learn to think and express themselves to the world through writing.
The rise of agentic AI models has shown how powerful language is when it is untethered from understanding. The notion that an AI model can independently manipulate aspects of our world to, for example, conceive and execute a money-making online business is amazing.
These powers of manipulation without understanding make it tempting to see this as the future of the deployment of language. But our experience of automated text is already sufficient to demonstrate that for we humans, this is a dead end.
Let’s treat it that way.
Links
In my Chicago Tribune column this week I laid out my desired response to Steven Rosenbaum’s publishing of fake AI stuff: Recall the book, pulp it, never trust him or his publisher again.
At Academic Freedom on the Line I published a post from Bradford Vivian calling out the hypocrisy of Yale Univeristy lecturing the world about trust in higher ed while throwing their own faculty under the bus.
Sam Kriss is someone else who has had it with AI “writing.”
Always love these pieces from book designer Nathaniel Roy exploring evolutions of cover designs, this time with author Jessica Berger Gross mulling the difference between hardcover and paperback versions.
Via Defector, an excerpt from Joshua Kendall’s new biography of Gary Trudeau of Doonesbury fame about the intersection of the comic and Donald Trump.
Earlier this week I spoke with Naomi Kanakia about her new book, What’s So Great About the Great Books, which is in a second printing in its first week.
From my friends McSweeney's a thematically apropos piece for this week’s newsletter: “AI Writing Jobs You Should Apply to Today” by Miles Kahn.
Recommendations
1. Even the Good Girls Will Cry by Melissa Auf Der Mau
2. Yesteryear by Caro Claire Burke
3. Love Is a Mixtape by Rob Sheffield
4. Bread of Angels by Patti Smith
5. Best American Essays: 2009 edited by Mary Oliver
Renee D. - Cody, WY
Renee is clearly drawn toward nonfiction, so I’m going to recommend a book by an incredible, but under-read essayist, Peter Coviello. I’ll let Renee choose the specific book, either Long Players: A Love Story in 18 Songs, or Is There God After Prince? Dispatches from an Age of Last Things.
So here’s an early alert for something that I’m participating in that I think will be very fun. The Mark Twain House and Museum in Hartford, Connecticut is having a two-day event (November 6 & 7) celebrating the awarding of the Mark Twain American Voice in Literature Award which just announced its longlist of contenders.
On November 7th I’ll be moderating a series of conversations and events about creative writing in the age of AI. More participants are yet to be announced, but in addition to yours truly, novelist and sponsor of the Twain prize, David Baldacci will be there, along with everyone’s favorite book reviewer, Ron Charles.
More details are here. Consider joining us!
Alrighty folks, I am off to have the rest of my weekend. I will see you next week, same time, same place.
JW
The Biblioracle
I think that this line from Mollick is illuminating, but not in a good way: "(I have AI check all of my writing and roleplay different reader perspectives to see if I missed something important)"
My experience as a writer is that I always miss something important. That's just the nature of the beast - there's no way to communicate everything, and you always think of other things that you could have said after the fact. AI's creation of what appears on the surface to be "perfect" prose is engaging with people's perfectionism in unhealthy ways, particularly students who struggle with writing, are trying to create an impressive academic resume, etc.
The Winthrop op-ed is also concerning - she writes that "the linguistic coverup worked; post-ChatGPT essays were rated as more “creative” by human judges, even if the substance of the essays trod familiar territory." So we appear to be heading towards a world in which writers of all sorts see AI as a tool that can make their writing "more perfect" when actually it's making writing bland and devoid of meaning.
As a writer and a high school English teacher of 20 years, I'm torn. If we do not get in the trenches with our students with AI in the room, we lose the opportunity to teach them its limitations. We punted social media outside the classroom and lost the chance to teach to prevent the hacking of our students' attention and attachment systems. I don't disagree with a lot of what you are saying, but I'm putting my labor behind trying to figure out how to build my students' self-regulation and awareness. I need to be in relationships with my students so that they can write when it counts and we need to move beyond transactional spaces in education so that engagement increases making it count more. AND we need to show them how some writing is transactional and needs to be efficient with AI co-drafting.