GPT and the Writing Uncanny Valley

I make my digital doppelgänger

Given that I am spending a significant portion of every week writing a book about reading and writing in a world in which generative artificial intelligence exits, I look forward to writing about something else in the newsletter as a literal palate cleanser for my brain, but this week I got a one track mind.

The proximate cause of my obsession is OpenAI’s release of GPT Builder, which allows anyone with a GPT+ subscription to create their own customized GPT. OpenAI intends to open a GPT store the will be stocked with some native GPTs created by OpenAI (see image below), but which will primarily have products developed by individual users.

These products are called “personalized agents” and they are almost undoubtedly the future of generative AI applications. As

Marc Watkins

observes at his Rhetorica newsletter, the interface users are allowed to play with is limited to creating your own chatbot, but OpenAI’s strategy is to use all of us who create these bots as an unpaid social media marketing army, introducing users to our bespoke chatbot products.

Both ChatGPT and GPT-4 (as well as other large language models) already supported this work, but GPT Builder makes it so any old idiot can make one of these things, including this idiot, as I’ll get to in a moment.

OpenAI CEO Sam Altman shared the company’s vision during a press event, saying, “Eventually, you’ll ask a computer for what you need and it will do all these tasks for you.”

I don’t know, maybe this sounds awesome to you, but before leaping into the future, I think it’s worth pausing and considering what Altman’s vision means in the context of these personalized agents.

Automation, not intelligence

The key word to remember when considering the use of the tools of generative artificial intelligence is not “intelligence” but instead “automation.” We should be very careful about what activities we seek to automate. We should be very thoughtful about what happens when we subject human activities to automation. We should be pretty frightened of the forces that will embrace automation, even when the products of automation are demonstrably inferior to the human product, but still manage to reach a threshold of “good enough” as measured in the marketplace.

I don’t know yet if this is a war over our values requiring serious resistance, but I’m not ruling it out. At the very least, caution is necessary.

Maybe it will help to share my experience with my agent(s).

Making my digital doppelgänger

It could not have been easier to train the bot. I merely described what I wanted it to do: “Produce an approximately 600-word column appropriate for publication in the Chicago Tribune in the voice and style of me, John Warner.”

There were a couple of other prompts for clarification, but the primary training was feeding it a dozen or so past columns to give it a sense of my approach. It literally seemed to digest these examples, spitting out a summary characterizing the column’s ethos.

Reader, I was a little shook, because yeah, pretty much this is what I’m after in that space. Do I know how the algorithm extracted this from my examples? While I feel I am quite informed about the architecture and operation of large language models, I must confess that I do not know how it created this summary from the example columns I fed it.

I decided to test my new assistant on a column I would not be writing about a new book, Opposable Thumbs: How Siskel and Ebert Changed Movies Forever by Matt Singer. I read and enjoyed the book over the summer, but by the time it was released, the Tribune had already featured it in two other articles, and my editor understandably didn’t see a need for a third.

Still, I’d spent a little time thinking about what I might have to say about the book and figured it would be a good test. I wrote a prompt that reflects the inchoate notions I often start a column with, while making sure to give it information on some of the nuts and bolts information I’d be expected to share.

This is what my Biblioracle bot produced. I encourage you to read it openly and in its entirety because you are my informal focus group for better understanding the implications of this technology.

What do you think? Is it any good? For those of you know me both here and in the Tribune, does it sound like me? Would you accept this as a short newspaper column? Are there any red flags that suggest to you it was created by an algorithm, incapable of thinking, reasoning, or feeling?

I don’t think I like this guy

How am I to critique something that is supposedly made from my own repository of language? I’ll start here: I would not write that opening line in a million years. It’s overgeneralized and uninteresting on its face. It’s sort of pure filler.

“Insightful and engaging book” is phrasing I might use, but let’s also acknowledge that it’s filler, though some parts of a column are inevitably going to be filler. I have not actually followed the “trajectory of film criticism” fondly or otherwise. The use of both “spar” and “parry” is weird and redundant. I would never say “my beloved city of Chicago” under any circumstances, even if I felt the sentiment.

“Not only revolutionized film criticism but also shaped the broader cultural conversation around movies,” is a useful phrase, but again is largely filler.

This passage makes me sound like a serious weirdo:

This is pure schmaltz. Even though I know that the bot has no feelings, I cringed for it.

These three consecutive paragraphs, on the other hand, are not bad. It is perhaps notable that it is the part drawn most specifically from my prompt. It’s still a little cheesy and I would cut the last phrase about being nostalgic for the conversation, but where I’d give the previous parts of the column a D to C-, with some tweaks, this is a B. While I strive to produce an A every time out, sometimes the best I can do in a given week is a B.

The kicker paragraph (“As I closed the book…”) belongs in the trash. Pure schmaltz without hitting any genuine sentimentality.

Honestly, the bot did better than I thought, certainly far better than previous similar experiments with ChatGPT. Trying to revise and edit the copy would ultimately take longer than simply writing the thing from scratch, but I cannot deny there is a world in which some editor somewhere would read the original bot copy and think that there is a slot to fill, and while the piece may not be great, it’s good enough, and look at how I don’t have to pay a human for their original work.

Writing in the uncanny valley

To me, my Assistant Biblioracle had a kind of Stepford quality to it, reasonably pleasing on a surface level, but all of it just too much.

As I looked at this and other examples generated from my Biblioracle bot, it became clear to me that I was experiencing a kind of textual uncanny valley. The uncanny valley describes how as robotic human avatars get more and more realistic, we experience a reflexive revulsion. The text produced by my bot did bear a certain resemblance to my writing, but was also off in ways that triggered disgust. I often write with a certain sentimentality in the column, but the bot almost invariably pushed past sentiment into schmaltz, making an attempt at emotional sincerity come across as insincere.

The Biblioracle bot is made from my words, and bears a resemblance to me, but it is not me in ways that are frankly, sort of disturbing.

I hope that others are similarly disturbed, but who knows? Maybe I’m kidding myself and all the time I’m spending on this book to articulate the necessity of maintaining reading and writing as human activities is a colossal waste of time.

A warning but hopefully not a prediction

Creating a viral GPT app could be a nearly instant path to real wealth, and this development is going to kick off a rush of speculation and activity, just as OpenAI wants it.

But most of these apps are going to be garbage. The problem of hallucinations (sharing incorrect information) has not been fixed and quite probably will never be fixed.

As someone writing a book about this stuff I felt obligated to experiment with the technology, even though I personally have no desire or need for a digital writing assistant, but I am considerably worried about what it means for these apps to be in the world, particularly when it comes to how we teach writing.

One of my strengths as a writer has always been my “taste.” I know what is good, and fresh, and interesting just by reading something. I’m good at recommending books because the books I recommend are good, and I understand what makes them good. In fact, my ability to discern quality in writing - including nascent quality that would benefit from feedback - is well ahead of my own ability to create quality work. For you Top Chef fans, I’m like Tom Colicchio, a professional chef whose food is ultimately not as exciting or innovative as the food of the contestants (or at least those who make it to the end) he’s judging.

The combination of my taste, my experiences as a writer, and my interest in wrestling with the problem of teaching is what has led me the path I’m currently traveling, advocating for a different approach to how we view reading and writing in school contexts, an approach that works, but also an approach that is at odds with the status quo of schooling.

The differences between what I advocate and what is currently privileged in our systems of schooling is not one of methods, but values.

I am very afraid of how this technology can be used to enshrine the status quo in a way that will only further alienate students from the fundamental necessities of reading, writing, and thinking.

At his newsletter,

Ethan Mollick

a professor of marketing at the Wharton School who has become a kind of Pied Piper of large language models demonstrated how to build your own GPT. One of his examples is a writing feedback assistant he titled “GPT Feedback Wizard.” I really wish he hadn’t done this. Mollick is a good faith interlocutor of education technology and knows his stuff better than just about anyone, but by his own admission he doesn’t know anything about teaching writing.

He shows how a GPT can autonomously deliver redline edits on a student paper about Macbeth independent of any human intervention. It is a very impressive technical feat that is also rooted in terrible pedagogy that will nonetheless by highly attractive as a substitute for the human labor of teaching because it resembles what a so-so teacher with too many students and not enough time might deliver.

I suppose the big problem is that this bad pedagogical approach is prevalent in our schools now, incentivized by a system predicated on standardization and assessment. I was struck by this comment on last week’s newsletter by

Ryan Barry

where, as a young person not far removed from his primary and secondary education, he wonders if we haven’t forgotten the “why” in the teaching of the humanities, leaving students disengaged and alienated from the material.

Indeed. Imagine schooling entirely mediated through chatbots like Mollick’s feedback wizard, or Khan Academy’s Khanmigo where students are trapped inside this uncanny valley, interacting with simulated human texts and intelligences not because they are good or effective or fun, but because they are inexpensive, and if you squint and ignore the downsides, “good enough” particularly as measured by performance on standardized assessments. If you think young people find school to be a misery now, just wait for what full automation has in store for them.

In fairness to Mollick, there’s no doubt that some edtech entrepreneur is already working on something similar to his feedback wizard, so whatever monster he’s created with his example is already being worked on in someone else’s lab. Mollick is, at heart, a champion of individual autonomy over these tools so we may use them in ways consistent with our values, but the forces of the market are going to overwhelm any individual teacher initiative, so what gets enshrined in the broadly available tools will be hugely influential on what students experience in school.

You will not find my Biblioracle bot in the OpenAI GPT store because I would never make any bot public, no matter how tempting it may be as a way to generate attention or money because at this time we have not done the work of figuring out what is lost when certain things are automated. I’m deep enough into the drafting of my book to know that when it comes to writing and reading, these losses would be profound. While it may be tempting to convince myself that I could make the bot that could avoid the pitfalls of automation, that would be a lie, and I’m not going to play in that space.

People have a right to know, explore, and express their own minds as the unique intelligences they are while interacting with and relating to other unique intelligences. Our bots are not us.

Denying students the chance to develop their unique selves because we’ve decided writing with, or to, or among bots is good enough for the purpose of passing assessments would be a disaster. Valuing the bot over the human because of its speed and efficiency is a choice to consign ourselves to an intellectual and emotional oblivion.

I understand that I’m probably coming across like a Cassandra, the Trojan priestess who foretold a doom so dire no one would believe her, but here’s the thing, Cassandra was right.

Links

My Chicago Tribune column this week is about how the death of Bobby Knight made me realize that we don’t get sports books like the classic, A Season on the Brink, John Feinstein’s embedded year with Knight’s Indiana Hoosiers anymore, and how that’s probably a good thing in terms of what it says about sports.

Warner Brothers has decided to shelve the fully completed, apparently quite entertaining movie Coyote v. Acme, drawn from Ian Frazier’s classic humor piece of the same name in order to benefit from a tax write-off so the movie will never been seen. Time to change the law saw that anything like this that is compensated with public tax dollars passes into the public domain.

The Aspen Words prize ($35k and presumably a trip to Aspen!) has announced its longlist of 14 titles.

At Esquire, Kate Dwyer asks, “Has it Ever Been Harder to Make it as an Author?”

At Bustle, Arianna Rebolini has seven books you should read, some old, some new.

Fascinating deep dive into what data about the judges and the nominees can tell us about who is going to win the National Book Award.

At McSweeney’s, Jeff Drake offers “This Week’s Punctuation Power Rankings.”

Recommendations

1. The Invisible World by Nora Fussner
2. Mumbo Jumbo by Ishmael Reed
3. Counterweight by Djuna
4. The Artifical Kid by Bruce Sterling
5. The Long Goodbye by Raymond Chandler

Sean W. - Los Angeles, CA

This is a very interesting mix of books. I’m going with a book that mixes in some noir with a little supernatural dystopia, Max Barry’s The 22 Murders of Madison May.

1. The Vaster Wilds by Lauren Groff
2. Drive Your Plow Over the Bones of the Dead by Olga Tokarczuk
3. Small Things Like These by Claire Keegan
4. The Spinning Heart by Donal Ryan
5. The Fisherman by Chigozie Obioma

Linda H. - Laguna Niguel, CA

Exit West by Mohsin Hamid combines allegory with the supernatural to reveal a universal story underneath. I think it’s a good fit for Linda.¹

There’s actually a bit of a backlog on recommendation requests, which is good because next week I’m planning a links and recommendations extravaganza because I’ll be off delivering a plenary address at the Lilly Conference on College Teaching at Miami University in Ohio on the day when I am usually writing this newsletter.

If you want to get those requests in, I’ll try to get to as many as possible.

Stay human, my friends.

John
The Biblioracle

1. All books linked throughout the newsletter go to The Biblioracle Recommends bookstore at Bookshop.org. Affiliate proceeds, plus a personal matching donation of my own, go to Chicago’s Open Books and the Teacher Salary Project, which is advocating to establish a federal minimum salary for teachers of $60,000 per year. Affiliate income is $259.50 for the year.