Thanks for sharing this, John. Uncanny valley is right, and filler is right, too. If you were just looking to fill space with something that resembled a book review, I suppose that PseudoJohn's performance might be adequate. But if you actually wanted to learn something about the book, or the subject, or the reviewer's perspective, there's simply no there there - no detail about Singer's book, no specifics about Siskel and Ebert, no distinct critical style or sensibility. Beyond maybe picking up on a few habitual phrasings, the bot doesn't actually sound like your writing at all. It's bland, superficially-plausible BS, which is to say, pretty much what I've come to expect from ChatGPT et al.
So, I'd say it's a pretty shoddy simulation of your writing, but beyond that, it's only a simulation. I'm reminded again of Maha Bali's thoughts on the automation of care. There is no meaning without caring, without a commitment to the belief that something matters. But ChatGPT can't care, and so it can't create meaning. When we remember, for example, that ChatGPT can't feel nostalgia, has never lived in Chicago, etc., whatever text it extrudes is exposed as a sham. If we encounter that sham in a context where we expect meaningful communication, not just filler, we would be justified in feeling that the person responsible (because there is always a person responsible) has violated our trust, and their own duty of care.
I'd note as well that my lack of Cassandra-like tendencies over AI come both from knowing about the moral panic when a computer beat a chess champion and from my conviction that AI might work for some writing (cover letters, strategic plans, book reports), it's never going to replace wonder at true human creation. People still play chess, after all. And I don't think AI could reproduce, say, the bone-chilling effect that Stephen King fans get from his tales, or the upper middle class humor of Laurie Colwin, or anything remotely like Shakespeare (except perhaps in vocabulary).
Whether anyone aside from a small group of people will care about all that I don't know, but the subset of people who love and care about books and writing has always been fairly small, and making a living from writing has always been hard, though the writers I mention mostly did it. (I didn't mention Melville here, but Moby Dick would be a good example, as would Emily Dickinson's poetry).
I agree that this tech is not a threat to people like me, but the way people are salivating to deploy it in education is deeply worrisome to me when it comes to the kinds of experiences students will have with reading and writing. They may literally never have the kinds of experiences that provide a window into why those of us who love reading and feel compelled to write do so. This is already largely what students experience. Generative AI will ratchet this up to 11 and beyond not because it's good, but because it allows for the substitution of cheap automation for expensive labor.
My son demonstrated to me how this works. “Write a column in the style of Dennis Byrne for the Chicago Tribune.” In a second it produced one that made more sense than my usual scribblings. I
The big red flags for me (beyond the schmaltz) are the lack of specifics. Without specific memories, specific quotes and passages that resonated for you, and the connections it drew you to make to other texts or thinkers or films, it doesn’t sound like you at all
Yikes, John, it's eerie how it is, as you said, superficially like you, but there's no THERE there. No one who pays attention to your work would be fooled beyond a minute or so. It honestly reminds me, as a former public middle school teacher, of a seemingly bright but lazy student, who is totally not me, trying to convince the teacher that she did the assignment, and did a good job, without putting any real time or thought into the essay. These AI products may glitter, but they certainly are not gold.
I think that losing the "why" of literacy has already hurt education (and therefore culture) in noticable ways. Everywhere you look people are talking about lower literacy levels in students and adults. Now, any educator knows that the so-called literacy level is a hairsbreadth away from complete nonsense, but something is certainly happening with reading and writing in this country. I tend to see it as related to the loss of why we, as a species, learn. Learning is what we do to more fully experience the world and then to build upon that experience and share it with others, enriching our understanding and lives in the process. Learning makes us more human. Replacing the human connection with utilitarian goals kind of eliminates the point of learning anything at all.
This was fascinating, if unsettling. Thank you for doing this 'test' and for writing about it. It brings the whole scary thing to life. Frankencolumns?
What I most noticed in the AI-produced column is that it said absolutely nothing about the book it was supposed to be reviewing/prompted by (the Opposable Thumbs one). Presumably this is because it has no access to that book. I also noticed how many paragraphs the AI essay spent on the nostalgia portion (both for watching Siskel & Ebert and for Chicago). That’s again because it had very little real content to work with. It could only expand on the prompt. Specifics overall were missing (which is typical of AI writing).
I've only recently discovered your Substack, but reading this, 'my beloved city of Chicago' immediately struck me too. 'The Biblioracle would never say that', I thought to myself. There are many other examples throughout the piece, but for me that was the first and most striking. It's just *not your voice*. And I'm new to your writerly voice!
I agree with jwr below (or above?) - no detail, no sensibility, no style, no 'there there'. There is just no *voice* to this piece of writing. It reads like it took the paragraph of instructions you gave it and just repeated it with slight variations over and over. It's like bad marketing copy!
My heart rends at the idea of this technology becoming part of the human landscape.
This comment is late to the game because I was visiting your lovely cities of Charleston and Mt. Pleasant that weekend for my son's wedding, and only today getting caught up to your newsletter. See what I did here? You write as yourself, who lives in the real world and shares that with your readers.
No, this is not you! Not your voice! It rings hollow. I am not as conversant as the other commenters in my ability to pull out specifics but the best comment below (or above, I can't figure out the order) is the "there is no there, there." The chatbot of this column has never lived in Chicago!
Damn, I was all set for the big twist: the ENTIRE COLUMN was ChatGPT generated.
Anyway, aside from not being the least bit funny, which removes all doubt that the Biblioflunky isn't your voice, I feel pretty confident I could tell the difference between it and you without being set up first. But if you told me that another human being had written it? I don't know if I could call the writer out as a bot. I've read actual pieces like that, and while it reads less as a column/review and more as an expansion of the book's dust jacket copy, it gets a lot of things right. I think you're right about the bot being (unfortunately) the perfect tool to make something that's good enough to fill space. Maybe ChatGPT's popularity will mean that people will start being able to identify it more readily. I just hope they think it's worth making the distinction.
This was frightening, yet enlightening. Thanks for the Sunday morning read and reflection.
Thanks for sharing this, John. Uncanny valley is right, and filler is right, too. If you were just looking to fill space with something that resembled a book review, I suppose that PseudoJohn's performance might be adequate. But if you actually wanted to learn something about the book, or the subject, or the reviewer's perspective, there's simply no there there - no detail about Singer's book, no specifics about Siskel and Ebert, no distinct critical style or sensibility. Beyond maybe picking up on a few habitual phrasings, the bot doesn't actually sound like your writing at all. It's bland, superficially-plausible BS, which is to say, pretty much what I've come to expect from ChatGPT et al.
So, I'd say it's a pretty shoddy simulation of your writing, but beyond that, it's only a simulation. I'm reminded again of Maha Bali's thoughts on the automation of care. There is no meaning without caring, without a commitment to the belief that something matters. But ChatGPT can't care, and so it can't create meaning. When we remember, for example, that ChatGPT can't feel nostalgia, has never lived in Chicago, etc., whatever text it extrudes is exposed as a sham. If we encounter that sham in a context where we expect meaningful communication, not just filler, we would be justified in feeling that the person responsible (because there is always a person responsible) has violated our trust, and their own duty of care.
https://blog.mahabali.me/educational-technology-2/on-the-automation-of-care/
Does not sound remotely like you, though I suppose it does cover the ground you asked it to.
I'd note as well that my lack of Cassandra-like tendencies over AI come both from knowing about the moral panic when a computer beat a chess champion and from my conviction that AI might work for some writing (cover letters, strategic plans, book reports), it's never going to replace wonder at true human creation. People still play chess, after all. And I don't think AI could reproduce, say, the bone-chilling effect that Stephen King fans get from his tales, or the upper middle class humor of Laurie Colwin, or anything remotely like Shakespeare (except perhaps in vocabulary).
Whether anyone aside from a small group of people will care about all that I don't know, but the subset of people who love and care about books and writing has always been fairly small, and making a living from writing has always been hard, though the writers I mention mostly did it. (I didn't mention Melville here, but Moby Dick would be a good example, as would Emily Dickinson's poetry).
I agree that this tech is not a threat to people like me, but the way people are salivating to deploy it in education is deeply worrisome to me when it comes to the kinds of experiences students will have with reading and writing. They may literally never have the kinds of experiences that provide a window into why those of us who love reading and feel compelled to write do so. This is already largely what students experience. Generative AI will ratchet this up to 11 and beyond not because it's good, but because it allows for the substitution of cheap automation for expensive labor.
"A delightful journey down memory lane." Yikes. That was the first clue that this was not the real John Warner.
My son demonstrated to me how this works. “Write a column in the style of Dennis Byrne for the Chicago Tribune.” In a second it produced one that made more sense than my usual scribblings. I
The big red flags for me (beyond the schmaltz) are the lack of specifics. Without specific memories, specific quotes and passages that resonated for you, and the connections it drew you to make to other texts or thinkers or films, it doesn’t sound like you at all
Yikes, John, it's eerie how it is, as you said, superficially like you, but there's no THERE there. No one who pays attention to your work would be fooled beyond a minute or so. It honestly reminds me, as a former public middle school teacher, of a seemingly bright but lazy student, who is totally not me, trying to convince the teacher that she did the assignment, and did a good job, without putting any real time or thought into the essay. These AI products may glitter, but they certainly are not gold.
It sounded like some facsimile of you - but I know you would not have used the term "nostalgia" that many times in a column
I think that losing the "why" of literacy has already hurt education (and therefore culture) in noticable ways. Everywhere you look people are talking about lower literacy levels in students and adults. Now, any educator knows that the so-called literacy level is a hairsbreadth away from complete nonsense, but something is certainly happening with reading and writing in this country. I tend to see it as related to the loss of why we, as a species, learn. Learning is what we do to more fully experience the world and then to build upon that experience and share it with others, enriching our understanding and lives in the process. Learning makes us more human. Replacing the human connection with utilitarian goals kind of eliminates the point of learning anything at all.
This was fascinating, if unsettling. Thank you for doing this 'test' and for writing about it. It brings the whole scary thing to life. Frankencolumns?
What I most noticed in the AI-produced column is that it said absolutely nothing about the book it was supposed to be reviewing/prompted by (the Opposable Thumbs one). Presumably this is because it has no access to that book. I also noticed how many paragraphs the AI essay spent on the nostalgia portion (both for watching Siskel & Ebert and for Chicago). That’s again because it had very little real content to work with. It could only expand on the prompt. Specifics overall were missing (which is typical of AI writing).
I've only recently discovered your Substack, but reading this, 'my beloved city of Chicago' immediately struck me too. 'The Biblioracle would never say that', I thought to myself. There are many other examples throughout the piece, but for me that was the first and most striking. It's just *not your voice*. And I'm new to your writerly voice!
I agree with jwr below (or above?) - no detail, no sensibility, no style, no 'there there'. There is just no *voice* to this piece of writing. It reads like it took the paragraph of instructions you gave it and just repeated it with slight variations over and over. It's like bad marketing copy!
My heart rends at the idea of this technology becoming part of the human landscape.
Hi, John,
This comment is late to the game because I was visiting your lovely cities of Charleston and Mt. Pleasant that weekend for my son's wedding, and only today getting caught up to your newsletter. See what I did here? You write as yourself, who lives in the real world and shares that with your readers.
No, this is not you! Not your voice! It rings hollow. I am not as conversant as the other commenters in my ability to pull out specifics but the best comment below (or above, I can't figure out the order) is the "there is no there, there." The chatbot of this column has never lived in Chicago!
Damn, I was all set for the big twist: the ENTIRE COLUMN was ChatGPT generated.
Anyway, aside from not being the least bit funny, which removes all doubt that the Biblioflunky isn't your voice, I feel pretty confident I could tell the difference between it and you without being set up first. But if you told me that another human being had written it? I don't know if I could call the writer out as a bot. I've read actual pieces like that, and while it reads less as a column/review and more as an expansion of the book's dust jacket copy, it gets a lot of things right. I think you're right about the bot being (unfortunately) the perfect tool to make something that's good enough to fill space. Maybe ChatGPT's popularity will mean that people will start being able to identify it more readily. I just hope they think it's worth making the distinction.