About a year ago, I began experimenting with having AI write a feedback letter to each of my students instead of using a rubric. I read their essay, spoke into AI my thoughts about what was impactful about it and what confused me or didn't land. I spoke about what the mentor texts for the unit did compared to what they achieved. I asked AI to use an encouraging tone in the letter. Sometimes I asked AI to explain a grammar rule with examples. I then asked it to summarize three next steps a student could take for the next essay. The letters it produced were better than I'd have the time for. They were well-received from the students. I asked them to take them out and read again the next steps after their first draft of their next essay. This process helped me realize a vision I had for how I might be able to manage feedback in a way that helped me avoid the procrastination I'd always felt around "grading." My procrastination was driven by a feeling that students needed more than what my feedback and my emotional reserves could produce so I just kept pushing it off and avoiding it. I hid from them that AI wrote these letters until our AI literacy unit this year. Then, when we discussed the use of AI in grading systems, I revealed my process and asked for honest feedback. They (at least told me) that they didn't care that AI wrote it as long as I read their papers, I gave the feedback ideas itself that AI put into an organized form, and that I read over the letters to make sure it was what I really thought. I still feel a little bad about using AI in this way. I'm not sure why...anyone have any ideas? I think it might have something to do with how I tend to only use AI for transactional writing and I am very much in a relationship with students. What do you think? Should I feel bad?
I don't think you should feel bad at all because it's clear your primary priority is student well-being and it's just as clear that you're working the problem with intention and purpose. This is all we could ask of anyone who works in the conditions that must of us labor (or in my case, labored) under. The structural constraints that govern the work of teaching are overwhelming, or at least I often found them that way. Even the phrase "emotional reserves" had me flashing back to periods where I'd be grading on a weekend and looking at the stack of assignments I still had to respond to and feel something like despair. That feedback was hugely compromised in those cases because I was just trying to survive it. I had all kinds of hacks I deployed - like about 30 macros of common comments in MS Word - that made the work manageable, but over time those hacks made me more and more distanced from the work over time.
I didn't feel guilt so much as deep frustration and easing that frustration became the goal. I wasn't going to ever have fewer students or "better prepared" students, so I had to iterate towards what felt meaningful. As motivators go I'll take frustration over guilt any day.
From my perspective of having gone through a journey of questioning before AI existed, I'd asked why this kind of feedback is "transactional." The question I kept asking myself over the numerous semesters of experimentation was essentially, "what if all of this is relational?" In terms of both my relationship with students and my students relationship to writing and expression. This is where my framework of the writing practice came from as I needed to have some way to talk about the writing without focusing on the grade or even what they should do next time. We were going to build a bigger thing than a method for writing a school essay.
The type of feedback I have became much less standard, and requiring students to reflect on their own work was ultimately the game changer. Once I committed to a full alternative grading scheme, everything locked into place.
And then, relatively shortly thereafter it became clear that I couldn't afford to teach anymore and so here we are.
Reading this piece reminded me of teachers who took me seriously. Thank you for those reminders. It also occurs to me that the writing that I did that they called me on was a lot like AI generated writing. That is, I allowed pre-fabricated phrases and conventional thinking to be the sum total of my work. Because it was fluent, and because it was technically correct, most teachers praised it. The few who didn’t taught me to push harder and think better. Thanks for reminding me of those great teachers.
Just asked students this week about the suggestion that teachers use AI feedback to support/expedite the feedback they leave for student writing, and almost unanimously this was the response: "why should we care about what we write if they don't care enough to read it and leave feedback?"
Taking the time to leave feedback is itself a signal of care, not to mention the opportunity it avails (particularly with experience!) to coach/motivate student writers over a school year.
AI threatens all of that—especially if this newest generation of teachers outsource this from the get-go and never develop the craft/capacity.
My first thought was “Perhaps the law professors are not very good at providing feedback” followed by “I did not go to law school, but perhaps there is a formulaic aspect to some learning there that would lead you to believe that formulaic AI feedback was useful” followed by all the descriptions I’ve heard of my father—nearly universally praised as the best teacher any student ever had—handing back a paper he’d ripped to shreds and telling the student to rewrite it—and then doing that over and over again, the whole semester, with just that one paper. And then I think about how few professors or instructor of any kind have the kind of time and freedom my father did to operate in that way.
Having teachers and professors who take the time to give detailed - and the kind of honest evaluation that can only come from the intimate contextual information available through observation (work ethic, classroom interest, questions, etc...) and, therefore, deeply human - feedback on your writing is a gift and a privilege many students do not get - and, if they do, they rarely get it from qualified or expert writers who have reached your level of insight from years of moving through the process you describe - most secondary school teachers are still mired in the rubric system and, even then, have difficulty applying it consistently. One of my colleagues who uses AI regularly in tandem to flag certain aspects of student work justifies his practice, in part, because AI will fairly apply standards across written work that doesn't reflect the frequent bias of teachers pre-disposed towards stronger students. This is certainly a result of the tyranny of the grading system which many of us are stuck in as a realistic constraint of our roles. So, there is no question that what you lay out is clearly the ideal. I still recall my college freshman writing instructor who observed that I was making significant progress halfway through a semester of over a dozen assignments strewn with red ink (and not graded which I think is the hugely and frequently overlooked point in your description) which spurred me on in the class. Just knowing that someone (who was a published author) showed that level of confidence in my improvement as a writer was valuable beyond words that no AI level of feedback could provide. But what about the thousands of students who don't have access to those teachers? Or any meaningful feedback in general? I don't know the answer but I am curious whether you think novice writers - at least those who want to reach a general level of competence (after all, the overwhelming majority of students will not become professional writers though everyone needs to learn to express themselves clearly through the written word, at least in a practical sense) can gain some value from the kind of feedback LLMs can provide. For those rubrics you lay out above, an LLM can offer highly specific and technical feedback well beyond what most teachers can possibly offer. I'm just wondering whether you think any of that can be valuable in tandem with what K.A. Keener explains in her comment. Is there a place for thoughtful teachers to couple the unique capabilities of AI alongside the more valuable and insightful feedback from a caring and careful reader? A teacher who does not - or cannot for reasons of workload or work ethic - read a student's work should not assign it (unless the goal is simply to have the student write for themselves in a private reflection). But for students who don't get any of this at all - they don't get feedback and just a letter grade - is there a place for them on their own to use LLMs to make them better writers? Many of my students have told me they get value from this though it's hard to know whether they are only thinking of that through a grading perspective. I think this is the question that many teachers are grappling with, especially those who have over a hundred or more students - how do you realistically assign multiple pieces of writing and provide ideal and sophisticated feedback with that many students? I loved the Great Books course I taught - and the short papers that came with it - primarily for the reasons you describe here. But it was unique because the reading and conversation and interest in the class overall made up the bulk of the grade - the written work was personal, reflective, and connected to the text - and I only had 9 students! Very, very few teachers have that luxury. Anyway, this was a very thoughtful analysis of what I think is going to continue to be a larger and larger conversation, not only as AI becomes more and more integrated into schools and student's lives, but as we watch the impacts of how much it is already affecting the billions of smaller pieces of business communications all over the world. I'm going to share this piece with our AI Roundtable that is happening next week. Thanks!
So many deep questions with no definitive answers, but let me work through some stuff.
"...a privilege many students do not get - and, if they do, they rarely get it from qualified or expert writers who have reached your level of insight from years of moving through the process you describe - most secondary school teachers are still mired in the rubric system and, even then, have difficulty applying it consistently. "
I think we need to continue to note and examine this missing "privilege," and for my part I'll suggest that one step might be to understand why we consider this to be a privilege as opposed to right. One thing I'd note is that this work does not require qualified, expert writers. I went to excellent public schools, but my teachers were also "ordinary" human beings. They were simply doing their jobs, doing them well (many of them, anyway), but not requiring any kind of special training, experience or effort. I don't know why we should see this as beyond our capacity as a society. Giving up on this as a possibility would be akin to giving up on democratic self-governance, which, maybe we have.
"One of my colleagues who uses AI regularly in tandem to flag certain aspects of student work justifies his practice, in part, because AI will fairly apply standards across written work that doesn't reflect the frequent bias of teachers pre-disposed towards stronger students. This is certainly a result of the tyranny of the grading system which many of us are stuck in as a realistic constraint of our roles. So, there is no question that what you lay out is clearly the ideal."
Couple thoughts. One is that I don't know why "consistency" has become some kind of idea, or rather I do, but this is another value we should interrogate and keep where it works and discard where it doesn't. The feedback we give students on writing should be in the service of helping them develop their writing practices. Given that students will be all over the map in terms of their development, consistency of feedback strikes me as a lesser goal, relative to useful/meaningful feedback to the individual.
For sure a grading system that is designed to rank and sort students inside a schooling context is a structural barrier, but this structure largely was not part of my upbringing (at least not until high school), so it's not as though we don't have experience with thinking differently.
"Just knowing that someone (who was a published author) showed that level of confidence in my improvement as a writer was valuable beyond words that no AI level of feedback could provide."
This is true, but I'd also say that this feedback need not come from a professional writer. It could be any other human, including a classmate/colleague.
"But what about the thousands of students who don't have access to those teachers? Or any meaningful feedback in general?"
This is a question where my friend Jane Rosenzweig's framing of "What problem is the AI solving?" comes in handy because what you're describing is a lack of access to human beings, not a flaw of writing pedagogy. I would prefer we attack that problem. Dave Eggers' 826 writing centers are designed to be helpful in this lack, providing an army of volunteer humans that are accessible to students in afterschool programs. I don't think we lack the human resources, but we may be hampered by the fact that the resources aren't going towards humans.
"I don't know the answer but I am curious whether you think novice writers - at least those who want to reach a general level of competence (after all, the overwhelming majority of students will not become professional writers though everyone needs to learn to express themselves clearly through the written word, at least in a practical sense) can gain some value from the kind of feedback LLMs can provide."
The question here is competence "at what?" Competence at doing school? Clearly LLMs can help. Competence at becoming thinking, problem solving humans with access to their own unique intelligences? I'm skeptical. The question of whether this is worthwhile depends on what you value. Over the years I came to put almost no value in the schooling part of school.
"For those rubrics you lay out above, an LLM can offer highly specific and technical feedback well beyond what most teachers can possibly offer."
Much to my chagrin, considering how many students I subject to those rubrics, they have nothing to offer if the goal is to help turn students into writers. To the extent that I helped my students develop, it was me teaching around my own rubrics, not through them.
"But for students who don't get any of this at all - they don't get feedback and just a letter grade - is there a place for them on their own to use LLMs to make them better writers?"
It's possible that students can use LLMs to help work on their own practices, but in every case I would ask why they are not just showing their work to an audience, any audience and seeing how that audience responds because in the end, we write for readers, not LLMs.
"I think this is the question that many teachers are grappling with, especially those who have over a hundred or more students - how do you realistically assign multiple pieces of writing and provide ideal and sophisticated feedback with that many students?"
This is the focus of the workshops/seminars that I do and while there's a lot of components that I work through in achieving these goals, the big picture is 1. Give students something interesting to do that requires them to draw on their unique intelligence and 2. Move away from traditional grading to a system that puts the onus on students to understand and express what they've learned. I started working this problem before LLMs showed up and can report that progress can be had.
"But it was unique because the reading and conversation and interest in the class overall made up the bulk of the grade - the written work was personal, reflective, and connected to the text - and I only had 9 students!"
I can report from my experience that this scales. I've had this kind of result with 60 student gen ed literature classes. It required me to back off grading as evaluation, but by all reports from students, learning happened. (Not universally, obviously because there's nothing that works for everyone all the time, but on balance, success.)
"I'm going to share this piece with our AI Roundtable that is happening next week. Thanks!"
Thanks you! This is the kind of mutual human exchange (emphasis on the human) that advances our understanding of these challenges
As an adult student of creative working, my professor pretty much exclusively used AI to offer feedback on writing. Once I figured this out, the class became far less interesting. I only stayed for the relationship building with other students as we shared our writing with each other. It would have been lovely to get a well considered critique from someone with more expertise, or honestly, just a recounting of "this was my experience of reading your work". Alas, was not in the cards.
On the broader topic of feedback, I love to recommend Carol Sanford's "No More Feedback". This, along with her later books of "Indirect Work" and "No More Gold Stars" offer a wonderfully contrarian perspective.
However, framing the post around your rubric and what you did with it keeps the focus on the giver of feedback. What matters most is what the receiver does with it.
David Nicol and David Carless are highly recommended. (Feedback research on qualitative student feedback is a fun rabbit hole to go down. I find it interesting that most of the top feedback researchers are not in the US)
When I evaluate the quality of feedback on my own work, the main component is its usefulness. I don’t care whether it’s self, peer, expert, or AI feedback. If the comment helps, I use it. If it doesn’t, I set it aside. And good feedback is much rarer than it should be, a shortage that reaches well beyond the classroom into the rest of life, where most of us reside.
So I’m curious about your opinion on this. For independent and lifelong learners, who often have no access to outside feedback, would you really steer them away from AI feedback?
For most of them, the realistic alternative is no useful external feedback at all. For adults trying to learn, grow, and hold their own in these complex and competitive times, that seems like the wrong path.
There is a difference between developing student learners and adults, so I am not going to tell a person who has their own writing practice that finds some aspect of LLM feedback useful that they shouldn't do it, but I have yet to see an example of this feedback that couldn't have been achieved through self-reflection, rather than soliciting the output of the LLM. I do think that incorporating LLM feedback could have the potential to degrade the capacity for self-reflection or limit the development of that skill.
The only person who has no human alternative for feedback is someone who has no access to other people. Feedback need not be "expert" to be useful. We write to communicate to humans. Find any old human and ask them what they get from a piece of writing and then use that feedback.
Now, if we're talking editorial advice, that may be something which requires some expertise, but this puts me back to my position that since LLMs don't read, I cannot trust that editorial advice. I write for humans, not predictive token generators.
"Maybe *showing we care* is the high calling of any feedback we can give" (my emphasis). What a great thought, not just for teaching, but as a way to be more human in anything we do.
About a year ago, I began experimenting with having AI write a feedback letter to each of my students instead of using a rubric. I read their essay, spoke into AI my thoughts about what was impactful about it and what confused me or didn't land. I spoke about what the mentor texts for the unit did compared to what they achieved. I asked AI to use an encouraging tone in the letter. Sometimes I asked AI to explain a grammar rule with examples. I then asked it to summarize three next steps a student could take for the next essay. The letters it produced were better than I'd have the time for. They were well-received from the students. I asked them to take them out and read again the next steps after their first draft of their next essay. This process helped me realize a vision I had for how I might be able to manage feedback in a way that helped me avoid the procrastination I'd always felt around "grading." My procrastination was driven by a feeling that students needed more than what my feedback and my emotional reserves could produce so I just kept pushing it off and avoiding it. I hid from them that AI wrote these letters until our AI literacy unit this year. Then, when we discussed the use of AI in grading systems, I revealed my process and asked for honest feedback. They (at least told me) that they didn't care that AI wrote it as long as I read their papers, I gave the feedback ideas itself that AI put into an organized form, and that I read over the letters to make sure it was what I really thought. I still feel a little bad about using AI in this way. I'm not sure why...anyone have any ideas? I think it might have something to do with how I tend to only use AI for transactional writing and I am very much in a relationship with students. What do you think? Should I feel bad?
I don't think you should feel bad at all because it's clear your primary priority is student well-being and it's just as clear that you're working the problem with intention and purpose. This is all we could ask of anyone who works in the conditions that must of us labor (or in my case, labored) under. The structural constraints that govern the work of teaching are overwhelming, or at least I often found them that way. Even the phrase "emotional reserves" had me flashing back to periods where I'd be grading on a weekend and looking at the stack of assignments I still had to respond to and feel something like despair. That feedback was hugely compromised in those cases because I was just trying to survive it. I had all kinds of hacks I deployed - like about 30 macros of common comments in MS Word - that made the work manageable, but over time those hacks made me more and more distanced from the work over time.
I didn't feel guilt so much as deep frustration and easing that frustration became the goal. I wasn't going to ever have fewer students or "better prepared" students, so I had to iterate towards what felt meaningful. As motivators go I'll take frustration over guilt any day.
From my perspective of having gone through a journey of questioning before AI existed, I'd asked why this kind of feedback is "transactional." The question I kept asking myself over the numerous semesters of experimentation was essentially, "what if all of this is relational?" In terms of both my relationship with students and my students relationship to writing and expression. This is where my framework of the writing practice came from as I needed to have some way to talk about the writing without focusing on the grade or even what they should do next time. We were going to build a bigger thing than a method for writing a school essay.
The type of feedback I have became much less standard, and requiring students to reflect on their own work was ultimately the game changer. Once I committed to a full alternative grading scheme, everything locked into place.
And then, relatively shortly thereafter it became clear that I couldn't afford to teach anymore and so here we are.
Reading this piece reminded me of teachers who took me seriously. Thank you for those reminders. It also occurs to me that the writing that I did that they called me on was a lot like AI generated writing. That is, I allowed pre-fabricated phrases and conventional thinking to be the sum total of my work. Because it was fluent, and because it was technically correct, most teachers praised it. The few who didn’t taught me to push harder and think better. Thanks for reminding me of those great teachers.
Just asked students this week about the suggestion that teachers use AI feedback to support/expedite the feedback they leave for student writing, and almost unanimously this was the response: "why should we care about what we write if they don't care enough to read it and leave feedback?"
Taking the time to leave feedback is itself a signal of care, not to mention the opportunity it avails (particularly with experience!) to coach/motivate student writers over a school year.
AI threatens all of that—especially if this newest generation of teachers outsource this from the get-go and never develop the craft/capacity.
My first thought was “Perhaps the law professors are not very good at providing feedback” followed by “I did not go to law school, but perhaps there is a formulaic aspect to some learning there that would lead you to believe that formulaic AI feedback was useful” followed by all the descriptions I’ve heard of my father—nearly universally praised as the best teacher any student ever had—handing back a paper he’d ripped to shreds and telling the student to rewrite it—and then doing that over and over again, the whole semester, with just that one paper. And then I think about how few professors or instructor of any kind have the kind of time and freedom my father did to operate in that way.
Having teachers and professors who take the time to give detailed - and the kind of honest evaluation that can only come from the intimate contextual information available through observation (work ethic, classroom interest, questions, etc...) and, therefore, deeply human - feedback on your writing is a gift and a privilege many students do not get - and, if they do, they rarely get it from qualified or expert writers who have reached your level of insight from years of moving through the process you describe - most secondary school teachers are still mired in the rubric system and, even then, have difficulty applying it consistently. One of my colleagues who uses AI regularly in tandem to flag certain aspects of student work justifies his practice, in part, because AI will fairly apply standards across written work that doesn't reflect the frequent bias of teachers pre-disposed towards stronger students. This is certainly a result of the tyranny of the grading system which many of us are stuck in as a realistic constraint of our roles. So, there is no question that what you lay out is clearly the ideal. I still recall my college freshman writing instructor who observed that I was making significant progress halfway through a semester of over a dozen assignments strewn with red ink (and not graded which I think is the hugely and frequently overlooked point in your description) which spurred me on in the class. Just knowing that someone (who was a published author) showed that level of confidence in my improvement as a writer was valuable beyond words that no AI level of feedback could provide. But what about the thousands of students who don't have access to those teachers? Or any meaningful feedback in general? I don't know the answer but I am curious whether you think novice writers - at least those who want to reach a general level of competence (after all, the overwhelming majority of students will not become professional writers though everyone needs to learn to express themselves clearly through the written word, at least in a practical sense) can gain some value from the kind of feedback LLMs can provide. For those rubrics you lay out above, an LLM can offer highly specific and technical feedback well beyond what most teachers can possibly offer. I'm just wondering whether you think any of that can be valuable in tandem with what K.A. Keener explains in her comment. Is there a place for thoughtful teachers to couple the unique capabilities of AI alongside the more valuable and insightful feedback from a caring and careful reader? A teacher who does not - or cannot for reasons of workload or work ethic - read a student's work should not assign it (unless the goal is simply to have the student write for themselves in a private reflection). But for students who don't get any of this at all - they don't get feedback and just a letter grade - is there a place for them on their own to use LLMs to make them better writers? Many of my students have told me they get value from this though it's hard to know whether they are only thinking of that through a grading perspective. I think this is the question that many teachers are grappling with, especially those who have over a hundred or more students - how do you realistically assign multiple pieces of writing and provide ideal and sophisticated feedback with that many students? I loved the Great Books course I taught - and the short papers that came with it - primarily for the reasons you describe here. But it was unique because the reading and conversation and interest in the class overall made up the bulk of the grade - the written work was personal, reflective, and connected to the text - and I only had 9 students! Very, very few teachers have that luxury. Anyway, this was a very thoughtful analysis of what I think is going to continue to be a larger and larger conversation, not only as AI becomes more and more integrated into schools and student's lives, but as we watch the impacts of how much it is already affecting the billions of smaller pieces of business communications all over the world. I'm going to share this piece with our AI Roundtable that is happening next week. Thanks!
So many deep questions with no definitive answers, but let me work through some stuff.
"...a privilege many students do not get - and, if they do, they rarely get it from qualified or expert writers who have reached your level of insight from years of moving through the process you describe - most secondary school teachers are still mired in the rubric system and, even then, have difficulty applying it consistently. "
I think we need to continue to note and examine this missing "privilege," and for my part I'll suggest that one step might be to understand why we consider this to be a privilege as opposed to right. One thing I'd note is that this work does not require qualified, expert writers. I went to excellent public schools, but my teachers were also "ordinary" human beings. They were simply doing their jobs, doing them well (many of them, anyway), but not requiring any kind of special training, experience or effort. I don't know why we should see this as beyond our capacity as a society. Giving up on this as a possibility would be akin to giving up on democratic self-governance, which, maybe we have.
"One of my colleagues who uses AI regularly in tandem to flag certain aspects of student work justifies his practice, in part, because AI will fairly apply standards across written work that doesn't reflect the frequent bias of teachers pre-disposed towards stronger students. This is certainly a result of the tyranny of the grading system which many of us are stuck in as a realistic constraint of our roles. So, there is no question that what you lay out is clearly the ideal."
Couple thoughts. One is that I don't know why "consistency" has become some kind of idea, or rather I do, but this is another value we should interrogate and keep where it works and discard where it doesn't. The feedback we give students on writing should be in the service of helping them develop their writing practices. Given that students will be all over the map in terms of their development, consistency of feedback strikes me as a lesser goal, relative to useful/meaningful feedback to the individual.
For sure a grading system that is designed to rank and sort students inside a schooling context is a structural barrier, but this structure largely was not part of my upbringing (at least not until high school), so it's not as though we don't have experience with thinking differently.
"Just knowing that someone (who was a published author) showed that level of confidence in my improvement as a writer was valuable beyond words that no AI level of feedback could provide."
This is true, but I'd also say that this feedback need not come from a professional writer. It could be any other human, including a classmate/colleague.
"But what about the thousands of students who don't have access to those teachers? Or any meaningful feedback in general?"
This is a question where my friend Jane Rosenzweig's framing of "What problem is the AI solving?" comes in handy because what you're describing is a lack of access to human beings, not a flaw of writing pedagogy. I would prefer we attack that problem. Dave Eggers' 826 writing centers are designed to be helpful in this lack, providing an army of volunteer humans that are accessible to students in afterschool programs. I don't think we lack the human resources, but we may be hampered by the fact that the resources aren't going towards humans.
"I don't know the answer but I am curious whether you think novice writers - at least those who want to reach a general level of competence (after all, the overwhelming majority of students will not become professional writers though everyone needs to learn to express themselves clearly through the written word, at least in a practical sense) can gain some value from the kind of feedback LLMs can provide."
The question here is competence "at what?" Competence at doing school? Clearly LLMs can help. Competence at becoming thinking, problem solving humans with access to their own unique intelligences? I'm skeptical. The question of whether this is worthwhile depends on what you value. Over the years I came to put almost no value in the schooling part of school.
"For those rubrics you lay out above, an LLM can offer highly specific and technical feedback well beyond what most teachers can possibly offer."
Much to my chagrin, considering how many students I subject to those rubrics, they have nothing to offer if the goal is to help turn students into writers. To the extent that I helped my students develop, it was me teaching around my own rubrics, not through them.
"But for students who don't get any of this at all - they don't get feedback and just a letter grade - is there a place for them on their own to use LLMs to make them better writers?"
It's possible that students can use LLMs to help work on their own practices, but in every case I would ask why they are not just showing their work to an audience, any audience and seeing how that audience responds because in the end, we write for readers, not LLMs.
"I think this is the question that many teachers are grappling with, especially those who have over a hundred or more students - how do you realistically assign multiple pieces of writing and provide ideal and sophisticated feedback with that many students?"
This is the focus of the workshops/seminars that I do and while there's a lot of components that I work through in achieving these goals, the big picture is 1. Give students something interesting to do that requires them to draw on their unique intelligence and 2. Move away from traditional grading to a system that puts the onus on students to understand and express what they've learned. I started working this problem before LLMs showed up and can report that progress can be had.
"But it was unique because the reading and conversation and interest in the class overall made up the bulk of the grade - the written work was personal, reflective, and connected to the text - and I only had 9 students!"
I can report from my experience that this scales. I've had this kind of result with 60 student gen ed literature classes. It required me to back off grading as evaluation, but by all reports from students, learning happened. (Not universally, obviously because there's nothing that works for everyone all the time, but on balance, success.)
"I'm going to share this piece with our AI Roundtable that is happening next week. Thanks!"
Thanks you! This is the kind of mutual human exchange (emphasis on the human) that advances our understanding of these challenges
As an adult student of creative working, my professor pretty much exclusively used AI to offer feedback on writing. Once I figured this out, the class became far less interesting. I only stayed for the relationship building with other students as we shared our writing with each other. It would have been lovely to get a well considered critique from someone with more expertise, or honestly, just a recounting of "this was my experience of reading your work". Alas, was not in the cards.
On the broader topic of feedback, I love to recommend Carol Sanford's "No More Feedback". This, along with her later books of "Indirect Work" and "No More Gold Stars" offer a wonderfully contrarian perspective.
Very interesting article, thanks for sharing!
However, framing the post around your rubric and what you did with it keeps the focus on the giver of feedback. What matters most is what the receiver does with it.
David Nicol and David Carless are highly recommended. (Feedback research on qualitative student feedback is a fun rabbit hole to go down. I find it interesting that most of the top feedback researchers are not in the US)
When I evaluate the quality of feedback on my own work, the main component is its usefulness. I don’t care whether it’s self, peer, expert, or AI feedback. If the comment helps, I use it. If it doesn’t, I set it aside. And good feedback is much rarer than it should be, a shortage that reaches well beyond the classroom into the rest of life, where most of us reside.
So I’m curious about your opinion on this. For independent and lifelong learners, who often have no access to outside feedback, would you really steer them away from AI feedback?
For most of them, the realistic alternative is no useful external feedback at all. For adults trying to learn, grow, and hold their own in these complex and competitive times, that seems like the wrong path.
There is a difference between developing student learners and adults, so I am not going to tell a person who has their own writing practice that finds some aspect of LLM feedback useful that they shouldn't do it, but I have yet to see an example of this feedback that couldn't have been achieved through self-reflection, rather than soliciting the output of the LLM. I do think that incorporating LLM feedback could have the potential to degrade the capacity for self-reflection or limit the development of that skill.
The only person who has no human alternative for feedback is someone who has no access to other people. Feedback need not be "expert" to be useful. We write to communicate to humans. Find any old human and ask them what they get from a piece of writing and then use that feedback.
Now, if we're talking editorial advice, that may be something which requires some expertise, but this puts me back to my position that since LLMs don't read, I cannot trust that editorial advice. I write for humans, not predictive token generators.
"Maybe *showing we care* is the high calling of any feedback we can give" (my emphasis). What a great thought, not just for teaching, but as a way to be more human in anything we do.