Can Humans Determine the Quality of AI Poetry Without Bias?
~ also, poemettes & Taylor Swift ~
It might be difficult, if not impossible.
‘AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably’ (Nature)
“In short, it appears that the “more human than human” phenomenon in poetry is caused by a misinterpretation of readers’ own preferences. Non-expert poetry readers expect to like human-authored poems more than they like AI-generated poems. But in fact, they find the AI-generated poems easier to interpret; they can more easily understand images, themes, and emotions in the AI-generated poetry than they can in the more complex poetry of human poets. They therefore prefer these poems, and misinterpret their own preference as evidence of human authorship. This is partly a result of real differences between AI-generated poems and human-written poems, but it is also partly a result of a mismatch between readers’ expectations and reality. Our participants do not expect AI to be capable of producing poems that they like at least as much as they like human-written poetry; our results suggest that this expectation is mistaken.”
*
Jen Benka dives into the weeds for the LitHub article, ‘On the Report of Poetry’s Death, or: What Does That AI Poetry Study Really Tell Us?’
“Oddly, the poet Dorothea Lasky, the only living poet whose poems are included in the experiment, does not even appear on the site Porter and Machery name as their only source for poems. (In an interview with The Washington Post about the study, an upbeat Lasky welcomed the “robot poets.”)”
“Participants rated the bot’s texts higher than poems authored by poets based on these characteristics. However, it should be noted that only rhyme is a specific and distinct formal element, making it easily recognizable. Rhyme, therefore, as in the previous experiment, could have influenced results favorably in the direction of ChatGPT.”
“This sounds less like an investigation into poetry and more into communication preferences. It also reveals a questionable assumption about poetry.”
Richard Hugo would like to remind us, “I caution against communication because once language exists only to convey information, it is dying.” (The Triggering Town)
*
Hand-holding
This is important.
The amount of hand-holding in poems has always been a careful balance. This, too, goes for tricky aspects in poetics such as walking a fine line without falling off the cliff into sentimentality.
Poems are not meant to be Hallmark cards.
Poems are not meant to be too easy.
Poems, typically, require inroads. Good poems, at least. If a reader cannot find an access point, then what is the purpose of the poem?
The issue of “soul” has been coming up a lot lately. Not just the soul of America. The soul in a poem. The authentic voice of the poet— that is what I think we’re circling.
I keep thinking “to err is human” and how this is becoming an essential aspect of all Art in the conversation with AI. Humans are, by definition, imperfect and fallible. This is why we are, well, interesting. Perfection is not human.
It used to be somewhat possible to have read all conceivable “great works” of The Western Canon. Harold Bloom could tell you all about it. Now, we could simply make an AI program based on the knowledge possessed by Harold Bloom and produce something akin to Bloomisms as criticism on contemporary literature. Do we need this? No.
But we do need contemporary critics. And we want them to be human. Opinions don’t really matter if you have all the information. I know, that probably doesn’t sound logical. What I mean is that the holes in our knowledge base are actually the key to our ability to form interesting perspectives. We want to avoid spreading misinformation and disinformation; however, we cannot presume to hold thoughtful opinions about Art (or other matters) without a semi-expert level of knowledge… but, again, not all available knowledge. If you have all knowledge, you would not formulate a particularly interesting opinion. Think about it.
*
Timothy Green (Editor of Rattle) pointed out on The Poetry Space_ that part of the reason a large language model (LLM) can even write halfway decent poetry is because it has, essentially, “read” a lot of poems—unlike mediocre poets.
You may remember a time when you, or a poet-friend, worried about the anxiety of influence. So much so that they dissuaded themselves from reading any other poetry for fear that it would somehow damage their unique voice. This is…naïve. It’s a fantasy. It’s just not how it works. You only get better by reading.
Another problem is that poets may not have read contemporary poetry. If you ask someone who does not regularly read poetry about poets they have read or can name check, they will maybe mention Whitman, Frost, Dickinson or Plath, Yeats or Ginsberg… maybe a handful of other long-time well-known folks. If they’ve read anyone contemporary, they may have been exposed to inaugural speeches such as those by Richard Blanco and Amanda Gorman. They may also have encountered Mary Oliver’s poetry or Billy Collin’s poetry by way of The New Yorker or quoted in the context of an article they read.
A longtime “problem” with poetry is that it’s been niche. But, it’s also a positive. It’s safeguarded poetry from being a microcosm of the larger culture. That was before. Since Instapoets like Rupi Kaur came on the scene, poetry entered the mainstream sideways via digestible bite-size poemettes— a cutesy term for something poetry-like that is mostly skin-deep. Instagram poetry, at best, has been seen as a gateway to getting people interested in reading more literary poetics. It probably has not worked much.
Given the title of Taylor’s most recent album, The Tortured Poets Department, and even well before then, it was clear that Taylor integrates literary and poetic devices into her songwriting. I decided a while back that it was likely Taylor Swift will go down as the millennial generation’s Bob Dylan and, probably (read: deservedly) win a Nobel Prize in Poetry in the future.
There’s a new poetry anthology out, edited by Kristie Frederick Daugherty, that tasked poets with writing ekphrastic work in conversation with a Taylor Swift song. This is a brilliant idea, frankly, to get everyday folks to consider reading weightier poetry. In the opening reading for the anthology, a contributor called attention to the fact that Swifties are notoriously close readers who love to search for “easter eggs” (clever, hidden allusions) in Taylor Swift lyrics. These are the perfect non-poets to attempt to draw into the world of poetry.
_ Originally drafted 1.4.25 _
_ Slightly revised Feb2025 _
Daugherty's book has an amazing group of poets responding to Swift... inspiring to see this kind of collaboration.
This is such a good read. Have been thinking that it's the poets who are teaching language-learning AI to question itself (the poets and the people who write in that odd science-speak found in research papers; the poets and the science speakers and the second language learners; the poets and the science speakers and the second language learners and the speakers of native tongues cut off from advertising-infected interaction; the poets and). Writing poetry feels like sitting in wait. It feels like being readied to offer an unexpected pathway through a subject. To reach out with language and feel the brambles. The way you slide down a wet bank when too close to the creek. The wet boot. And its seems as though the unexpected characterizes the moments humans break through to solve the biggest problems. AI itself can be seen as a huge problem. At the moment, anyway. It eats energy like crazy. It's hackable. It's way too easily weaponized. Now that some forms have taken on language, the weaponization is even clearer to those of us who haven't thought much about it until now. I don't know. But maybe we can take heart in the fact that humans have had language for a very long time and here we are still. Despite all the wars. All the struggle. I don't know. But this is such a good read. Thank you.