Large Language Models As The Tales That Are Sung

Gene Wolfe, Albert Lord, machine culture.

[“A Fully Cut Fairy Tale”. Paper cutout by Hans Christian Anderson. From the collection at the Met, and remarkably apt for the post below]

When I wrote some months back about various ways of thinking about AI as a cultural technology, I left one approach out: humanism. This is a loose family of understandings that oppose Large Language Models on the grounds that they replace human culture with something that is machinic and alien. I couldn’t see a good way to reconcile this approach with the others that I discussed, which think about culture in different ways. Leif Weatherby, for example, is sharply critical of what he describes as ‘remainder humanism,’ which he sees as ignoring the large systems, most notably including human language, that actually produce culture.

I’m intellectually sympathetic to Weatherby’s arguments; but I’m also sympathetic to certain flavors of humanism, and have been thinking about how to put the two in conversation with each other. Writing a piece that tries to do this has taken a while. These are not the kinds of argument I specialize in as a scholar. The weird intellectual path I’ve chosen leads through the science fiction of Gene Wolfe and the folklore studies of Albert B. Lord. I also take liberties with the ideas of friends and others, who bear no responsibility for any intellectual abominations that I commit. Caveat lector.

Both Wolfe and Lord died before LLMs became a thing. Nonetheless, both thought about the relationship between the immensities of language and human culture and the more particular desires of human beings to tell their own stories. I read their ideas as as implying that LLMs have much in common with the long existing traditions that storytellers draw upon. It is not simply that the compressions that LLMs derive from enormous corpora of text resemble, and arguably incorporate, the characters, stock phrases, tropes, situations and narrative structures that storytellers weave together. It is that LLMs and storytelling traditions are similar structures - large scale bodies of generic cultural knowledge that are drawn upon in somewhat predictable ways to create specific cultural instances.

These are not original claims on my part. Their proximate beginnings lie in an emailed side-remark by Cosma Shalizi about LLMs and The Singer of Tales, and his extended discussion of LLMs here. Kush R. Varshney independently struck upon the same comparison in this paper, which investigates the relationship between LLMs and Lord’s ideas from an engineering perspective. [Update: also Bill Benzon, David Smith, and, I suspect, others too]. I’m also influenced in ways it would be hard to set out by the work of my grand-aunt, Maire MacNeill. She was a folklorist who died too early for me to know well, but her writing and translations shaped my childhood; The Festival of Lughnasa, fairy tales collected from the oral tradition in West Donegal; the book of the storyteller, Sean O’Conaill, who lived on the same peninsula that John McCarthy’s people came from. More even than most children, I grew up amidst the cultural structures of folklore, without ever really thinking about what they involved.

LLMs resemble folklore in structure, but differ in their relationship to intentionality and performance. In one sense, stories are assemblages of tropes about third sons, enchanted rings, animal helpers and the like. In another, which is equally important, stories don’t exist until they are spoken and are products of the circumstances of their speaking. Their meaning doesn’t just reflect a shared tradition, but the particular circumstances under which they are told: an individual human speaking to other humans, snipping or stretching out the cloth of the tale as it is woven, to suit the desires, expectations and responses of his or her audience. In this, stories differ from LLMs, which instantiate the tradition itself, in its indifferent and inhuman nakedness. LLMs are not the singer, despite their apparent responsiveness, but the structural relations of the tales that are sung. Still, we can now listen to and even interrogate those structures without immediate human intermediation. That is new.

*******

Gene Wolfe is one of the great science fiction writers, though he is little known outside the genre. Ted Chiang cites him as an important influence. Ursula Le Guin compares him to Melville, who, like Wolfe, freighted adventure stories with theological and metaphysical speculation. Kim Stanley Robinson likens him to Proust, whose prose style and sense of time as a lucid dream was put by Wolfe to his own purposes. Wolfe has had little influence on debates over AI, unlike some of his lesser peers. Perhaps that should change.

Wolfe was an engineer, but his great love was for stories. As Severian, the imagined narrator of his great work, the five “New Sun” books, remarks after a story-telling competition:

it often seems to me that of all the good things in the world, the only ones humanity can claim for itself are stories and music; the rest, mercy, beauty, sleep, clean water and hot food (as the Ascian would have said) are all the work of the Increate. Thus, stories are small things indeed in the scheme of the universe, but it is hard not to love best what is our own—hard for me, at least.

From this story, though it was the shortest and the most simple too of all those I have recorded in this book, I feel that I learned several things of some importance. First of all, how much of our speech, which we think freshly minted in our own mouths, consists of set locutions. The Ascian seemed to speak only in sentences he had learned by rote, though until he used each for the first time we had never heard them. Foila seemed to speak as women commonly do, and if I had been asked whether she employed such tags, I would have said that she did not—but how often one might have predicted the ends of her sentences from their beginnings.

I’ll return to “this story” (told by an Ascian prisoner) below. For the moment, just note the tension that Wolfe identifies. Severian (presumably speaking for Wolfe) describes stories as something that is uniquely “ours,” uniquely human. Yet he also notes that they are composed of language that is predictable, a commingling of stock parts or “set locutions,” as is our ordinary language itself. Those locutions are joined together into sentences that are highly redundant.

As for language, so too for the tales we tell with it. They too are regularly built from stock parts, as Propp and others have argued. In an earlier book in the New Sun series, a child asks Severian to “find a story with a boy in it who has a big friend and a twin. There should be wolfs in it.” Boys, animal friends and twins appear regularly in folk tales, together with many other stock elements. One consequence of this is that folk tales can be recombined more readily than other narrative forms such as novels or written poems; they are mishmash from the beginning. The story that Severian tells the boy commingles The Jungle Book with the rescue of infant Moses, the rivalry of Romulus and Remus (who, like Mowgli, were suckled by wolves), and the founding of colonial America. In Severian’s far future earth, our stories and our history have commingled to become part of the common stock of collective human wisdom.

Misprision becomes a source of new variations. Another story recounted by Severian confounds the hero Theseus with a graduate student’s thesis, while the Minotaur and the steamship Monitor are merged into a fearsome giant with a “naviscaput,” roaming a maze of silted channels purpose-built to exhaust his prey. Wolfe resembles Joyce as well as Proust and Melville in his delight in wordplay and sometimes terrible puns.*

Yet this play is not completely open-ended. It builds on and from a system. Wolfe’s remark that “how often one might have predicted the ends of [Foila’s] sentences from their beginnings” is strongly reminiscent of the ideas of Claude Shannon. Such predictability is the core insight of Shannon’s predictive account of language, which in turn was the intellectual starting point for the development of Large Language Models. A “statistical model of language,” which makes the open-endedness of language (its particular version of the ‘curse of dimensionality’) tractable, combined with the particular affordances of the transformer architecture, powers the next-token prediction of LLMs.

Wolfe had great love for stories, but he was also an engineer. I suspect that the resemblance between Severian’s critique and Shannon’s ideas is the product of deliberate artifice. There’s no explicit evidence I’m aware of that Wolfe read Shannon (perhaps such can be found somewhere in his technical essays for Plant Engineering, which have never been collected), but elsewhere in the books Severian presents a theory of the knowledge of magicians that is based on signaling theory and lossy transmission.

Yet Severian does not love stories because they are a statistical extrapolation of underlying structures in language and culture. He loves them because they are human. So how do human stories emerge from vast systems, without being reducible to them?

*******

That question is explored in another extraordinary book, Albert B. Lord’s The Singer of Tales (available for free online). Lord built on the work of Milman Parry, who he had helped to record Yugoslavian singers of tales in the mid-1930s. From this experience and archive, Lord constructed a broader account of the oral tradition, which he believed encompassed Homer and other singers of epic histories. By extension, his argument also covers other forms of the oral tradition, such as the Irish fairy tales that my grand-aunt and others translated, compiled and sought to understand.

Parry and Lord listened to singers, whose tales described great heroes, villains and battles of the past (unsurprisingly, given the asperities of Yugoslav history, Christian and Muslim singers disagreed about who were the heroes and who the villains, though Christian singers could calibrate their stories for Muslim audiences and vice versa). The anthropologists recorded these songs on crude discs. These songs were the last whispers of a tradition passed down by illiterate singers and story tellers (although when Parry and Lord heard them, their singers were already beginning to be influenced by written culture). Some of the songs were very long, but accomplished singers could sing very many of them. How, given the faults and frailties of human memory, did they manage this?

The answer, according to Lord, lay not in rote memorization, but the nature of the oral tradition that they drew upon. While the singers themselves insisted that they could reproduce the songs perfectly and without error, their sense of perfection differed from ours. Neither the epic songs, nor the broader tradition they drew upon were unchanging texts. Instead, the specific epics they sang, and the epic form itself was better understood as a kind of “dynamic structure” that could generate new cultural instances.

This structure was a kind of ‘grammar,’ a set of rules that determined not simply the meter, but how the story flowed, which themes were emphasized and which suppressed, and so on.

In studying the patterns and systems of oral narrative verse we are in reality observing the “grammar” of the poetry, a grammar superimposed, as it were, on the grammar of the language concerned. Or, to alter the image, we find a special grammar within the grammar of the language, necessitated by the versification.

Singers constructed their tales around set “themes” or situations, such as the “council meeting” in which the possibility of battle is discussed. To describe these themes, they drew on a “common stock” of formulas they then deployed to reconcile characters, the style of the story, the expectations of the audience, and the meter of the song. Different singers had slightly different repertories of formulas, which might be greater or smaller, depending.

The singer never stops in the process of accumulating, recombining, and remodeling formulas and themes, thus perfecting his singing and enriching his art.

Singers might come up with their own variations, which could in turn be copied by others. All this perpetuated a tradition that was continually recreated in its own performance. In the oral tradition, epics are not fixed texts, but generative systems of songs, endlessly articulated, re-articulated and adapted in varying forms, according to the particularities of the singer and the circumstances they sing in. As Lord puts it: “we cannot correctly speak of a “variant,” since there is no original to be varied.”

As Cosma has pointed out, there is a remarkable resemblance between LLMs and Lord’s depiction of the oral tradition. That is because LLMs, like Lord’s singers, have mastered the tropes from which tales are made, but on the level of written culture itself rather than any narrative form or genre therein.

A huge amount of cultural and especially intellectual tradition consists of formulas, templates, conventions, and indeed tropes and stereotypes. To some extent this is to reduce the cognitive burden on creators: this has been extensively studied for oral culture, such as oral epics. …

The formulas make things easier to create and to comprehend once you have learned the formulas. The ordinary way of doing so is to immerse yourself in artifacts of the tradition until the formulas begin to seep in, and to try your hand at making such artifacts yourself, ideally under the supervision of someone who already has grasped the tradition. (The point of those efforts was not really to have the artifacts, but to internalize the forms.) Many of the formulas are not articulated consciously, even by those who are deeply immersed in the tradition.

Large models have learned nearly all of the formulas, templates, tropes and stereotypes. (They’re probability models of text sequences, after all.) To use Barzun’s distinction, they will not put creative intelligence on tap, but rather stored and accumulated intellect. If they succeed in making people smarter, it will be by giving them access to the external forms of a myriad traditions.

There is a lot of speculation that LLMs are returning us to something like oral culture. There is rather less that engages in any very intelligent way with the particulars of how oral culture works. Oral culture, like LLMs, involves lossy abstractions that also serve as generative systems. It too produces myriads of variations on common themes, adapting them to particular prompts and circumstances. It too is indifferently geared for verbatim transmission of the work on which it has been trained. When Varshney describes Lord’s formulas as “heuristic solutions to constrained optimization problems that must be solved in real-time,” he is using language that Lord might perhaps have found peculiar (though also perhaps not; Jakobson was on his dissertation committee), but that Wolfe would readily have recognized.

Again, it is a mistake to treat either oral epics or LLMs as fixed texts. From an analytic perspective, they are systems for producing particular cultural forms, which adhere to particular rules, stereotypes and expectations. They can be expected to be lossy in loosely similar ways. Lord notes that singers employ the “principle of thrift.” Once they have discovered a good technique for solving a particular class of textual problem (e.g. reconciling a particular kind of description with the expected meter), they will deploy it again and again. LLMs have their own principle of thrift, so that their compressions emphasize commonly encountered cultural patterns to the detriment of those encountered rarely in their training data.

As it turns out, LLMs are not very good at the rhyme and meter of language (perhaps since they are trained on written text rather than its heard performance).** But many aspects of human work and culture involve broadly similar combination of templates and stereotypes to those employed by the singers of tales. I suspect that this helps explain the facility of LLMs in carrying out many programming tasks, since programming too involves figuring out how to apply a common formula to a particular problem. The poiesis of the programmer is closer to the heroic poiesis of the bard than we think. As one of my old XKCD t-shirts puts it: stand back - I know regular expressions! So too for performance in math olympiads.

And perhaps also for much of the practice of social science? Dani Rodrik has written that a great deal of the art of the economist consists in accumulating a large mental library of mathematical models, and building an intuitive grasp of which model one ought to use when. Equally, there is reason to suspect that there are sharp limits to the capacities of LLMs to apply their compressions and transformations to unexpected challenges, and to aspects of the physical or social environment that don’t translate easily into spreadsheet form.

The fundamental point, then, is that a lot of human culture and endeavor does not just depend on lossy compressions, but involves dynamic systems that combine and apply these compressions in useful ways. These combinations may be surprising (involving unexpected juxtapositions), although they will likely have limited originality. Equally, originality may be unnecessary for many tasks, and even over-rated: Lord hints that our desire for artistic originality may be related to the anxieties of print and the other technologies of accurate production that have overtaken oral culture:

Expression is [the business of the singer], not originality, which, indeed, is a concept quite foreign to him and one that he would avoid, if he understood it. … There are periods and styles in which originality is not at a premium

Perhaps we might expect that in the near future we will return to some form of those norms of expression, to the extent that LLMs become culturally dominant. Perhaps instead, we ought anticipate a cultural reaction against them, doubling down on originality.

Whatever happens, Lord suggests that human culture does not have to be original to be human in the ways that writers such as Wolfe care about. Structural systems such as epics can produce possibilities for human meaning. As he says, “the style is not really so mechanical as its systematization seems to imply.” The tales would not work if they were mere jumbles of expected cliches.

Most importantly, stories are performed in settings that emphasize the relationship between the singer and the audience. When the singer sings, “the song produced in performance is his own. The audience knows it as his because he is before them.” Lord describes how the singer (apparently always male in 1930s Yugoslavia, though of course women are storytellers too) adapts his performance of the story to the audience and what they seem to want. If the audience appears to be growing restive, the tale may be changed, or truncated. While the tale draws on structure, it is performed in the moment, and in the context of human intentions and human relationships shared by singer and listener.

That, then, is an important - and from some perspectives crucial - difference between traditional generative cultural systems such as the oral epic tradition in Yugoslavia and its Homeric ancestor and cousin, and algorithmically generative cultural systems such as LLMs. The former are inseparable from their performance in human contexts: the system can be abstracted from their performances, but does not have any substantial being independent of them. The tradition’s existence is manifested through the stories that humans tell each other. LLMs, in contrast, are a condensation and actualization of the tradition itself and all the other traditions that have been folded and compressed into its statistical weights, stripped of the specifics of human relationships and reapplied to them as an algorithmic process. LLMs speak only when prompted, but their continuations of those prompts are not expressed through two-way human relationships (though the words used by the prompter, training via RLHF and fine tuning etc obviously affect the outputs). The person who prompts the LLM is interrogating a lossy representation of the tradition itself. Of course, LLMs may then shape human understandings, and fool people into treating them as human, but they are not. Their tangible abstraction is something that is novel and different in its application than the previously intangible cultural knowledge that they summarize.

LLMs, then, can reasonably be understood as a summarized composite of the tales that are sung. In their particular applications, they are even a version of the singing of the tales. But they are not, and cannot be, the singers. Our relationship to them, and our interpretation of what they tell us is necessarily different from our relationship with and interpretation of what we tell each other.

*******

To understand this, it’s helpful to return to Wolfe’s Severian. As I’ve already noted, Severian makes his remarks about stories in the context of a story telling competition. He has fallen ill, and is stuck in a lazaret with soldiers recovering from their wounds, where there are few other ways to distract from current circumstances than to talk. Two of the other inmates are vying for the hand of a third, Foila, who has said that she will marry whoever can tell the best story. This framing story itself partakes of the logic of the folk tale, but it is also a space for the play of agency. Foila changes the rules of the competition twice to suit herself; once to allow a third inmate, an Ascian prisoner, to join the contest; then to enter into the competition on her own behalf by telling her own story. For after all:

Even a man who courts a maid thinking he has no rivals has one, and that one is herself. She may give herself to him, but she may also choose to keep herself for herself. He has to convince her that she will be happier with him than by herself, and though men convince maids of that often, it isn’t often true.

Severian wonders whether the Ascian prisoner too has picked his story to convey a hidden message, and why Foila had allowed him to participate at all. None of this is obvious - the competition is a maze of ambiguous intentions.

I had not learned those things I had most wished to learn as I listened to the Ascian and to Foila. What had been her motive in agreeing to allow the Ascian to compete? Mere mischief? From her laughing eyes I could easily believe it. Was she perhaps in truth attracted to him? I found that more difficult to credit, but it was surely not impossible. Who has not seen women attracted to men lacking every attractive quality? … And what of him? Melito and Hallvard had accused each other of telling tales with an ulterior purpose. Had he done so as well? If he had, it had surely been to tell Foila—and the rest of us too—that he would never give up.

Stories can convey messages, but those messages are likely (except in the most debased and simple tales) to be partly obscured, and sometimes more obscure even than that.

The Ascian’s story is constructed as a most difficult case for individual meaning. He comes from a future North America that seeks deliberately to stamp out individuality in favor of imposed culture. In Ascia, people apart from children can only speak by reciting rote phrases, taken from a small set of approved texts that they have memorized. Hence, the story that the prisoner tells is a sequence of a couple of dozen of these phrases, from which the careful and informed listener can construct a story.

On its face, this seems a fantastical version of Orwell’s Newspeak, a victory for the imposition of the unchanging printed phrase, and the antithesis of the dynamic oral traditions from which new usages and conjunctions may appear. Yet even in this deliberately cramped and pinched linguistic system, ambiguity of intention can seep from the cracks, interstices and implied spaces of imposed tradition. The tale cannot fully be confined within the artificial prison house of language.

I learned once again what a many-sided thing is the telling of any tale. None, surely, could be plainer than the Ascian’s, yet what did it mean? Was it intended to praise the Group of Seventeen? The mere terror of their name had routed the evildoers. Was it intended to condemn them? They had heard the complaints of the just man, and yet they had done nothing for him beyond giving him their verbal support. There had been no indication they would ever do more.

Severian does not grasp the intention of the tale, nor the Ascian’s intention in its telling. Yet his belief that there is some intentionality to both the tale and teller, however murky or unclear - perhaps unclear even to the Ascian himself - is what makes the story human. Telling a story is a human speech act, through which one human looks deliberately to communicate with others. The selection of certain rote phrases rather than others combines with the circumstances of their utterance to convey meaning and ambiguity, both so entangled as to be impossible to separate. As Wolfe says elsewhere, the speaking of any word is futile unless there are other words, words that are not spoken. The meaning behind the choice of some words or stock phrases, certain tropes rather than others is hard to grasp, yet we struggle to understand this meaning, because there has been some choice that matters to us. We are more than the bearers of structure and ideology.

It is this that distinguishes human stories from the productions of LLMs. On some dimensions, LLMs are much more open-ended than the little red books of the Group of Seventeen. Indeed, as Varshney points out, LLMs are not in fact well captured by the common stock phrase “stochastic parrots,” since they can come up with novel combinations. Even back in 2023, they could do weird and unexpected things, such as inventing cod-Latin phrases, as I discovered when I prompted one to rewrite the plot of Hamlet in the style of a chemistry textbook.

However, they’re definitionally incapable of the kind of ambiguity that the Ascian prisoner can achieve through the mere combination of rote catchphrases. There is no valid room for wondering about LLMs’ motives for saying things, since they don’t have any motives to wonder about. As Weatherby argues, LLMs have “heat-maps” of summarized correlations between themes, tropes and words instead of intentions.

As I noted at the beginning, Weatherby dismisses “remainder humanism,” ideological efforts to distinguish human culture from the systems that perpetuate it. However, he still acknowledges the distinction between poetry - a system of conventions and usages, and the individual sonnet that “unites poetry with an intention.” Even if LLMs are made out of poetry, they are incapable of producing poems. Or in Wolfe’s language, both the epic form and LLMs are story, but are incapable of telling stories. That requires the marriage of structure and intention that human mediation provides. LLMs are a kind of composite of the singing of tales, but are not singers, even if we sometimes misconstrue them as such.

*******

Thinking about stories - and perhaps art more generally - in this way does not lead to any very emphatic conclusions about the relationship between humanistic values and vast impersonal structures. At most, I think, it provides a common space where people concerned with the one can more easily talk to the other, and where the spaces of agreement and disagreement become more visible. And perhaps not even that: I’m venturing well outside my own areas of expertise, and may have blundered. Even if so, I still believe that humanism is not incompatible with the acknowledgement of large structure, nor structuralism with acknowledging the importance of human intentions.

This leaves open the question of whether LLMs and their cousins (e.g. diffusion models) can produce art. Does art necessarily involve human intention? Perhaps not necessarily. I suspect that as artists begin to use these tools, many of the effects they produce will rely on what Mark Fisher calls the “eerie” - the sense that agency is missing where it ought to be, and that something strange has crept in to fill the void. Others may dispute whether such productions are, in fact art, or something different. Others still may argue that even if AI is at a remove from human intentions, it is still human made. And there will be all sorts of hybrid productions.

I’m not the right person to mediate such disagreements, let alone resolve them, but I would like to see them better and more sharply articulated. Technologies such as LLMs are neither going to transcend humanity as the holdouts on one side still hope, nor disappear, as other holdouts might like. We’re going to have to figure out ways to talk about them better and more clearly.

* One joke that I have not seen picked up anywhere involves the “Cumaean,” who, in Wolfe’s version is a serpentine alien, covered in scales that resemble human faces. Naturally, Wolfe never uses the word “pythoness” in connection to the Cumaean or otherwise, leaving the discovery of the pun as an exercise for the reader.

** Casual experimentation suggests that the most recent version of Claude Opus has gotten quite good at reproducing iambic pentameter (a meter that I’ve chosen for the semi-arbitrary reason that a few of Wolfe’s aliens speak exclusively in it). Ironically, Claude Sonnet cannot. Nor can the most recent version of GPT.