Truth and consequences

Last week, Emanuel Maiberg of 404 Media reported on the ghost kitchens that use AI-generated images on GrubHub and DoorDash to represent the food you might order. These images are superficially plausible but have the usual uncanny anomalies upon closer inspection:

There’s a Shrimp Scampi featuring a shrimp with two tails, a Breaded Shrimp Parmigiana Pasta featuring unidentifiable breaded objects (no shrimp), and some Frutti Di Mare (Seafood) with Marinara Sauce that has some identifiable clams and calamari, but also a lot of creatures that look like they came from the mind of H.R. Giger.

In noting that such images are “misleading,” the article seems to suggest DoorDash users deserve “real” images of “real” food. But no one uses DoorDash because they are invested in integrity. I don’t suppose anyone ordering from theses sites has been persuaded or tricked by these kinds of images; I assume the images are clearly understood to be placeholders fulfilling the requirements of the templates the delivery sites impose. The images are not there to make you think that anyone involved with the product cares about how it is made or perceived or what anyone will think about it. They are not there to heighten your anticipation of the food or make you believe anyone will envy your consuming it. Instead, they are there to signify that a rock-bottom sort of transaction is about to take place where quality and reputation won’t be admitted as relevant.

There is a certain homology between “food I seem to download instantly from an app” and “images instantly generated from text prompts”: The low-effort fakes convey the impression that no one has wasted any time trying to dignify the exchange that is about to transpire, and no one has put in the kind of labor into any aspect of it that would require you to pay a premium. Instead, the consumer knows they are getting a product that has been made as cheap as possible through deskilling and other forms of labor coercion, corner-cutting, and system-gaming.

Like the random-letter-scramble brand names adopted for knockoff products on Amazon, these images ooze indifference, mirroring the customer’s de facto indifference to the conditions under which the products are produced and delivered. As Nathan Jurgenson put it when he referred this article to me, “It would be beside the point for the image to be accurate when you’re trying to buy crappy food, unethically made and unethically brought your door.” That is, the AI-generated images are entirely accurate in this context, not in how they represent the food but in how they represent the business model and the sorts of customers it cultivates. When I use these services, I am to human consideration what the generated images are to actual sandwiches.

The process of simulation is often meant to disappear seamlessly, to conceal the “inauthenticity” of representation and persuade by passing as a true version of what is being simulated, whether that is the content itself or the process of making it. But some of the most effective generated images make the fact of simulation more explicit. They depict simulation itself more than whatever content they are purportedly simulating.

Such images are not trying to deceive — we are not supposed to seriously entertain the idea that the CGI sandwiches are real anymore than we are supposed to judge special effects in movies by their realism alone. What they convey instead is reality’s futile resistance to technology. They don’t trick us into believing in something untrue; they let us identify vicariously with an overriding power to shape reality on whim. It calls to mind Karl Rove’s assertion about American unilateralism during the George W. Bush administration: “We're an empire now, and when we act, we create our own reality. And while you're studying that reality—judiciously, as you will—we'll act again, creating other new realities, which you can study too, and that's how things will sort out.”

The tech companies are the imperialists now, and generative images convey their ability to create realities others must accept, regardless of their flimsiness or shoddiness. A chatbot’s gibberish is of world historical significance because the most valuable companies in the world say so. And rather than waste our time “judiciously studying” what they are doing as though it were some complex historical process rather than a reflex of caprice and greed, we might find more gratifying to simply accept their authority, as though that would allow us to participate in it and harness it for ourselves. Using generative models and embracing their outputs as having some kind of pertinence or validity aligns us with contemporary history’s apparent victors. This can be more rewarding or at least more comforting than trying to speak with one’s own voice or contend over ground truths.

To put this idea another way, people may look to generative images not for an accurate representation of truth or a particular prompt but for a visible manifestation of power. The recent anxiety over a generated image showing Trump arm in arm with black supporters speaks to this. Journalists have been quick to report the fakeness of this image, which, as Joan Donovan pointed out in a comment for this Washington Post article, served only to give the stunt more attention. (There are no doubt many documentary photos of Trump posing with Black supporters.) The generated image, like any other meme, was meant to circulate, and the extent of its circulation is the main fact about it that matters. The image (like any other generated image, in a sense) depicts an enthralling ability to dictate a reality and try to attract attention to it. Its meaning is not validated by reference to the “reality” of what it represents; it is established by the networks of circulation it can activate and the ways people talk about it.

That is to say, the “truth” of a generated image is not in what it shows (everything it shows is synthetic) but in how it is used. This makes it a bit preposterous when, as Wired reports here, “generative AI companies say they’ve put policies in place to prevent their image-creating tools from being used to spread election-related disinformation.” The companies can’t control how people use images or circulate them; they can’t stop innocent-seeming images from taking on nefarious connotations. All they can do is make half-baked gestures toward content moderation and bask in all the reiterations of how powerful their models are, that they stimulate such concern. To report that “DreamStudio prohibits generating misleading content” is just to repeat the company’s nonsense: All generative content is fundamentally “misleading” when the models are construed as fact generators. There are no stable standards for what is “misleading” for whom and under which circumstances. You may as well state that “DreamStudio prohibits using your imagination.”

Researcher Callum Hood, quoted in the Washington Post piece, argues that generative models have “radically lowered the cost of time, effort and skill that’s needed to create convincing fake images,” which seems like an incontestable view. But political disinformation hasn’t been bottlenecked by a lack of material. Anything at all can mischaracterized or recontextualized to support a particular narrative, especially when a targeted audience is already inclined to believe it. Another expert cited in the article, Nina Jankowicz, offers the reassuring point that “common sense is still a really great deepfake detector,” which is a bit like saying, “Trust your imaginary relation to the real conditions of your existence. Ideology’s never been wrong.”

The challenge in disinformation is not in producing misleading evidence but in gaining access to credible distribution channels. This was once a matter of controlling “the media” (getting credulous journalists to circulate your spin), but it now also includes deploying influencers and gaming algorithms. But none of these strategies are hampered in any way by flimsy guardrails on an AI model. Imposing content moderation at the level of the generative tools themselves is beside the point; the only meaningful moderation occurs at the level of distribution. And social media platforms have largely given up on that.

Fakes like the Trump one often circulate not as documentary evidence but as ad hoc political cartoons among already convinced partisans. They serve as accurate representations of those partisans’ sentiments; they circulate as a form of vernacular political expression. Even if platforms wanted to moderate the circulation of this kind of content — as with Instagram’s nonsensical “no politics” policy, as if any kind of communication could ever avoid expressing a politics, conveying values and norms and aspirations — they can’t do it effectively without enforcing their own politicized version of reality (what conservatives and fascists never tire of complaining about) or shutting themselves down completely (which is not an altogether bad idea).

As with previous outbreaks of fact-checking fetishism, pointing out a “fake” seems to promise that there is still an accessible common ground of truth, an objectivity that the legitimate press can occupy. Conjuring fears of technologically abetted political disinformation helps establish the fiction that there are aspects of the political process that are not propagandistic, that there can be neutral presentations of political concerns and that there are “true” political images rather than a series of deliberately staged constructs and endless posturing. The idea that AI-generated images are a special kind of fakery we should be extra-specially concerned about helps further normalize all the rest of the ritualized fakery, including the press kabuki and kayfabe, that is already standard practice. (I could continue this diatribe, but I could also just refer you to what I was writing about “fake news” in 2016 or political deepfakes in 2019.)

The concern over fakes is sometimes framed as though there are naive voters out there looking for honest information in all the wrong places who end up being persuaded against their better judgment to cast a vote for candidates they otherwise would have rejected. But this is a bit like saying the DoorDash customer would have ordered healthier food if the AI-generated image hadn’t steered them toward something impossible. A delivery site is not a place where information is innocent or objective; neither are the places where people engage with politics.

Often, complaints about disinformation end with calls for more information literacy, and advice about verifying sources and that sort of thing. (Do you know CRAAP when you see it?) But this advice tends to conflict with other commonsense postures that value “evidence” for being self-evident, that treat words and images as having fixed and obvious meanings, that posit an objective world of things that is readily and unproblematically available to our senses — in general the idea that truthfulness, reality, corresponds somehow to its degree of obviousness. It is as simple and straightforward as “I think, therefore I am.”

In an interview published in the 1983 afterward of Paul Rabinow and Hubert Dreyfus’s book Michel Foucault: Beyond Structuralism and Hermeneutics, Foucault situates that attitude toward truth historically:

In European culture up to the 16th century the problem remains “What is the work which I must effect upon myself so as to be capable and worthy of acceding to the truth?” To put it another way: Truth always has a price; no access to truth without ascesis. In Western culture up to the 16th century, asceticism and access to truth are always more or less obscurely linked.

Descartes, I think, broke with this when he said, “To accede to truth, it suffices that I be any subject which can see what is evident.” Evidence is substituted for ascesis at the point where the relationship to the self intersects the relationship to others and the world. The relationship to the self no longer needs to be ascetic to get into relation to the truth. It suffices that the relationship to the self reveals to me the obvious truth of what I see for me to apprehend that truth definitively. Thus, I can be immoral and know the truth.

If AI images could be said to be “immoral,” it wouldn’t be because they fail to document the objective truth, because they make hands with too few fingers and so on, but because they administer to this kind of subjectivity, grounded in the idea that knowing truth requires no effort and has no price. You can be whatever sort of person you want to be and still have “your truth.”

Generative models excel at making ideas obvious; they make it easy to manufacture obviousness, not truth. But we are well accustomed to conflating the two.