April 4

Post-content, AI agnotology, heaven-banning

Life after content

If “generative AI” reaches the point of proliferation and saturation that its promoters predict for it, it will mean that anyone with access to a content generator will be able to chat the kind of content they want into existence simply by putting it into words. If you need to know how to do anything, you can just ask an LLM model to explain it. Economist Noah Smith argues that “because LLMs are great at summarizing the broad corpus of human written knowledge, they can also act as a how-to manual for any task that can be explained in writing — much like YouTube acts as a how-to manual for any task that can be demonstrated with a short video. LLMs won’t just explain how to do things, either; they’ll draw on the collective accumulation of human wisdom to offer ideas on how to solve problems.” You won’t have to know how to research and you won’t have to respect any copyrights on that “collective accumulation of human wisdom” (which, after all, amounts to nothing more than the largest number of words assembled in one place); you can just ask a model what to do and then follow its orders, if it can’t already execute the task itself. The gap between the idea and the execution will apparently be closed, which would almost seem to make execution redundant. If you’ve imagined something, it’s already done.

But the gap between imagining and executing is not inert dead time, anymore than the process of reading is a wasteful inefficiency arresting the flow of information. That gap is where engagement occurs and is sustained. Chatbots and other content generators are sometimes promoted as offering shortcuts around the work of being invested in something, so you can just get to the thing itself. This makes sense only for things you aren’t interested in thinking about in the first place. What they produce won’t be a revelation; it instead marks the absence of a process of discovery, an unmarked grave for a moment of curiosity.

What I usually take away from interacting with chatbots, more than any information they supply, is a sense of immediacy, a visual spectacle of words coming from nowhere unspooling themselves on the screen. This still feels novel, that I can set this process in motion and sit back and watch it produce. When the words appear, I try to see what’s produced as setting a baseline for what the most unimagined or uninspired version of an idea looks like, what the least satisfying answer is. I want to think of it as a rough draft for “real” content, as something demanding further attention from me, setting me on a course of further exploration and refinement. But typically none of that happens. Rather than engage with the chatbot and take a kind of ownership over the output I’ve prompted it for, I find that I expect it to continue on its own momentum. I want it to be responsible for holding my interest; I want it to show me better criteria for why I should even care about what I was asking. Having asked it to think for me, I want it to do the metacognitive work of supplying my intentions too.

Generative AI could save us from having to think about unfulfilling obligatory tasks so we have more cognitive power left for the “real” content we want to think about. To put that another way, AI’s usefulness lies in convincing us that it’s not a tool that we are obligated to use but an “intelligence” that saves our own. But if that’s the case, it will be oriented toward producing our ignorance as evidence of its efficacy. It may work to habituate us to an immediacy that makes all thinking appear arduous, eliminating the perceived difference between tasks worth automating and those worth engaging. If “content” denotes something that produces engagement, generative AI may be understood less as a content generator and more as a content destroyer. Instead of content, it just yields output. Instead of everyone becoming “content creators,” we’ll all become content to be inputters.

Brain capers

In a recent piece for The Conversation, sociologist David Beer argues that “there is a good chance that the greater the impact that artificial intelligence comes to have in our lives, the less we will understand how or why.” This is because current AI is predicated on neural nets, which seek to emulate the brain in its unfathomable complexity by simply replicating it without understanding it. As Beer puts it, “the unknown and maybe even the unknowable have been pursued as a fundamental part of these systems from their earliest stages.”

Tracing the history of neural nets back to the mid-20th century, Beer details how the “ethos driving them has a particular and embedded interest in ‘unknowability,’” which is taken not as a problem to be resolved but as a guarantee of a process’s sophistication. We don’t understand the brain, ergo to make a mechanical brain, we can’t be able to understand it either. Hence the development of a layered process of interconnection among a system’s “neurons” to make an AI system impossible for even its designers to understand, thus allowing it to exceed them. The system will be “self-organized,” which allows for a leap into speculation about its autonomy.

The consequences of this design approach is that output results are impossible to reverse-engineer through the layers and layers of interconnected neurons, the millions and millions of parameters shaping the unfathomable “latent space” that models develop in being trained. With no definitive explanations, users are invited to speculate, much as algorithmic feeds invite users to develop folk explanations of how and why content is displayed when it is. The Loab phenomenon has already demonstrated how quickly that can turn into conspiracy theorizing.

Yesterday, Midjourney announced a “describe” feature that outputs four text prompts for an uploaded image. (Here is an example from Twitter.) This would seem to allow some insight into how its model “sees” images, but it remains largely a kind of Ouija-board approach to revere-engineering. It seems to suggest that the model “thinks” in a language we can understand, whereas the neural nets aren’t working semantically but instead filling in massive matrices with numeric values. “Describe” seems like it is revealing the system’s magic words, but a long string of hexadecimal code would probably be more accurate.

“AI as anti-social media”

Sam Lessin posted a piece that takes up the “heavenbanning” idea, imagining that platforms will be populated with AI-generated fake people who like our content and tell us how great we are. In other words, generative AI could work within social media as its opposite, replacing interaction with other users with more or less disguised chatbots. This will eventually drive all users into isolated worlds where they feel heard even though no one is listening, and it will neutralize the usefulness of social media platforms for any kind of organizing work.

Setting aside the fact that most online metrics are already unreliable and bot-ridden, and that algorithmic feeds already make any content they display implicitly about the users themselves, I’ve never been especially convinced that people would want an army of yea-sayers love-bombing them. Not to mystify the struggle for recognition, but it seems as though it’s generally obvious when it is not taking place, when there is no one on the other side mediating your self-consciousness with their self-consciousness and nothing is really at stake. Maybe when bots can convincingly mimic having intentions of their own, this will change.

It strikes me as somewhat more likely that social media will cease to provide useful, uncontaminated data for training AI models. Eryk Salvaggio pointed out that “Something worth noting about current generative AI is that it is dependent on networks to exist but displaces their collaborative capacity.”

It’s pure speculation on my part (see the previous section), but it seems that LLMs work by modeling the sociality embedded in language. Useful, valid information derives from collaborative activity, from the gestures that establish consensus. Without examples of that, models couldn’t assign probabilities to various patterns of words — they are only “probable” in relation to some intended meaning, to some receiver who is trying to understand or some purpose that someone is going to evaluate. When the makers of LLMs can no longer suck social interaction out of existing networks — when all the conversations among the living have moved elsewhere — the models will perhaps be left to feed only on themselves, devolving into echolalia.