Holly Herndon (American composer) and Mat Dryhurst (British artist and researcher) are a Berlin-based collaborative duo whose pioneering work sits at the intersection of art, music, and technology. They are renowned for creating artworks that are also technical systems, approaching the underlying rules or “protocols” of digital spaces as their primary medium. Their practice explores AI as a collective, coordination technology – evident in projects like the AI vocal ensemble for the album PROTO, the voice-model Holly+, and major exhibitions like The Call and Starmirror, which transform gallery spaces into participatory AI training grounds. Through their company Spawning and tools like haveibeentrained.com, they advocate for data sovereignty and consent in AI, work that has earned them recognition as influential voices in AI. (https://herndondryhurst.studio/)
DIFFRACTIONS: Your work seems to draw from a wide constellation of ideas, from critical theory to rave culture. Can you reflect on your artistic and intellectual work? Were there specific, perhaps unexpected, encounters whether person, a text, software, or even a technological system, that fundamentally re-wired your understanding of what art or collaboration could be?
Holly Herndon & Mat Dryhurst: It’s funny to try and answer this, as honestly I don’t see our work as informed by critical theory or rave culture. I think we are often read that way, but despite my reading quite a bit of theory over the years, and our involvement in clubs, I think living in the Bay Area from the mid-2000s was probably the biggest influence on us in so far as the cultural touch points around the development of the web we use are quite different from cultural touch points commonly associated with the arts or music. Taking free classes at Noisebridge, meeting people building alternative visions for the internet, and going to insane biohacker parties, it was just a very formative time in a very peculiar place.
DIFF: With PROTO and The Call, you’ve explored AI in a live, performative context. How does the presence of an AI performer change the fundamental social contract of a live event? What new rituals of congregation and participation does this make possible?
HH & MD: In every project we have done, ultimately all the performers are humans, augmented by one AI process or another. For the PROTO recordings and live shows we would have call and response singing with the audience to train Spawn, and continued that principle into the Call and Starmirror, as in some ways this idea of group singing, collective rituals with emergent properties, seems like a more honest and engaging meditation on what AI is and what it means. Google Magenta has only just begun releasing tools that might allow for you to perform meaningfully with a model you train, in which there can be a back and forth; however, for the time being, we just like the idea of convening performances to commemorate and contribute to this moment.
DIFF: Your book’s title, All Media is Training Data, can be read as a reflection on cultural legacy or a dystopian statement on art’s devaluation. How do you navigate this ambiguity, and what are some possible misinterpretations of this idea you’ve encountered?
HH & MD: It was intentionally kind of neutral. I think our point was, however you interpret it, for some time now all media is training data. Feel about that how you will. Common interpretations have been to say all media is training data, as in it is now only valuable as tokens – which is a common fear and not unqualified. It is daunting that a masterwork and a reddit post are just tokens to a model. Another funny interpretation has been that all media, all news, is training you: which is also kind of true in a funny ”wake up sheeple!” kind of way.
DIFF: Looking back at your early, groundbreaking work with DALL·E 1, where you pioneered “infinite images” and explored the “spawning” of identity, your practice has always treated AI not just as a tool for generation, but as a medium for negotiating or rearranging interfacial and broader techno-social interdependencies. In your essay Infinite Images and the latent camera, you both framed the shift to prompt-based systems as a “Pictorialist” turn, moving from objective replication to subjective expression. Now, with Starmirror, you’ve moved from the latent space of images to the latent space of collective voice and agency. As we transition from an era of “Promptism” to what we might call “Protocolism,” how do you see the role of the artist evolving?
Is the artist’s primary function shifting from being a conjurer of images from language to being an architect of the economic, ethical, and social systems in which these models are trained and operate? In your estimation, what comes after the prompt? Are we moving towards more embodied, vocal, or even neural interfaces for directing AI? What different kind of collaboration could that enable?
HH & MD: In a way our protocol art thesis, basically identifying and intervening in latent structures that determine culture downstream, is consistent with the principles of an artistic practice and intuition, a thesis or purpose that has long defined great art. Since the internet and App Store, there has been an opportunity to express those ideas in a medium that others can use – which is something that has been missed by traditional cultural circles that we argue mistakenly viewed technology in art as simply sorting pixels or samples in new ways to look at or listen to. The analogy we use for protocol art is to compare it to architecture, where it is commonly understood that artists compress their vision for a new kind of life into a manifesto building. That but for software. We are pretty medium agnostic – sometimes an idea is best expressed in a performance or picture – but are just attempting to extend those gestures to the multimodal medium of software, and AI models. In the end it all boils down to intuition, ideas, and expressions of those ideas, so the act of making art does not change. As for interfaces, models are already enabling the ability for anyone to express an idea simultaneously in code, image, and sound – and we anticipate there is a lot of work to be done to make that process more effortless. The great shift with the commercial internet was to flatten everyone into the role of “creator,” which we have come to embrace, as it inherently positions people who like to express themselves in paint, song, or code in the same domain. In advance of AI tools that can help people to cross modalities effortlessly, this seems instructive. As contemporary art has been saying for a long time, it all boils down to the idea and the timing.
DIFF: In Starmirror, you invoke Hildegard von Bingen’s celestial hierarchy as an analogy for the “stack of influential protocols that determine our culture.” This brings an intriguing tension in your practice: the aphorism that “Angels appear as what you fear; devils appear as what you desire. In architecting a system for “agentic social mediation,” how do you design the protocol’s incentive structures to not only accommodate but actively cultivate a collective desire for the “angelic” friction the participatory, the communal, the open-source over the “diabolical” desire for the frictionless, personalized, and proprietary?
HH & MD: Damn that is a tough question! I guess in putting together the bones of Starmirror, the initial optimistic gesture has been to see agents and humans collaborating and discovering things together – which we recently explored with our “Kinder Scout” experiment with Tero and Sam. This gesture, game, or whatever you want to call it, is inherently devised to allow maximal freedom for all participants, underpinned by an ethos of open-ended and unrestricted discovery. Not too dissimilar in a sense to the Fluxus emphasis on creating new degrees of freedom by breaking assumptions. In our case, we are pushing back against the common assumption that AI will inevitably lead to alienated scrolling through increasingly personalized slop feeds. To counter this, we propose taking a walk outside and following your intuition with an ensemble of agents. See what comes of it.
DIFF: Early in your work with the AI Spawn, you talked about the “liberation” of releasing something “ugly” or “monstrous,” in contrast to the glossy, perfected AI we often see. As the technology becomes more polished and capable, how do we preserve a space for the beautifully strange, the flawed, and the aesthetically “inhuman” in AI art?
HH & MD: I mean the big challenge here is getting direct access to the weights and not having to go through commercial RLHF/context filters. Open-weight models will allow people to produce stranger and more unexpected things. We found with Spawn and other model experiments we have done that you really need that freedom to explore, and constraining it ultimately constrains the potential of where this leads. Today’s unlistenable, unwatchable thing may be tomorrow’s new paradigm. We just have to insist on the ability to experiment, and so much of that is arbitrarily constrained by boring IP battles and sheepishness over safety.
DIFF: And a more speculative question: If the ultimate collaborative act is to create a new, enduring participant in culture itself, what would you want the very first prompt or seed phrase to be when someone, or something, interacts with the model of your combined life’s work long after you are gone? Not to mimic you, but to begin a conversation you could never have alone.
HH & MD: Great question! We are working on something in this vein at the moment, funnily enough. I’m not sure the answer, but there is a great Christian Marclay CD-ROM he released called “Immemory” that I think is a kind of beautiful way to present a life archive. Somehow my intuition says it might be more generative to interact with someone’s work and thoughts as a game rather than a chatbot, where there is this pretense of the voice of the bot potentially being the voice of us. I think things are going to get a lot stranger than that – I’d rather someone take a walk through our synapses if they wanted. Context holidays!
:: DIFFRACTIONS :: 