Maintaining Composure: An Interview with Tamiko Thiel
Sarah Cook is a curator and professor in Information Studies at the University of Glasgow who has written about artists’ uses of AI.Read full Bio
As Tamiko Thiel’s work Lend Me Your Face! is shown on The Photographers' Gallery website –waiting for the gallery's reopening for it to be shown on the Media Wall-, she talks with curator Prof. Sarah Cook about deepfakes, identity and user agency across a brief history of participatory networked art projects, political incitement, and how we all instinctively react to facial expressions.
View the online commission Lend Me Your Face: Go Fake Yourself! by Tamiko Thiel and /p.
This conversation was recorded over Zoom, extracted from an automatically-generated transcript and then edited and expanded. Cook and Thiel began by talking about the media coverage of the riot at the US Capitol building on January 6 2021, specifically about how visually identifiable some of the rioters were, and of the efforts by activists, artists and hackers to capture social media posts and digital assets which could help document the perpetrators and explain how the riot came about.
Many rioters who stormed the US Capitol building on January 6 did not hide their identity behind a mask and even took selfies while engaging in clearly illegal activities. Selfie culture is such a compelling addiction that it overwhelms caution. That's part of what I play on with Lend Me Your Face!. On the one hand, seducing people to lend their faces; on the other hand, exposing them publicly, mouthing the words of public figures they may or may not actually support. Playing with that dynamic, that tension between people's desire to see their faces on the big screen and their realisation that it could be problematic for them personally.
That tension is a very conscious part of the project and that's what makes the piece stronger: when you participate and soon thereafter see your face in public on the large screen, as in the premiere of Lend Me Your Face! in the Artists’ Association show “Götzendämmerung” in Haus der Kunst. In that group exhibition, our assistants asked people waiting in line to enter if they wanted to be photographed to be part of one of the artworks. By the time they got to our hall, their deepfakes were on the large screens.
The personal, private net art version we’re showing at The Photographers’ Gallery, Lend Me Your Face: Go Fake Yourself! doesn’t have this immediate public exposure due to the coronavirus lockdown. But you are taking the photograph yourself, uploading it yourself, and (unless a huge number of people do it simultaneously) you see the first deepfake of yourself in about a minute, with the rest trickling in soon after. You personally choose to be involved in the process and see very quickly how little is required to create the deepfakes of you doing something that you haven't ever actually done. These are the tensions that are really important for me to play with, in different versions of this work.
You invite people in the web version of the work to upload their own picture, and yet, when I did it yesterday, I didn't know who I would be turned into – I didn’t want to be Boris Johnson! But suddenly I was there, speaking his words, with that kind of embodiment. I think there are other interesting examples of works of art, even from 20 years ago, before we started talking about facial recognition software or deepfakes, where users were invited to upload pictures in order to become someone else, such as irational.org (Heath Bunting and Olia Lialina)’s Identity Swap Database (from 1999) or Cheryl L’Hirondelle’s Treaty Card (from 2002), both now critically endangered works of net art, in which you could put your photo in and the system would make you new pieces of identification which you could actually use in the real world. Those works were giving the user agency, offering an invitation to the viewer, to give their face, but for a clear reason that they wanted to be part of – to swap, confirm or create a new identity, not just to be used as a mouthpiece, as it were.
An emotion I want to play with is the exhilaration of giving up your individuality to become part of a larger, more powerful "body". Of course, this is not just a Big Brother/Fascist attraction, or the euphoria of being with the winning team, etc. This is why I included Greta Thunberg: she’s become a figurehead for a movement that I myself very strongly believe in, value and participate in. I believe in and repeat her words, and feel she speaks for us, fellow believers. So in Lend Me Your Face! (the title comes from Shakespeare’s play Julius Caesar, where Mark Antony says, “Friends, Romans, countrymen, lend me your ears!”) you're becoming part of someone else's chorus, for good or for evil – and each person might have a different idea of who is good, and who evil!
Yes, you get this similar feeling of an incitement to act when watching Julia Schicker’s work Utopia Generator 1.1 (2019) – a deepfake video of a right-wing politician giving a talk about climate change, saying exactly the opposite of what he believes.
With this piece, people often ask me, “Can I choose who I get to be?” My answer is, “No!” That's the whole point: we're putting our faces out there on social media, they're being harvested, we're not being asked for permission at all, we're not even being told that it's happening. In Lend Me Your Face!, within the constraints of German data privacy laws, I'm trying to give you as much of that experience as possible.
That does make Lend Me Your Face: Go Fake Yourself! very different from earlier works which let users choose to be somebody else. I’m also reminded of Fantasy A-list Generator by Active Ingredient (a duo of artists) which I commissioned in 2008 - where gallery visitors could dress up in wigs and hats and press a button to be interviewed by a piece of software pulling questions at random from a database populated by an archive of television interviews of celebrities (the resulting videos were projected on the gallery wall outside the interview booth, and visitors could also post them to social media; a bit like when people use Snapchat, YouTube or TikTok to enhance their online personas or to try and become celebrities). In this case, you’re working from speeches – putting words in someone else’s mouth – rather than trying to make meme-worthy celebrities of us all.
I also think this is where this work really stands out, as the algorithm plays a key role in creating a personal and emotional moment. The movement of one’s face is not something that is on an identity card even though control systems seek ever greater granularity in identity certification, like the way you bop your head up and down, or other gestures or movements, not just the shape of your ear or the colour of your eyes. The ability for the impersonation or fake to be realised not just at the level of static representation - in likeness only - is really interesting too. But the moment of recognition, and our immediate emotional response to seeing another’s face smile or grimace – the firing of a mirror neuron maybe – is what makes us all human...
Yes, and so watching our “self” mirror someone outside of our control is uncanny and provokes a very unsettled emotional response. This comes out of the theory of abstract dramatic structure I developed for my time-based artworks, drawing on Emotion and Meaning in Music by Leonard B. Meyer. Meyer talks about playing with people's expectations within a known framework, in order to trigger feelings of surprise, relief, elation, frustration, and specifically keeping people in a space of uncertainty that engages their attention.
With Lend Me Your Face!, I was surprised myself by the subtlety in the facial movements produced by the deepfake software – and I think my surprise was because I saw it happen to me, so I paid much more attention to the details. These are not the most accurate deepfakes by any means, and perhaps there’s already software out there that makes these seem primitive, as researchers continue to push the technology. But the fact is that with this minimal input of one photo you can already generate a deepfake video with a good level of fidelity in about a minute, running a neural network trained on a very general data set on a single PC without cloud computing!
Of course, this is no comparison to very refined deepfakes, for instance of Barack Obama, where it seems that he, in his own voice, is saying words that he has never actually said, or with which you can manipulate George W. Bush’s video in real-time. From what I’ve read these deepfakes are very involved: constructing a 3D model and using many different videos showing the target person from different angles to train a neural network. The point of our project is that with this software it is really fast and easy to create a pretty good deepfake of YOU with very little data, just one photograph of your face as input.
There is something about the importance of, first of all, the viewers recognising those characters (the Queen, the Prime Minister) and then recognising themselves, and having that moment of uncanniness – hearing someone else's voice come from their face. You break that familiar recognition moment...
Yes, specifically creating an uncanny moment, rather than trying to avoid the uncanny valley, which is almost impossible especially with your own face. We are very sensitive to what we look like.
Perhaps even more so now that we’ve been staring at our own facial expressions in Zoom meetings for the better part of a year! I'm sure there are technical reasons for the clips that you picked, as well as ideological ones in terms of the content of what the people are saying?
The clips that work best are reflected in the instructions of what input photo works best: facing straight ahead, mouth slightly open but not smiling - and whose head movements are relatively limited. The Queen is perfect: very still, calm, looking straight into the camera…. I found it really interesting that despite her very static body language and elocution we're still transmitting subtle but clear emotion in the deepfakes, for instance when she says, “We shall meet again”. With just a very subtle movement of the head and the voice dropping down a notch, she created a moment that is incredibly calming.
The mix of clips gives the piece a dynamic range of emotions: Boris Johnson optimistic and bouncy, Attenborough nodding with grave severity, Greta’s face distorting with rage and fury. The Queen is then an extremely calm, reassuring presence, creating a balance to them all.
In our earlier correspondence, you referred to AI technology that detects emotions. I guess I don't believe enough in that research to find it interesting. I'm half Japanese and my whole life I've noticed how Westerners often misinterpret Asians’ or Asian Americans’ emotions. For instance, in the courtroom scene in the novel Snow Falling on Cedars, the accused Japanese American shows absolutely no emotion. A Japanese American might say, “He fought to maintain his composure despite the fact that his insides were being torn apart and his soul was crying to the heavens,” because a stony face under extreme duress is a sign of great emotion and heroic control. From a Western viewpoint, however, it is interpreted as a sign of inhuman coldness. So I’m very suspicious of the inherent cultural biases in this sort of “research.”
Artists’ works are always brilliant at pointing out inadequacies in technical systems to capture the nuances of human experience. I was thinking of Kyle McDonald and Lauren Lee McCarthy’s work Vibe Check, which uses facial recognition combined with expression analysis to identify and quantify the emotional effect that visitors to the exhibition have on one another; so rather than the system identifying you as happy or sad, it calls out the other person you’re with and announces to all gallery visitors that they’re making you feel anger or glee or whatever. It’s very pointed because it exposes its own inaccuracy and it makes it personal.
In terms of facial expressions communicating points of emphasis in speeches, which I'm focusing on in this piece, you are much more aware of those sorts of facial expressions when you see your own face moving in ways that it normally does not.
You've picked archetypal expressions as well, haven’t you? You've picked optimism and gravitas and disgust or concern.
Yes, a lot of that selection process was figuring out what will get people to engage, right now at this moment. For this second version, I swore if we could get Trump out of office I’d never use him again – he’s over, I've had enough of him, I really don't need to see him any more. I wanted to take the viewer on a sort of rollercoaster ride, creating a dramatic arc between positive and negative, between “yes, that's me, that's what I believe in, this is wonderful!” versus, “oh my god, what am I doing here, let me out!” Giving them too much control over who they can be, makes it too harmless and therefore not so emotionally engaging.
This interview takes place as part of your exhibition at The Photographers’ Gallery in London, so it would be interesting to also talk about photographs and networked images. Most AI art relies upon the creation of a big database of images, or the borrowing of training sets from something like ImageNet. Not in this case, although it is based on a form of machine learning – a piece of code that's enacting something it has been trained to do, but it has been trained on other images than the ones users are providing. Although you say you're not keeping them, and users can delete their deepfakes, you nevertheless end up with a database of people's photographs?
Your private deepfake videos that are generated with the Go Fake Yourself! app are not being identified, and can’t be viewed by anyone except yourself, on the device you used to generate them. That’s different from the ones on public view on the Media Wall (or if it’s closed, The Photographers’ Gallery’s website). We do request that if your image works well and you'd like to be part of the public presentation, please send us that photograph. On the public displays, we will only use photographs for which people have given us their explicit permission. That is a completely separate data set from the deepfake files you generate on your device.
The Photographers’ Gallery has previously shown works of art about scraping databases, for instance how the ImageNet database has been used to create training sets with images or videos uploaded by people who never thought they’d be used in that way and were also not asked for permission to use them in that way...
Such as in the work of Trevor Paglen...
Yes, but in our work, we’re not really focusing on what is in the database used to train the software. We do use a neural network from A. Siarohin et al. that was trained on the VoxCeleb data set scraped from YouTube, but for us, the question was more, what can we do with an existing deepfake system? How can we use it to create a personal encounter with the technology for our users?
We had discussed using input photos of celebrities or using artificial faces generated by another deepfake software. But for me, they were too pretty, too remote and I specifically want the users to realise, hey, this can happen with any of us, not just celebrities who have always been in the public eye and the subject of cheapfake memes.
I can imagine the third iteration that has the surveillance camera built-in, where you walk past The Photographers’ Gallery and then the Media Wall pops up your image – “we've just taken this photo of you, look here you are in this deepfake video...”
Yes! I don't know what the privacy rules are about that in the UK. I know in Germany you wouldn't be able to do it without consent in some form, at the very least a sign saying if you go past this sign, you have given consent to be recorded. But, yes, I would love to do a version like that!
Tamiko Thiel is a visual artist exploring the interplay of place, space, the body and cultural identity. She works in a variety of media ranging from supercomputers to digital prints and videos to interactive 3d virtual reality worlds and augmented and mixed reality artworks and installations. She and her co-artist /p are showing Lend Me Your Face!, an AI deepfake video artwork at The Photographers’ Gallery from January 18th - March 17th 2021.
Lend Me Your Face! uses an open source deepfake neural network framework developed by A. Siarohin et al., with "deep-animator," a wrapper created by Dimitris Poulopoulos and extended by Christoph Clement.