unthinking
photography

Many rioters who stormed the US Capitol building on January 6 did not hide their identity behind a mask and even took selfies while engaging in clearly illegal activities. Selfie culture is such a compelling addiction that it overwhelms caution. That's part of what I play on with Lend Me Your Face!. On the one hand, seducing people to lend their faces; on the other hand, exposing them publicly, mouthing the words of public figures they may or may not actually support. Playing with that dynamic, that tension between people's desire to see their faces on the big screen and their realisation that it could be problematic for them personally.

That tension is a very conscious part of the project and that's what makes the piece stronger: when you participate and soon thereafter see your face in public on the large screen, as in the premiere of Lend Me Your Face! in the Artists’ Association show “Götzendämmerung” in Haus der Kunst. In that group exhibition, our assistants asked people waiting in line to enter if they wanted to be photographed to be part of one of the artworks. By the time they got to our hall, their deepfakes were on the large screens.

An emotion I want to play with is the exhilaration of giving up your individuality to become part of a larger, more powerful "body". Of course, this is not just a Big Brother/Fascist attraction, or the euphoria of being with the winning team, etc. This is why I included Greta Thunberg: she’s become a figurehead for a movement that I myself very strongly believe in, value and participate in. I believe in and repeat her words, and feel she speaks for us, fellow believers. So in Lend Me Your Face! (the title comes from Shakespeare’s play Julius Caesar, where Mark Antony says, “Friends, Romans, countrymen, lend me your ears!”) you're becoming part of someone else's chorus, for good or for evil – and each person might have a different idea of who is good, and who evil!

With this piece, people often ask me, “Can I choose who I get to be?” My answer is, “No!” That's the whole point: we're putting our faces out there on social media, they're being harvested, we're not being asked for permission at all, we're not even being told that it's happening. In Lend Me Your Face!, within the constraints of German data privacy laws, I'm trying to give you as much of that experience as possible.

Yes, and so watching our “self” mirror someone outside of our control is uncanny and provokes a very unsettled emotional response. This comes out of the theory of abstract dramatic structure I developed for my time-based artworks, drawing on Emotion and Meaning in Music by Leonard B. Meyer. Meyer talks about playing with people's expectations within a known framework, in order to trigger feelings of surprise, relief, elation, frustration, and specifically keeping people in a space of uncertainty that engages their attention.

With Lend Me Your Face!, I was surprised myself by the subtlety in the facial movements produced by the deepfake software – and I think my surprise was because I saw it happen to me, so I paid much more attention to the details. These are not the most accurate deepfakes by any means, and perhaps there’s already software out there that makes these seem primitive, as researchers continue to push the technology. But the fact is that with this minimal input of one photo you can already generate a deepfake video with a good level of fidelity in about a minute, running a neural network trained on a very general data set on a single PC without cloud computing!

Of course, this is no comparison to very refined deepfakes, for instance of Barack Obama, where it seems that he, in his own voice, is saying words that he has never actually said, or with which you can manipulate George W. Bush’s video in real-time. From what I’ve read these deepfakes are very involved: constructing a 3D model and using many different videos showing the target person from different angles to train a neural network. The point of our project is that with this software it is really fast and easy to create a pretty good deepfake of YOU with very little data, just one photograph of your face as input.

Yes, specifically creating an uncanny moment, rather than trying to avoid the uncanny valley, which is almost impossible especially with your own face. We are very sensitive to what we look like.

The clips that work best are reflected in the instructions of what input photo works best: facing straight ahead, mouth slightly open but not smiling - and whose head movements are relatively limited. The Queen is perfect: very still, calm, looking straight into the camera…. I found it really interesting that despite her very static body language and elocution we're still transmitting subtle but clear emotion in the deepfakes, for instance when she says, “We shall meet again”. With just a very subtle movement of the head and the voice dropping down a notch, she created a moment that is incredibly calming.

The mix of clips gives the piece a dynamic range of emotions: Boris Johnson optimistic and bouncy, Attenborough nodding with grave severity, Greta’s face distorting with rage and fury. The Queen is then an extremely calm, reassuring presence, creating a balance to them all.

In our earlier correspondence, you referred to AI technology that detects emotions. I guess I don't believe enough in that research to find it interesting. I'm half Japanese and my whole life I've noticed how Westerners often misinterpret Asians’ or Asian Americans’ emotions. For instance, in the courtroom scene in the novel Snow Falling on Cedars, the accused Japanese American shows absolutely no emotion. A Japanese American might say, “He fought to maintain his composure despite the fact that his insides were being torn apart and his soul was crying to the heavens,” because a stony face under extreme duress is a sign of great emotion and heroic control. From a Western viewpoint, however, it is interpreted as a sign of inhuman coldness. So I’m very suspicious of the inherent cultural biases in this sort of “research.”

In terms of facial expressions communicating points of emphasis in speeches, which I'm focusing on in this piece, you are much more aware of those sorts of facial expressions when you see your own face moving in ways that it normally does not.

Yes, a lot of that selection process was figuring out what will get people to engage, right now at this moment. For this second version, I swore if we could get Trump out of office I’d never use him again – he’s over, I've had enough of him, I really don't need to see him any more. I wanted to take the viewer on a sort of rollercoaster ride, creating a dramatic arc between positive and negative, between “yes, that's me, that's what I believe in, this is wonderful!” versus, “oh my god, what am I doing here, let me out!” Giving them too much control over who they can be, makes it too harmless and therefore not so emotionally engaging.

Your private deepfake videos that are generated with the Go Fake Yourself! app are not being identified, and can’t be viewed by anyone except yourself, on the device you used to generate them. That’s different from the ones on public view on the Media Wall (or if it’s closed, The Photographers’ Gallery’s website). We do request that if your image works well and you'd like to be part of the public presentation, please send us that photograph. On the public displays, we will only use photographs for which people have given us their explicit permission. That is a completely separate data set from the deepfake files you generate on your device.

The Photographers’ Gallery has previously shown works of art about scraping databases, for instance how the ImageNet database has been used to create training sets with images or videos uploaded by people who never thought they’d be used in that way and were also not asked for permission to use them in that way...

Yes, but in our work, we’re not really focusing on what is in the database used to train the software. We do use a neural network from A. Siarohin et al. that was trained on the VoxCeleb data set scraped from YouTube, but for us, the question was more, what can we do with an existing deepfake system? How can we use it to create a personal encounter with the technology for our users?

We had discussed using input photos of celebrities or using artificial faces generated by another deepfake software. But for me, they were too pretty, too remote and I specifically want the users to realise, hey, this can happen with any of us, not just celebrities who have always been in the public eye and the subject of cheapfake memes.

Yes! I don't know what the privacy rules are about that in the UK. I know in Germany you wouldn't be able to do it without consent in some form, at the very least a sign saying if you go past this sign, you have given consent to be recorded. But, yes, I would love to do a version like that!