The future of live performance
36:13 Share

This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. I thought it would be good to revisit the original intent of this show. In 2017, when we started, we wanted to create a forum to dive into and discuss the motivations and the research that my colleagues do across the campus in science, technology, engineering, medicine, and other topics.

Stanford University and all universities, for the most part, have a long history of doing important work that impacts the world. And it's a joy to share with you how this work is motivated by humans who are working hard to create a better future for everybody. In that spirit, I hope you will walk away from every episode with a deeper understanding of the work that's in progress here and that you'll share it with your friends, family, neighbors, co-workers as well.

For me, I was having these like really important experiences with and relationships through digital technology, whether it was like, you know, talking to my parents or romantic partners. I was like, oh, this doesn't show up on stage. And so it prompted this weird shift for me as a

director and theater maker, I was like, I think I need to figure out ways to incorporate more technology into the kinds of performances that I'm making because that's actually reflective of my life and the life of most people in the audience. And it's kind of weird to say, oh yeah, we're going to pretend for an hour and a half that no one has a cell phone. ♪

This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. If you're enjoying the podcast, please consider rating and reviewing it. That's a great way for us to get feedback and also to spread the word about the podcast. Today, Michael Rau will tell us that theater performances are increasingly using digital media and even AI to expand the creative space of theater and make new things possible. It's the future of live performance.

Before we get started, a reminder to rate and review the show. We like to get a 5.0, but only if we deserve it. I think all of us have been to the theater to see a live performance. It's really talented people doing what they do best: singing, performing opera.

doing a dramatic presentation of a play that maybe you know by heart, or maybe it's the first time you've seen it. But you know what? Digital media is starting to change that. There are new creative possibilities, and people are trying to figure out how do we sensitively, creatively, and artistically integrate digital media and AI into theater.

Well, Michael Rau is a professor of theater and performance studies at Stanford University. He's a director. He's directed plays, operas, other performances, and he's also interested in digital media and AI and how they may transform performance. He has really interesting thoughts about the relationship of gaming to theater, the relationship of individual human audience members to the performance that they're watching or experiencing, and many others.

Michael, you have focused on digital media in theater and in performance, among other things. What's the opportunity there? Why are we talking about that? Well, I think the opportunity is, you know, to make new kinds of art, to kind of take this art form that's been around for centuries and re-examine it or put it into a slightly different medium or make some slight alterations and say like, okay, what types of

experiences and stories can come out of that. That's option one. And then option two is that in the theater, there's always this idea that the theater should reflect life, right? The life that we live. And increasingly, more and more, our lives are mediated by technology. And so one of the things for me that I had like a weird crisis in my

roughly like 2011, where the plays that I was making on stage

weren't reflecting the life that I was living. Like it's pretty rare when you watch a play to see someone text someone else or do a Skype call, especially back then in 2011. But for me, I was having these like really important experiences with and relationships through digital technology, whether it was like

you know, talking to my parents or romantic partners, I was like, oh, this doesn't show up on stage. And so it prompted this weird shift for me as a director and theater maker. I was like, I think I need to figure out ways to incorporate more technology into the kinds of performances that I'm making, because that's actually reflective of

my life and the life of most people in the audience. And it's kind of weird to say, oh yeah, we're going to pretend for an hour and a half that no one has a cell phone right now. Right. Right.

Okay, so that's great. And my head is now officially exploding with follow-up questions. But let me just ask, I think we need to establish a denominator here about the kinds of things you're looking at with respect to things like AI, virtual reality, because then we can have a conversation about how your colleagues think about it, how the audience is like it, what it means for the existential human questions.

But so give me a sense of the kinds of technologies that you're experimenting with, you and others I know are working with. Like I learned a little bit about hamlet.ai, which looked really interesting. And tell me about it. Well, so I,

I, there's a bunch of different areas of exploration and, you know, I, and I think it's important to give like full context to the kind of work that I do. Um, there are times where I will just do a Shakespeare play, like a normal Shakespeare play and totally fine, you know, and I'll work in opera. And then there are other times that I will, you know, say, well, what are the sets of technologies that exist in the world and how can I bring those into

into a performance. And so those have ranged from, like, as you mentioned, like, currently I'm kind of exploring what are these different machine learning algorithms and what are the ways in which I can incorporate them into the theater. Earlier than that, it was kind of how do you do spatial storytelling? How could you use, like, kind of augmented reality or even like Bluetooth beacons to

uh, kind of tell a story through a space using phones. And then earlier than that, I was like, I got obsessed with email. I was just like, you know what? This is a thing that people spend hours doing. Um, and it's really, uh, a technology that has infected every bit of our lives. And can I tell a story through email? Can I make a theatrical performance that, uh,

explores that particular medium. And so it's kind of a real range of technologies, but the guiding principle is like, what are people using and how is it affecting their lives? And how can I then reflect that

back to people in that theatrical performance. Okay, so thank you. So that really does set a very nice baseline of the kinds of things and I'm really glad you pointed out that you are a fully credentialed normal director, forgive my use of that language, where humans are in front of you and you're telling them where to go and what to say and how to do it. So okay, so the first set of questions are:

How do you sit within your colleagues with respect to... Is this a mainstream within theater and performance now? Or as a professor, are you pushing a boundary that may or may not yield a big change in how the regular practitioners do their craft? I think the way that I like to talk about it is...

Because it can be really big and kind of complicated to talk about theater in all of its forms and to kind of locate it more specifically. And so I kind of use this analogy to food. So when we say theater, let's say food. And within food, you can have certain regional dishes like American food, German food. And there are times where I make –

American fare when I'm here at Stanford. And then there's times where I'll get invited to Germany and I'll make something that's more culturally flavored within food. You can also be like, I just want a simple recipe that is time trusted and true. And, you know, people have been making it since the 1800s, you know, or sometimes I want to go to a high end, very fancy modernist cuisine where there's a tiny little thing and a, a,

foam and you know, like all that. Like the bear, like the bear. Yeah. You know? And so, uh,

When we talk about theater, it's like that. There are some times, and particularly I would say this work is much more of like, here's this very narrow niche where some people are really excited and interested in it. And sometimes I like to make shows in that way. And then other times I will say, you know what? Like the show that I'm doing right now at Stanford, this is just going to be a very...

not traditional, but like a recipe that people already know can appreciate. There's nothing like other

avant-garde or funky about it. And oftentimes I think people, when they talk about art, they only want to say they want to be like really fundamentalist about it. We're like, Oh, we only have to push things forward or like, no, we have to keep things in the past. And actually as human beings in this society, we're quite good at letting these two things exist simultaneously, at least in like the restaurant world. And for some reason we don't.

That is super, super helpful. And I'm very aware that I'm not versed in the theory of theater and live performance. So some of these questions might be extremely basic. But I was struck when you said about electronic media is so pervasive in our life. And yet you go to the theater and nobody's texting, nobody's using email. Maybe we should address that.

But then my initial, as you were saying that, I was thinking, well, we don't, we also go for walks in the forest. And in general, there's not a huge clamoring, although we all do know that the first thing we'll do is check to see if we have reception. If we're lucky, there won't be reception and we'll actually have a fully walk in the forest. And what you just said kind of answered that, my concern about that, because

what I was thinking was, well, wait a minute. I'm not sure I need to have electronics and virtual reality in the theater because I have it everywhere else in my life. And maybe this is more like a walk in the forest. But then you said, Russ, you can walk in the forest, but you can also walk in VR and it's all allowed and it's perfectly reasonable. So I withdraw that question that I never asked. And

And I want to go to something that might be, I don't know if it's weird, but a lot of what you're talking about reminds me of gaming. And I think about gaming because I don't do it at all. And I might, my partially because of my age, it just wasn't a thing. I did play some games, but

I have seen the growth of the gaming industry. And at first I was extremely judgmental and extremely like, what are they doing? But then I talked to some thoughtful gamers who told me these are about stories that are reconfigurable. You care about the characters. You kind of become and then interact with characters. And it's this whole thing that made me say, oh my goodness, maybe this is the next big form of like, well, it's...

trite to say the next big form of entertainment because it's humongous business. But it took me a while. But then I began to understand why this, why really good storytellers might go into game creation, why really good directors might start directing games. So tell me about, is there a relationship in your mind between theater live performance and the gaming industry? And how should I think about all of that?

Well, I definitely have an opinion about it that I don't know how the gaming folks will feel about it. But my kind of like attitude on it is that I – and it totally comes from my own biases. But I always see the world of gaming as coming out of the world of theater. Well, there you go. I'm taking over gaming. I'm saying actually you're part of theater. Yeah.

that for me, theater in its like basic form where is a game, right? I'm going to pretend to be somebody else for the next 20 minutes or hour and a half, or I'm going to pretend to be a guy called Romeo. And then these things are happening. And it's a game that only works if the audience chooses to pretend that I'm somebody else. Right. And so I actually, you know, we call a play theater.

play you know that like it's all built into that kind of same if i want to use a fancy word like ludic component to uh to these systems of entertainment and i think it's really just about like how much of a how much towards a sort of like mechanic approach where like

So I certainly did grow up playing video games like, you know, Super Mario Brothers is about like jumping and hopping on these other things. And it's less about the story. There's like a very simple story about a plumber, but mostly it's about the mechanics of the thing. You can lean in towards the other side, which gets more theatrical or many of these games that are much more like what people call cinematic. And I would call theater is really about telling these elaborate, beautiful stories that are really complicated and,

I think it's all actually the same thing because I, and,

and here's my argument for why it's theater, it requires the same suspension of disbelief by the audience member to say, instead of this guy standing in front of you and pretending to be Romeo, I'm going to say that these pixels in front of me as they move are a character that has emotions and feels, and I'm going to ascribe all of those things to these moving sets of pixels on a screen. I actually don't think that there's that much of a difference, but

So at its core, the experience of the audience is kind of the same. Yes, thank you. That actually, I love that answer. And I don't know what the gamers think, but I'm done. I'm done. Gaming is a branch of theater. Let's move on. So another thing I wanted to ask you about is one of the reasons when I go to a theater and really any performance and really anything,

I'm looking for people who are at the pinnacle of capability in whatever they're doing. And typically it makes me cry. I'm not even kidding. Like an operatic singer who's just killing it or a performance. And I wanted to ask about that part of the theater, at least for me, and part of your answer, of course, could be, Russ, I already told you that we can have normal, normal, I don't even want to use traditional. We can have traditional theater, of course. But

what is the role of the human in theater? Um, and especially when I think about like what you just said, that there might be pixels that are making me cry. Is that going to happen? Um,

How should I think about the connection to a human who I think this is talent plus hard work all putting on display for me and it's going to make me cry right now? And I know you deal with real actors and you direct them. Is there going to be changes in how we relate to the objects of our entertainment? Or am I going to still see or is it just going to change the way absolute excellent pinnacle human performance gets portrayed? Absolutely.

I don't think so. Okay.

Sorry, I asked a long rambling question. Well, and it's also a very complicated one, and there are many people in the theater industry who get very upset by the answer that I like to give, but it's my true response. So what you're talking about is this question of liveness. Is liveness important to the theatrical event? And there are some people who are like, absolutely.

Absolutely. Like it's not theater if there isn't a person on stage doing that thing. It doesn't count if they're not breathing with the audience. And also a little bit at risk, right? A little bit like something could go wrong, but the professionals and the killers, it never seems to go wrong. They just kill it. Sure.

take a different approach and I would say that there are because again like I have this idea about like

Computer games are a portion of theater, and you can get moved by a computer game in the same way that you can if you've ever seen a really amazing puppet show. There's no live actor on stage. You're watching a piece of wood get jiggled around at certain times. And at the same time, if you've seen incredible puppetry, it is deeply moving. Yes. And I...

as I was telling you, like the, I had this crisis back in 2011. I got really, I decided I'm going to make a piece that's, um,

that pushes this question to like the forefront. And so I did the craziest, weirdest thing possible. I built a show where there are no actors and it's just an office cubicle and you walk into the office cubicle. And this is when I was obsessed with email and you just get an email from your boss

that's addressed to you that says, hi Russ, sorry, I'm at an offsite today. I just need you to do this Excel work. And the entire performance is you sitting at a computer terminal, answering emails and doing calculations in Microsoft Excel. And I'm, I'm not joking when I say this is the most successful show that I've ever produced. Wow. It's been running, uh, it won, uh,

some of the top awards in the Edinburgh festival. What's the name of the show? It's called temping. So yes, you know what like temping is and where do you come in and do? I did it in college. Yes. It's exactly that experience. But I set out the challenge for myself to, can I create a, a satisfying theatrical story? Um, just using these tools. So you never see anyone else. You're just alone in an office cubicle, but, um,

What you kind of through reading things and doing these sets of actions, you are taken on this really intense journey that asks you to ask really big questions about your life and the meaning of life. So I was still kind of able to kind of.

kind of mush those things in there. And that was a sort of deliberate provocation to say, I don't think that a live actor is necessary to have a satisfying theatrical experience. And people will quibble with me, where they'll be like, well, the audience is alive or like, and I'll be like, that's part of it. - But now you're at the boundary and it's very different from the question that I asked. - Yeah. - This is the Future of Everything with Russ Altman. We'll have more with Michael Rau next.

Welcome back to The Future of Everything. I'm Russ Altman, and I'm speaking with Michael Rau from Stanford University. In the last segment, Michael told us how theater should be broadly construed. Performances can involve humans. They can also involve augmentation with digital media and AI. In fact, he started to tell us about the show called Temping, which he created, that's very successful and very unusual. I'll ask him more about that in this segment.

Michael, you told me about one of your most successful performance or presentations, which was temping. And it was a person, a single person in a cubicle. How did that work? So you said there was some Excel, some email. Was this a highly branched logic where you said if they do this, then do this other thing. But if they do this, do this. Or was it kind of linear and you tried to keep everybody on a similar path? So thank you. I love this question. It was both. And

what we were trying to do is to like think a little bit about how to tell a story right with no actors, but also how to make it theatrically satisfying. So what we did is we based it off a real job that people do every single day, um, uh, uh, called being an actuary and an actuary. What they do is they calculate how long people will live, uh, for life insurance payments. Right. And like statistically, uh,

based off all of this, you can actually, uh, it's not that hard of a calculation to make. And we have all of these tables and you can get really precise data on like, um, uh,

how long, based off of a couple different factors, how long someone is going to be living. And I got fascinated with this idea. And so what we did is we put that into the show. So you would sit down at the cubicle, you'd get a couple emails from your boss, and they would say, all right, we just need you to do this calculation.

And you'd kind of look, open up this Excel file. It's a bunch of people kind of anonymized with like a certain couple little factors like the age, the gender, whether or not they're married or divorced, because that apparently changes your life expectancy. And we would teach the audience how to do this calculation. It's really simple. And each time you would update, oh, this person has 20 more years left to live, 30 more years left to live.

The lights in the cubicle would change. This music would start playing and your printer would turn on and print out a picture of that person's face and a moment from their life that was really personal, like a father watching. So they're not doing their own actuarial calculation. Sorry, sorry, sorry. So you're doing your like normal, boring office work. You're getting a couple of emails from your coworkers that are like,

making some jokes or kind of squabbling with each other. Like I also wanted to capture that feeling of being CC'd on an email you weren't supposed to receive because that's something incredibly human that I, again, I couldn't figure out a way to represent that on stage, but I knew as someone sitting down at a cubicle that,

That feeling of like, whoa, this wasn't addressed to me, but I'm going to scroll down and read all of this. But it's kind of interesting. Yeah. So I wanted to tell a story that way. And so this life expectancy calculation keeps coming up and you keep doing it for people and kind of having this realization of, oh, this...

That person only has X number of years left to live. And there is a you asked about branching. There's a script that based off of how well you do this job, you get different stories if your boss appreciates you or doesn't appreciate you. And then we have a kind of proposed. There's a little moment where you're presented with a chance to like.

Do you want to calculate your own life expectancy? And people, when they hit that part of the show, they freeze. Some people don't want to do it. Some people do do it. Some people spend 10 minutes there going through themselves, their whole family, all of their friends, calculating that stuff. And then the piece ends. So it always has the same ending. I'm kind of ruining the show now where the,

the desk that you've been sitting at, you find out that you're just replacing someone who's been laid off. And depending on how well you've done your job, you get offered the chance. Do you want to keep doing this job? Do you want to stay? Or do you want to kind of move on into the,

next thing. And so it takes a huge emotional journey for an audience member, right? It's a very personal, really powerful thing to both think about your own mortality, to think about other people's mortality. But at the same way, or at the same time, it is what

When you describe it, it's deeply boring. I say to people, yeah, you just sit in an office cubicle and you do Excel calculations for an hour. It's amazing. Okay, so just for painting the picture, there is nobody watching this. You or somebody in the back could be watching the terminal and everything to make sure everything is working. But there is no other audience. No. And I also am very interested that you refer to the person in the cubicle as the audience because you could easily –

have referred to them as the star of the show or the actor, right? Sure, totally. So fascinating. Okay, so many lessons to be taken and does sound moving. And just when you describe the printing out or telling the good news or the bad news about somebody who they may have just formed a little tendril of a connection with, and then you're kind of saying, well, let me tell you the end of the story before you get... Okay, so that sounds fabulous. And that was, you said, quite a while ago.

I mean, I made that show. I had my crisis in 2011. I made that show in 2014 and it has been running since. Um, the next time we're going to do it, it'll be in June in Munich. Um,

And so 10 plus years. And so that leads me into my next topic, which is kind of obvious. 10 years ago, we did not have the kind of AI that we have now. And I'm going to guess that these new large language models and other aspects of deep learning and AI have also caught your attention. And what are you making of them? What are you making of them? And how are you processing the opportunities that AI might be giving you in your work? So,

Thank you. I love this question. I've been kind of – I mean I'm still trying to figure it out. It's the most honest answer. And it seems like everyday stuff changes. And so things that I couldn't do two months ago, I'll wake up and read a paper and be like, oh, well, I guess I know what I'm doing today is figuring out if I can do this thing.

I've been telling colleagues that most conclusions about AI have an expiration date of three to six months. And so you're allowed to have your opinion and you're allowed to say it stinks at X, but you can only say it stinks at X for about three to six months and then you better check to make sure it still stinks at X. Yeah.

Well, so the thing that I'm trying to do because it's changing so much is to think about, okay, what are the ways in which I can use this in a theatrical performance that doesn't replace the interesting work that a human can already do? Right.

Or what are the ways that it can augment the work that a human can already do as opposed to just do the whole thing? And that's been amazing because what I quickly learned as I spent a lot of time experimenting is like just putting large language model outputs or using image generation with like not much of a prompt is actually pretty boring. And it doesn't make for particularly interesting art. And so what I've tried to do is I kind of built a like

close to real-time image generator that takes video or images from a live performance that people are doing on stage in front of you, runs it through an image generator and then alters that slightly. So you'll be able to still see the actor on stage and then up above the stage, a projection of the kind of like AI dream version of it. Yeah.

And so they're still keeping the actors on stage. You're still letting them do the story, but then you're kind of adding this other layer to it that feels like a secondary story or that because the image generation is so interesting and so good, it can kind of pull

or reframe things that you kind of already see on stage and see it in a totally new light. Do you inject, as you're doing that image generation, do you inject a perspective or a bias based on what's going on? Like now,

Like, now make them ugly. Like, they're all beautiful people, but they've just said a bunch of very ugly things. So, like, make them ugly in the... Or something like that. You can do stuff like that. You can... You know, because I've been very careful about, like, avoiding kind of intellectual property. Like, I'll turn them all into Greek statues. Make this...

And seeing look like a cityscape at night. And exactly, you know, and, and so it's a way of kind of like using AI to really quickly reframe the thing that's happening on stage. Um,

So that's like one thing that I've been doing. And I can imagine that being powerful. Just based on your description, I can imagine that affecting my, as an audience member, my interpretation and understanding of what's happening with the humans by seeing this kind of projection of what's happening in this fantasy but related to reality space. Exactly, exactly.

The other thing I've been exploring with has been kind of using LLM texts as kind of moments where we could like add a little bit of additional script or script that's based off of something that's happening today, right now in the room to kind of quickly rewrite things. So like in a traditional theatrical performance, the actors spend a month memorizing the lines that are written by the playwrights or, you know, and they're very precise about

I'll add lovingly written by the playwright. And I was like, well, that is a beautiful, wonderful, I love making theater that way. But what I did is built another system that uses kind of in-ear monitors. So like what you're wearing, Russ. So there are these very small things where you're

Wow.

That allows them to still be looking at each other, not to hold a script in their hands or to be reading off cue cards, but just to listen in-ear and say the thing that they're hearing. So is it fair to call that some kind of augmented improv? Yeah, I think that's a really great term for it.

And how does it go? I mean, so, okay, now we're going back to my question about like humans doing things that they're really good at. I'm just going to speculate that there are people who are going to be really good at accepting that and just smoothly integrating it into their character in a way that makes you say, holy cow, like I can't believe this person is doing that. And then there are others who become useless, confused, disoriented, and just...

not very entertaining. Is that what you're seeing? Is this now creating new pressures on human actors to acquire skills that they've never been asked to do before? Yeah, a little bit. I mean, it's actually like a pretty old technique in the theater world

But it was just more of like, if there was a certain actor who really couldn't memorize their lines, you would give them an in-ear monitor and then someone backstage would be whispering the lines to them. So that part is, is like that. We've seen this before. We've seen this before. Yeah. More of like, I wanted to say, well, what if we use a, an LLM to kind of put some of that stuff in and I, you know, like that's the other thing that I think is really complicated and interesting that I'm still struggling with, with the,

with incorporating AI into a theater is that when I do Shakespeare or when I do, you know, like a play that's been written a hundred years ago, the story is the same night after night and everyone is prepared for that. If I start to add these kind of like

both LLM text or these like image generation videos, those things change and they change constantly and they aren't consistent at all. And I think that's kind of beautiful if they're not consistent. But then there's that question of like, well, if it's not consistent, will it still be good night after night? Right, right. The New York Times reviewer may have gotten a good or a bad night.

Exactly. And so it kind of questions or it shakes up what is usually like the very strong foundation of what we say is good theater. And instead it adds this like maybe like other live aspect to it or it makes everything feel shaky, which can be really exciting. Like I watch these actors do this and everyone is like, oh my goodness, what's about to happen? And at the same time,

Sometimes it's really great and sometimes it's not. But my brain is flashing the word jazz in all caps, in all caps. And I think that you have some precedence where, hey, this can be spectacularly magical and every now and then. But as we get better at it and as we learn the rules, we can try to make sure that more often than not, it's just beautiful jazz music and not cacophony.

Exactly. And that's the thing that like with some of these productions that I'm working on right now is actually about like, how do you train that set of skills and how do you learn how to accept these things and really kind of view these AI components as tools?

co-creator or an assistant working with it as opposed to I want this thing to do all of the work for me which just never ends up being all that interesting. So we could go on forever but we're going to run out of time so my last question is the issue of disclosure to the audience. So for this example, for what you're talking about with the augmented improv, do you tell the audience because I can imagine a big long

Yeah. I, for me, like, I think it is better to say that up front because then people actually have an appreciation for it. And the way that we're working with it right now is we...

ask for the audience's input. So we bring them into it. And that kind of, I mean, it's a little bit of like, it makes them be like, oh, yeah, they really are doing that as opposed to, oh, this is just. And you also kind of brilliantly get them to be allies with the whole experience. Yeah, they have that experience. I mean, and I think that's one of the things we did with Temping and some of the other Interactive

components that I think is really beautiful and can be really exciting for an audience member is when they see themselves in the performance where something that they said or something that they felt or something that they did shows up back that allows them to be reflected back in that because for me what actually matters what deep down is important to me about making theater is like the audience's presence is crucial less about the actor's presence but

but the fact that you showed up to come see this art today, that's the thing that I'm really interested in. So if you see yourself on stage, if you see some element of something that you did, that's the heart of the theater for me. - Thanks to Michael Rau. That was the future of live performance. Thank you for tuning into this episode. Don't forget, we have more than 250 back episodes, so you can explore the future of everything for a long time.

Please remember to hit follow in whatever app you're listening to. That'll guarantee that you get alerted to all of the new shows and you never miss the future of anything. You can connect with me on many social media, usually at RB Altman or at Rust B. Altman on Blue Sky, Mastodon and Threads, and of course, LinkedIn. And you can also follow Stanford Engineering at Stanford Engineering or Stanford ENG.