Interviewing users with Steve Portigal
39:25 Share

Season 2, Episode 18. That problem, which is an organizational one and an institutional learning one, is addressed or is hoped to be addressed with a software solution without asking some larger questions like, what information do we need to save? Hello, I'm Pat Expo. And I'm James Roy Lawson. This is UX Podcast. We're in Stockholm, Sweden, and you're listening to us all over the world from Somalia to Bangladesh.

Today we are chatting with Steve Portugal, a vastly experienced professional in the user research space with over 25 years of experience interviewing users and guiding organizations in building and maturing their research practices.

Steve has written two books, User Research War Stories and the classic Interviewing Users, which is now in its second edition, which we are going to talk about today. Interestingly, well, I think it's interesting, when the first edition of the classic book Interviewing Users came out, we didn't actually talk to him about the book.

We didn't, no. We talked to him, though. We did talk to him around that time, but not about the book. So what makes this especially fun is that we're getting to talk about the book when it has its second edition. You are releasing a second edition of the book. And so you have all this experience over time. And you've probably reflected now on the fact that even something that people think of as asking questions to people, doing that has changed over time. So...

Talk to us a little bit about how has interviewing people changed in the past 10 years? Right. 10 years is the gap between the first and second editions of interviewing users. And, you know, I'll just say, back up a little bit and say I was resistant way back when to doing a second edition because my...

thought was, well, nothing has changed about the person to person activity of interviewing. So like, that's what the book is about. It tells you how to listen and ask followups and, you know, body language and tone of voice and all that. That's all in there. So it's, it's, um, you know, it's evergreen. So why would the book change? Um, yeah. Um,

Yes. So, you know, the book stands. And, you know, it's been a successful book for a long time, first edition. So, yeah, why would it change? And I guess the short answer is that, you know, that doesn't change, but the things around it have changed. The book is not just about asking questions, right? It's about doing interviews. It defaults to or assumes a context of questions

doing research in kind of a business context to learn from people to make an improvement to the product or service or user experience. And that is what's changed in 10 years, right? The context in which user experience is practiced or user experience research takes place. So for example...

You've got to go back a little bit further than 10 years, but at one point, this work was done entirely by consultants. There were no in-house teams. As you progress over time, you start to have more researchers inside the organization, but you don't have...

So it's not sort of a, it's not a function in a corporation that is managed. That means that, you know, how it's structured, where it's placed in the organization, how individual researchers are given tools, how they're given encouragement or support for their careers or developing their skill set. That's not being handled with sort of the,

special attention and care that would be placed to a discipline that's seen as itself. So if you have user researchers reporting to a head of design, for example, that makes for a certain kind of advocacy for research or championing of it or building up what that capability is. As you start to see

research leadership being seen as a thing and people with that title and that responsibility, then the manner in which researchers practice, and again, asking questions and asking follow-up questions is still the same,

But, you know, working with stakeholders and colleagues to understand what the need is, you know, collaborating during the course of a project, knowledge sharing, delivery of information, having an impact, all that stuff is very different than when it was just consultants. Now it's primarily internal people. And then I think an evolution beyond what I'm describing is,

You have lots of people doing research who don't have that in their title, don't have user experience in their title. The term that Kate Tauzy coined is PWDR, people who do research. And that's such a great term, I think, because it sort of says like, hey, you've got researchers and you've got people who do research and they both have different kinds of needs.

Um, so I think, you know, 10 years ago, we just didn't have that model of sort of who was doing research and where were they and what do they need and what are they concerned about? So that larger context, I think, is something that's changed. Um, and I'll just, I'll add one more thing here that, um,

Even these sort of fundamental human activities of listening and asking questions and so on, I, as the author, I have 10 more years of experience in doing it, making all kinds of mistakes while doing it, reflecting. I know I've had those things where it's in the book.

And then I do it wrong and I go home and stew about it. Then I realize, oh, yeah, I'm now failing to practice my own best advice and realizing how that screwed things up. So, Steve, I mean, just picking up on that. Yeah. Given that you have now 10 more years of research experience, what's your biggest take home from those 10 years? Yeah. What's my biggest take home on those 10 years?

You know, I think and maybe this is just age in general. I don't know if this shows up in the book necessarily, but just thinking about the way I work with people as a researcher myself, but also as a trainer or coach or whatever, helping other people. I think I started off.

and this is not your question, but you can... This is not your question. I think I started off with the belief that, you know, I can articulate the best way to go about something, so I'm going to pass that along, and people are going to go and do it. But then I guess, like I said, I make those mistakes myself. And I think what has changed maybe in just my overall dialogue with people is...

is helping people with their confidence and maybe even a little bit of acceptance. Uh, that these are, there's just mistakes that we make that are sort of the, the conflict between human nature and how we converse socially and what you need to do in an interview. Um, and I think I'm more accepting of the fact that there are lots of, there's lots of

right ways to do things. There are just so many different paths that any interaction can follow. We all make mistakes. We can recover from some of them. But I think, you know, we have to forgive ourselves for these things. And, you know, I think like I'll find myself in a training situation and people will describe something that's really hard for them. Like I have a bunch of questions on a discussion guide and I don't really know which one to go to next.

And they kind of bring that question up. And I think maybe their hope is that I can say, you know, in the first eight minutes you would do this and like I should give them some formula. And I think what I end up doing, for sure, there are some practical tips that I can always give to some practical question. But I think I also want people to hear that.

Yeah, this is, you described, you've got it, right? That struggle, where do I go next? What questions do I ask? How do I, do I keep going on this thing or do I move on? Is there more? You don't know. And so I think it's a more elevated way of thinking about what the interview is. It's an exploration with no map, right? And you've got your little...

your little flashlight, but you don't really know where you're going or how to get there. And so that means, yeah. So it really, it's a journey of the self-discovery, even just being a researcher, you're learning about others, but you're learning about yourself along the way as well. And you're learning that,

I can master this approach in a different way than Steve does. I think, I mean, I'm hearing here, it's like that difference between thinking you can or have to fix something compared to improving over time. Yes. And the fixing is sometimes about the other person, right? I think the naive research question, the naive researcher asks,

me or you, hey, how can I get this other person to stop doing this or to start doing this? And then the more one progresses, there is that self-discovery like, boy, it sure is uncomfortable for me as the researcher when this other person does or doesn't do this or that. And what am I going to do with that feeling? And I might have to sit with it and be uncomfortable. And actually, that's OK, because there is no formulaic answer.

So, again, you know, you're asking me, like, that's a personal takeaway. I don't feel like, oh, if you read the two books side by side, you would see... I don't know that that's in there. I haven't really... You're asking me a question I'd never really given a voice to, so it's not like I wrote the second edition trying to convey...

I guess a somewhat more nuanced take on what the goal is.

I mean, I think it's in there as I'm just thinking about how I explain this example or this tactic and so on. But, you know, it's not it wasn't in the book proposal, for example. I didn't set out to say, hey, I used to think A, now I think B, this book is going to revisit this topic so that we make it clear this. But, yeah, I agree.

Thank you. Yes, it's written from a, I'm 10 years older, I have 10 years more experience. I'm writing from that point of view, or I'm rewriting or revising from that point of view. Thinking about changes that are in the book, I mean, you've added two more chapters this time around. And you have a ninth and a tenth chapter.

what's the story behind those two chapters? I mean, why, why were they not there in the first place and what led you to actually add them? Yeah. Well, you asked, I haven't said what the two chapters are though. I haven't even said that. Right. The two chapters are on, uh, uh, first analysis and synthesis. Like what do you do with your research data? And second, um, you know, how do you have impact? How do you influence the organization? Um,

Those were topics that were addressed, but they were kind of, in the first edition, they were kind of like, all right, everybody, time to wrap this thing up. Let me just kind of acknowledge there's more. In some ways, that was an artifact of what does it mean to write a book? Like I had never written a book before and sort of starting off with a lot of experience and a lot of

I'd been doing the work for a long time before I wrote that first book and I had information a bunch of different places that I had to sort of organize into this book. And so how do you draw that kind of dotted line around what's in and what's out?

And I made probably the, I don't know. I mean, I don't want to sound regretful about it. I made the choice at that time that like, yeah, that's, it's about interviewing. It's in the title. And so, you know, all the stuff that happens after is important. I want to acknowledge it, but that's a different book. Like maybe I would write a followup book and I would write that book. That was kind of my way of rationalizing the,

yeah, I don't want to get into this. Or maybe less selfishly, like, I don't think it fits within sort of the remit of what this book is supposed to accomplish. But I think early feedback said, like, hey, where is this stuff? Like, where's the thing about what to do next? So it was clearly a gap or something I wanted to address in the future fairly early on.

I was wondering, though, whether the context of the time, 2013, like you've already mentioned, where we're looking at more agency-based models and outsourcing research, and now it's in-house and in-house teams. Is that something maybe that...

that led to some of these more, what do we do afterwards topics, not really maybe getting the focus that now we seem to think that's the course to get that focus. Yeah, you're right. I think if these two chapters, analysis and synthesis, was something that I sort of

felt like was outside that now I think is really important. Um, and this, this, this discussion of having an impact, um, yeah, you're right on. It represents the change. It's, um, a thing that comes up more in workshops. It comes up more in my own work that is more germane to, you know, these in-house teams and these in-house leaders. Um, so yeah, I think, um,

It was less than obvious a mission 10 years ago, and it's clearly a necessary... It's the topic that everyone wants to talk about. Yes, yes, yes. But my stakeholders, my team, how do I kind of have this impact? So it's something I've certainly been thinking about for 10 years and talking about and writing about and applying myself. And, you know, I had more to say now. So I think it was easier to...

it was easier to kind of commit to having this be something to talk about because I guess I practiced it differently than I did 10 years ago. As you were describing this shift to in-house teams, I almost felt...

this sense of jealousy in that in-house teams then get to work with research over time in a way that I as a consultant cannot and they have a shared experience and they can even reflect back on research they did two years ago based around the same product or service so that that has to be a lot different I guess when when you talk to people about research and how

you're doing the same type of research around the same type of product service for a long time. Right. And as I'm a consultant myself, right, and I see this with my clients that they live with some space, they live with some product, topic, set of stakeholders, set of users. Some, I've talked to researchers that

meet with the same set of users over years and they, they build up these kind of longitudinal relationships. Um, and I'm jealous of them too. Um, I'm also personally happier where I am. I think I bring value not being in that, right? I think, I think when you get, um,

When you sort of, there's a rut, I think that is easy to fall into. And the rut has all these wonderful attributes, like knowing the people and their limitations. Like if you have their limitations, their strengths, their preferences, if you're working with a bunch of different stakeholders and they have different communication styles or they, they have different ways that they engage or different availability, you know,

You, the in-house person, do not have to figure that out every time. I mean, yes, as the team changes, you're going to be constantly adapting your styles. And there's even a mention of this in the book, I think, because one of the people I quoted describes how they...

try to have a communication approach that is inclusive but spread enough to sort of deal with the different needs and expectations of everybody. And so, yeah, right as consultants, we come in and we don't have the lay of the land. And I think sometimes that's an advantage because you don't

I think we're in... This may be arrogant sounding, but I think we have the responsibility to speak truth to power. We also have the opportunity. We're a little less constrained by risk as consultants. And not as biased maybe as well. My compensation isn't tied to the success of the product. Those models create all sorts of interesting incentives that...

You're part of an organization. You're part of a corporation, if that's the domain that you work in. And your success is tied to its success. And as a consultant, it's not. Yes, the company does well. They hire us back. We want things to do well, but I don't think we have the same kind of incentive model. So I'm glad there still are consultants because I think there's a nice...

there's a nice triangulation or a nice partnership that can happen. I love working with somebody that has the long view inside and they can kind of give me the highlights of who and what. And, you know, I don't have to do everything my way, but, you know, we can negotiate kind of approaches and practices. And, you know, I can be that unbiased voice just by the fact that I'm not part of it.

Yeah, exactly. You've got that, you know, the fresh eyed approach that you can cut, you can offer us as a consultant coming, coming in. But, well, then we've got the, the opposite edge of that. And you mentioned this in, in chapter 10, making an impact that, you know,

Inside, internal research organizations, they need to keep track of what they know or what they don't know and what have we already researched. And that opens up a whole different aspect of historical record keeping, I guess you could say, which as a consultant, maybe we didn't have to deal with that. It was an assignment you were given and then you delivered. Yeah.

I will say, and I've got to imagine this has happened to both of you though, where as the consultant, you serve as the kind of the offshore institutional memory, right? Where somebody writes you and you haven't worked with them for a really long time and they'll say, hey, didn't we do a project about this or that? Do you know what it was? Do you still have the thing?

And that happened to me like, I don't know, six weeks ago. Someone had an anecdotal memory of something and they weren't involved and no one was left working there. So that organization wasn't doing a great job of sort of documenting whatever, different initiatives and so on. And that they were, you know, I could put my finger on it like I actually had the document quickly. Yeah.

So, yeah, it's, I think there's an ideal of whatever knowledge management, institutional memory. Um, what have we researched? What do we learn from it? Um,

Yeah, I think it's a hot topic that people are working to kind of try to define. I think what I get nervous about is where that problem, which is an organizational one and an institutional learning one, is hoped to be addressed with a software solution without asking some larger questions like, what information do we need to save?

Like, is it the existence of that report? Is it the report? Is it the person who, you know, could kind of talk you through it? Is it the raw data? Is it the decisions that we make? Is it the recommendations? Like, you know, that's just me riffing on like, what could we, what do we want to track? And who's going to kind of query that information? Is it somebody that is, um,

you know, quote a researcher or is it, um, someone that wants to, you know, Hey, do we know anything about X and like, and, and, and just kind of discover it themselves. So I think there's just a huge amount of, of, of challenges there. Like what do you, what are you creating different things to archive? Uh, are you archiving them in a way that they're retrievable by who, at what point, um,

Just a number of years ago, people in research organizations that were growing were talking about bringing in somebody who is a reference librarian, whose job it is in other contexts is to be a human being that interfaces between people who need information and storage of information. Yeah. A librarian. And that's very different than self-service. So this is like a huge kind of...

Culturally based, because there's no, right, you know, any two organizations are going to deal with this differently, what they expect people to be able to do and who there is to do it. There's a technical need here, but there's also just a process and an understanding of what information, why do we want to sort of look back to what end and building those use cases in.

And I think it's turning out to be trickier maybe than maybe software vendors had kind of promised us it would be. Yeah.

And my biggest worry now is that people will just give up. They'll just load it into the database and now they'll ask the AI, can you bring me the most important points from that research? And that's, of course, already happening. And that brings another aspect into this whole picture of storing that data and that is something that's changed now in the past 10 years. And you actually mentioned in the book as well the GDPR and privacy and how people are becoming more aware of privacy, how privacy

People are more aware that they have to consent to things. So I think that introduces an aspect that we didn't pay as much attention to at least 15, 20 years ago. And so how do you think that has changed how we do interviews now? I was just going to add a pair there. I mean, I think 20 years ago, maybe we would focus on an NDA. So the company's privacy. Right, exactly. But we wouldn't care less about the individual's privacy in many situations. Yeah.

Yeah, and this is, I think this is, you know, we're sort of talking about how the way in-house teams work and in-house researchers work. You know, we didn't have research ops as a discipline 10 years ago. And research ops as a part of the organization that can

build those tools and processes for, you know, whether that is about the archiving stuff that we're talking about or, you know, what's the compliance required workflow? Because I think there's all sorts of legal aspects. And so the legal department of an organization, and I'm speaking very broadly, but

I think in the past, maybe they were someone to be avoided. And in the ideal situation now, they're a partner so that legal can help, research ops can help. How do we set up whatever documentation, storage of signatures, storage of material? Do we have to anonymize these things? Do we have to expire them after a certain time?

that stuff that, yeah, a while back we would just ignore or just hope it would be fine. I guess the good thing is that companies that have customer data of any kind have processes in place, so it isn't necessarily on research to invent those in the first case, although I think research leaders and research ops leaders have to help

like compliance people sort of understand, no, this, this is a different set of data. And so we do need, here's what the needs of research are and here's how they're different from, I don't know, let's say a marketing email blast program or something like that. Uh, or, um, you know, anonymize usage logs that we can use to, to find bugs or see when there's, there's downtime that this, this something like an interview is a different, it's a different piece. Um,

And I would say just this is very impressionistic. I think there might have been some growing pains in figuring that out. But I feel like when I talk to research leaders and ops people, they have more successes than non-successes in setting that up. That there is, you know, if you believe in research...

then you need to create the conditions in which it can succeed. And so figuring out, yeah, what do we store data in? What are we allowed to do? What happens to it? Who's responsible for that? Um, you know, I think has been a, a growth. It's, it's, it's a maturation. It's one of the axes of like the maturity is, is how sort of sophisticated that is. Um,

Because that's also true if you don't help people feel safe. And because they are aware of their privacy now, you need to help them feel safe to be able to have them give you what you sort of require from them in that they share their information, they share their experience. And at the same time, you have to do that in kind of a...

human-centered way or a non-lawyer-facing way. So, right, there's a lot of, you know, nine-point type that is, you know, required language for something like that. You know, I like to see researchers giving people the information that they need that say, like, to set their expectations around what

what data we're collecting and why, like stuff that a good consenting process says. Here's what we're collecting and why. Here's what's going to happen with it. Here's what your rights are in this interaction. And you can do that in a way that's like legalese, that's not very friendly. Or you can do that in a way that is very, yeah, compassionate and helpful and is good for the interviewer

to set up the relationship in a kind human to human kind of way there's an example that I mentioned in the book that the Sesame Workshop the Sesame Street people did this piece of research where they well we've talked about this haven't we we've interviewed you ended up interviewing them I'm sorry it's thanks to you Steve that we actually heard about them and got in touch so we're grateful to you for highlighting it

So the folks at Sesame Workshop, the organization that produces Sesame Street, were looking at how to provide consent in, I think, low written language literacy environments. I may have some of the details wrong, but...

They created a bunch of like Muppets videos that would have a little scene with a family interacting. So adults and children, because that's also the situation where you have, you know, a mix of people that can consent and people that can consent on behalf of others. And when they made these videos in like all these different languages, they were all

And I thought that was just like, it's cool to see Muppets doing anything, of course. But to take something that could be as dry as this document that gets created with all this awful legal language that me as the interviewer, I hate starting off my relationship with somebody by putting this in front of them and then trying to crawl back up to kind of some, you know, empathy-based human-to-human relationship.

creative discussion. So just the creativity in that and like, wow, here's something as dry as consent and here's how much creativity can be put into that. So those folks, I was just super impressed by that as just a possibility of like...

There's a lot more that I think we can do in some of these operational processes to make them. I love that because it also brings attention to the fact that not everybody understands what consent is. You can't use that word all the time, which means that you have to be more informed in different ways and find different ways to... You have to adapt to your context and find the right angle to come in at. Steve, I've got another question for you to wrap up with, actually. Okay.

If we came back in five years to have this conversation again, what would be different? There won't be a third edition in five years. So we can eliminate that. Um,

Yeah, I mean, I think we've talked a lot about the organizational context of research, and that continues to change. We're talking right now in a moment where there's a lot of stress and uncertainty, and certainly in the...

the tech environment that I'm in, in Silicon Valley in North America, there's a lot of anxiety. Um, a lot of people sort of throwing up their hands and saying, um, you know, the, it's the end of design thinking, uh, it's the user research moment of reckoning. Um, so I think there's confusion based on, uh, like, yeah, lots of layoffs. Um, and, uh, you know, is, is,

you know, I think we, it's, we, we, we don't want to be alarmist about it, but we also don't want to be sort of, um, pretending that everything is fine. And I think, you know, historically like pendulum swing back and forth, um,

But I think when you're surrounded by people that are laid off or you're the only one left on what was a 10-person team, I think it's easy to feel under threat right now and that there's an emotional impulse to answer your question and say, well, we wouldn't have this conversation in five years because we're just going to be the smoking ruins of a former practice that doesn't exist anymore. I think that sentiment is there. And if you sort of give in to your dark fears, you can feel that.

So I'm telling you what I don't think it's going to be. I mean, I think the pendulum will swing back. I sort of have to think that because I'm not making, you know, plans to get into a professional bunker. I think this is important work that people still care about. So...

I think this... We've talked about sort of the influence, the having an impact. And yeah, organizations change. And we haven't used the democratization word. That's like a word that excites and upsets a lot of people. And I think to me, it's Kate Towsie's people who do research and researchers. So who does research and what do we mean by research? I think it continues to evolve as

Um, and you know, we are making trade offs and compromises as we always have. Um, so I don't know. I think I'm going to acknowledge the, the, the, the fear that there is a, um, you know, we won't be here in five years, but I think things change more slowly than that. Um, and that, you know, I,

So I'm still avoiding your... I don't know if I have a concrete what will have changed. You can't answer it really, can you? I mean, it's a future question. So there's no right answer or wrong answer. It's an impossible one. But I personally, I agree with you. I mean, five years is not that long and I think there's still a lot of things...

The pendulum will swing. And I think about the times where I've taken developers to observe on research sessions, and it's the first time they've ever met users of their products. And I still dream of the day where that's not the case, and every developer has seen a user use their product. So if we look at that as a bar...

then we're years away from that being the case where all developers have had some experience of seeing their product being used and researched. And those developers, for the most part, in my experience, they want that. They're not saying like, I don't care, I don't need this, this isn't important.

And so, and it depends on the discipline, but those folks are probably not people who do research. Those are people who go along with research or people who observe research. I don't want to make up another acronym to go with that. But so there's a need right there, right? It's just a very, almost a tactical need is just providing exposure. And that's not going away.

So what is sort of the mix? I think the mix may continue to shift. You know, who's doing it? What are we doing it? What are we trying to accomplish? Even just exposure is sort of a low. That's not even, you're not even talking about evaluative or generating new ideas or, you know, finding the future of the business, but just,

Exposure is such a big bang for the buck. If you can get that, you can make cultural improvements just at that. And of course, there's more to keep doing. So I think the need is there. The demand is up and down. The understanding is up and down. But the need, I think, is not going away.

And as you were talking about all those fears, all I kept thinking was that research is part of the solution because it's about human connection. It's about that curiosity. It's about being open to others and allowing yourself to be surprised about what you can learn about others and challenge your own biases. Yeah. Can we applaud that? Can we get in some applause? That's what I take away from your book. So thank you, Steve.

Yeah, that's a lovely way to put it. And yeah, again, I think that fear is we care, but no one else cares, right? I think there's a sense of isolation when you go through changes and crises. And there is a crisis, I think, in the field for a lot of people right now. But to come back, as you're saying, to these principles, this is what

This is what we, I mean, the biggest we, I can, you know, be an inclusive version of we. This is what we're good at. This is what we care about. This is what we bring. And we know this is what business needs. This is what design needs. This is what technology needs. And, yeah.

you know, long-term or medium-term, it's what I want to believe it's what will win out, you know, understanding people, caring about them and doing the right thing for them with this information that helps us make those decisions. I don't think you can kind of invent your way into, like consistently invent your way into success. It does take this

this, you know, understanding and caring for the human aspect of people. That's a perfect note to end on. Thank you very much, Steve, for being with us today and chatting. Thank you. It's great to chat with you both always. It's going to be all right. We can be reassured. I mean, we've reflected a fair bit during the chat and

talked about the ins and outs of user research and ebbs and flows of it all. But I agree with Steve here. It's not going away. We're going to need user research, and AI is not going to replace it. It's nice to just have that feeling of being able to be comfortable being uncomfortable. And that makes it all right. Yeah. And good, I think, listening now is a reasonably recent chat.

our chat with Meena Kodendaraman about asking the right question. Excellent chat we had in this series. So episode 12 or series 2, episode 12, episode 322, if you're going by the big numbers. And that's a really useful one to dive into straight after this. Excellent.

And if you want us to, James and Per, as part of your next conference, event or in-house training, we are offering workshops, talks and courses to inspire and help you grow as individuals, teams and organizations. Just get in touch by emailing hey at uxpodcast.com. And if you'd like to contribute to funding or helping with transcripts and references, then visit uxpodcast.com slash support. Remember to keep moving. See you on the other side.

So what is the most common job for spiders? I don't know, James. What is the most common job for spiders? Web designer. Oh,