32. Conducting Research with Employees (feat. Angie Li, UXMC, Senior Manager of Product Design at Asurion)
43:15 Share

This is the Nielsen Norman Group UX Podcast. I'm Therese Fessenden. Now, our last episode, we featured a member of our UX Master Certified community.

To be a part of this community, in other words, to be UX master certified, you need to take about 120 hours of UX training and pass 15 different course exams. Now that alone is already an impressive feat. But what's especially impressive is what these folks go on to do after they receive their certification, often extending their knowledge to others and changing the way that their teams do UX work. Our next guest is Angie Lee.

Angie and I crossed paths when she first started at Nielsen Norman Group years ago. But since then, she's gone on to do lots of great things, working for a number of firms, including GSNF, an advertising agency in Nashville, Home Depot, and eventually Asurion, which is where she works now. She's a senior manager of product design. In this episode, we discuss what her work looks like now.

what the difference is between customer-facing UX work versus employee-facing UX work, and the special considerations with balancing priorities, like getting really solid buy-in while also protecting research participants. It's a tricky balance to strike, but we hope that you enjoy this episode and us talking shop about UX. ♪

Angie, welcome. Thank you for joining us today on this horrendously rainy day outside. How are you doing today? Where are you right now? Yeah, I'm in Nashville, Tennessee, right downtown. I'm actually in the office, so I'm in the Asharian office in

Um, we have flex days on Fridays. So I'm one of the only people here. Sometimes it can be a little bit lonely or like, it's kind of weird to like, to have those echoey offices. I remember there were some days we had flex days at Microsoft back in the day and I would walk in and there would be nobody in any of the cubicles around me. There might be like some folks over there. There might be some folks over here. And if I happen to run into someone, it was like, Whoa, Hey, a person I know, thankfully.

I'm actually one of the people waving around to turn on the lights that automatically turn off. Yeah. You gotta just remind the building that you're in fact there. Yeah, that's me. Always fun. You're at Asurion right now. What do you do? How would you describe the work that you're doing there? So I am a senior product design manager here and we separate ourselves into domains. Right now I support the expert workspace domain,

which is helping our frontline agents with selling, with supporting on tech-based questions. Got it. So a lot of internal support and making sure that employees are given the tools, techniques that they need, ideally maybe improving their experiences. It's got to be a lot of fun and kind of interesting. Like how has that work differed compared to say some earlier work you've done before?

Or is it the same? Oh, I would say it's very different. I think it becomes a lot more complex. There's a lot of patterns that you can follow and learn and observe from public facing sites or consumer facing sites. You're also like the end user of things like, I don't know, on my phone, I might have like a Lyft, Uber, Instacart apps, all of these things that you can test and learn.

just by downloading and using them. When you talk about internal facing tools, you really have to get in the mind behind like the business needs and the goals of that individual. So I think it, I find them to be a lot more rewarding in that they're more complex. Got it. That's, that's gotta be kind of fun to go from kind of predictable problems to maybe less predictable or more complicated problems.

challenging research scenarios as well as design scenarios. So when it comes to like the user experience for your work, like you mentioned that you have internal staff, like what kinds of users and situations are you considering in this work? Yeah, I might've mentioned this. So, but what we call frontline agents are, we call them experts. So they're the people that when a customer needs help, um,

because their phone won't connect to Wi-Fi or they're having trouble with their Amazon Alexa. So when they call in or when you call in, someone has to be able to answer that phone. And oftentimes they're managing a lot. They're looking at the customer's account information. They're looking at their history as a client. So I help manage those tools so that they can do their job easily. And we're looking at

augmenting that with generative AI to make that even more seamless. That's exciting because certainly I feel like everyone's talking about generative AI and there are so many different applications of it.

There's generative AI creating imagery, creating designs. So I would love to hear if however much you're allowed to share with us, I would love to hear what that ultimately might mean as far as the things you're designing. Yeah. Yeah. So, okay. One of the things, one of the challenges just as a

client commitment and obligation is that we have a lot of sites locked down so that our frontline agents who we are calling experts can't access them. So I'll just give you an example. Like, let's say they can't go to a Reddit thread or like,

They wouldn't be able to go to a Reddit thread. They wouldn't be able to go to Google. But we still need to give them resources to feel empowered and informed to answer tech related questions about pretty much anything that has a power button. So generative AI allows us to have some boundaries and control, but also widens like a plethora of information that could come in seconds.

Yeah, that's really cool. So it's interesting that, yeah, people can't necessarily go to Reddit. I can imagine there might be some risk of discrepancy and information given to people if you're like, yeah, just find the info on Reddit. It's totally fine, right? Like maybe there is good information on Reddit, but there might be some need to, like to your point, control and kind of have a more consistent experience both internally

To the actual staff member, but also to the people who are on the receiving end. So it sounds like you have like interactions. You're not just accounting for, oh, what tools are staff using, but also the interactions between staff and employees.

customers or end consumers, right? And we want to protect their information too. I think that's really at the heart of why some things are restricted. It's unfortunate because, you know, you would think like, well, if we give them everything that they need to access, then they would be able to do their job more easily. But the control piece is protecting their

us as a business, protecting our clients' name and protecting the customer's information. So we don't want the ability to like take PII and put it somewhere in the cloud. So there's, you know, it's just pretty standard across a lot of tech companies now. You see it everywhere. Yeah. Yeah. The data protection piece is really crucial. And I can just already imagine how many people you have involved just by virtue of the things you've just said, right? There's

Creating designs for internal staff. There's creating designs for the customers who are interacting with this staff. There's generative AI that gets factored into this. There's also data protection. So how does your team organize all of this work? Like how is your org structure? Can you give a kind of Cliff Notes version of it?

Okay, so that's a great question because we just went through a little bit of a change where we're trying to be more intentional about how we partner with data science. So for a while, data science org has been

supporting and getting input from what we find out in the field and applying that into the generative AI models. But we found that there could be a more effective way that we partner so that there's less lag between all the communication channels. So one of the things that we've done is one product team, which I think is pretty standard. You would see a product lead, you'd find

a product design lead, and then an engineering lead, and several engineers to help execute the work. Well, now we have that same framework of the three, product lead, product design lead, engineering lead, and now a data science lead embedded in the same team. So that's pretty much

a snapshot of what one team would look like. It seems a bit more decentralized. So you have product design, you have product development, perhaps you have engineering, you also have data science. So...

They're all kind of working together as teams as opposed to being in kind of these larger departments. Is that correct? I mean, there probably still are. Are they? Yeah, they still all report up their verticals and through their org. But the fact that our product teams are working together and we meet weekly, we have standups together that really then helps us operate as one team, although we're representing like four different areas.

Got it. That seems like a much better way to kind of keep that consistency and that flow as opposed to, hey, we have a project and then there's this other project fitted in when you can, right? Because that can sort of stretch out project timelines naturally. Yeah. Yeah. I think it is good. It's brand new. So we're still learning how to work in the right sequence and cadence because, you know, you might have seen things like design ahead and then the design will be a sprint ahead.

Well, data science also needs a lot of lead time because the complexity of the models themselves. So we're still trying to figure out how to weave in the best practices to support everybody.

Yeah. Yeah, that's definitely something to factor in as well. And I know for me, at least for design, we have this perspective of lead time with research and also lead time with the actual design process and factoring that in. But to your point, data science itself needs a bit of time to work these models, which are very complex and have a lot of variables and things like that. So

Okay, we have generative AI, we have research with employees, and you've recently done some research with employees, right? Yeah, our team, all of the designers in this domain, there's five of us, went to Orlando recently to do some research. And it was really eye-opening just to sit next to our frontline agents, our experts, and look at them and how their day goes.

So that work, was it like, I don't know how much you can share of like what it's like to be an employee, but is this like you would go to a call center or is it like you would go to people's houses because people are working remote? Like, I'm just kind of curious what that looks like. So we do have both types of experts. We have our, and it depends on kind of like the,

the client and the effort. But for this one in Orlando, we were in a call center and we would sit next to people who, you know, they'd have their headsets. They'd have like their setup with the two monitors and looking back and forth, maybe something, some handouts and material to reference. So we really got to have a true feeling of their work because we have other tools. We might have screen recordings, things that they share, but it's just,

doesn't replace being able to sit side by side with someone and also witness because it's a call center, the background noise because people are having conversations and support calls all around them and kind of that distracting environment. Yeah. It's a witness like the actual context that someone has to do their job in and

I can imagine that being really loud. I mean, especially you've got calls going on all over the place, right? And everyone's sort of trying to resolve the issue, speak at the right volume so that their customer, whoever's on the other line can hear it. Right. But at the same time, maybe not too loud. So they're not like disturbing their, their next door or next, you know, couple cubicles over whatever it is.

So yeah, how would you say that research is different from research you maybe have done with customers in the past? So research is different when you're dealing with internal employees or experts or frontline agents because they often work with trainers, with managers about performance. And sometimes there are audits because we have client commitments to doing the right thing and compliance. So

There's also the risk that you are perceived as somebody who might like, you know, report on bad behavior or tell on somebody. So I think that's,

Making sure that you go in sharing that your intent is only to make improvements to their day-to-day job and observing the tools that they use, that they don't use, is really different than when you're working with an end consumer. I think it's pretty common practice to understand what a focus group or a user test is, and they have no relationship with you as being a coworker.

I can totally see talking to someone, a customer, I'm going to say like, hey, I'll give you $50, $75 for your feedback. There's no reason for them to feel like they're going to hurt your feelings as much as someone you work with. Yeah, I can imagine the stakes a lot higher when you're working with coworkers. And like you're saying, there's often this context of audits where people are observed and have

gotten used to being observed, but not for the purposes of I have a say in the design that gets updated, right? It's usually more, oh, I'm observing your performance, right? And that can be super stressful.

And people might feel the need to perform during this usability test or whatever sort of research session it is, even though maybe that's not the intent. Yeah. Yeah. I think I've noticed that. So in the beginning, if we're just warming up to the conversation and it's a one-on-one interview or discussion, there's sometimes...

that rapport building needs to happen in the beginning because there might be a tendency to go like straight off the book. Like this is what I do and this is why I do it because I have been told to do this thing. And instead I'm like, you know, do I maybe share a little bit about myself, my goals and try to make sure that everyone is feeling comfortable and safe to share and reveal a little bit more.

Yeah, I can imagine that being super helpful, like maybe that added warm up time that maybe you wouldn't normally need in a context where it's a customer you'll see once and never again. And, you know, or maybe you do see them again, but generally that relationship is going to be a little bit different, right? A little bit more casual versus this one, which is, you know, perhaps different.

going to need that trust really well established in order for that honesty, in order for someone to admit like, okay, this is what the book says. It's not what I do. Right. Yeah. Yeah. Sometimes these practices, these are good practices to have. Building the rapport is great for internal employees and for customers themselves, but it's even more so invaluable when someone feels like you're going to report on them or they're going through an audit because an end customer doesn't feel that pressure. Yeah.

Right. So as far as like setting up studies for employees, like when you're actually studying these internal experts, what would you say is like a common rookie mistake that either you've seen other researchers do or that, you know, maybe you yourself have done and you kind of learned from it? I'll tell you one that's

I think pretty tactical and just like a logistics thing. So when you're setting up an interview, you often have to reach out to a few people because depending on how a frontline agent is paid, they might be paid per call. They might be paid per hour, but they need to carve out that specific training time for this call. So there are maybe two or three people involved before you can even get there.

Now, one of the risks that people run is that invite just continues to get forwarded to a few other stakeholders. And then suddenly you've got, and I have my product team with a data scientist product lead and engineer lead. So now we have 10 people invested in hearing this person share their insights and perspective. And that's just not the right environment to engage.

To feel like you can have an open dialogue about what you like and what you dislike. So I'd say that's one of the things that I've learned. I've made that mistake before, not knowing that things would just keep getting forwarded. So trying to manage the people who are in the invite and finding other ways to distill that information out.

Yeah, I've been there before where it's like observer sprawl, right? Where it's just people. And on the one hand, it's a good problem because it's like, wow, everyone is super interested in what is being said here. They're super interested in making things better. And that sort of buy-in is wonderful to have. But like you're saying, it's not really going to do wonders for honesty and openness and vulnerability if there are 10 other people that this person doesn't really know very well.

observing. Yeah. I can see that as being really tricky. How did you, or how did you like kind of learn how to manage that? So there are a few things that I've done. I mean, I, I, if I set up the invite, I might make sure that then near the closest stakeholders to that individual, it might be there. We call them coach someone who they meet with train training weekly. And

That they understand that this is a one-on-one conversation. If there's anything that I'd like that they need to know, then I'll share back separately. So it's a whole bunch of like relationship building conversations that people understand the process and the intent. I have actually heard

someone said to me once like, oh, but I work so closely with my team. There's no reason that anything you ask them, you should be able to ask in front of me. And it was really about building that relationship so that they can understand like, well, you're their boss. And although you guys seem to have a good working relationship, you need to understand like our process is to do things one-on-one and I'll make sure to keep you in the loop. So I'd say a tip for

for someone starting out to try to set this up would likely being to manage those expectations, reach out to people, turn the forward, disable the forward option on the email invite, and then make sure that the agent themselves even know, like just because there's a calendar invite, what to expect. You might say no prep needed. This is really just about you and me and the design.

Right. Yeah. Just making it, making it a bit more insulated for the person who's participating, but also not necessarily neglecting or just not inviting all those folks because, you know, we also want them to kind of feel involved, especially if they have that buy-in to feel like they're,

getting some knowledge out of this and still willing to provide access in a way. Because like you were saying, you have to kind of get people to approve this time with these employees. And so we don't want to just say, oh, you're not invited and then cause this more adversarial relationship where now someone's like, oh, well, this team is doing research and I never know what they're doing and they're taking away from my projects. Right. We want this to be something where

those staff members feel like they're benefiting as well. And there's something in it for them.

Yeah. And we also have this concept of, you know, we want to be conscientious of how many people we're taking off the floor because that takes like a population of folks who aren't able to answer calls. And it depends on the time of day, like, or is that a high volume day and all of those nuances. So there's a lot of people involved. One thing that I've helped

to make people feel like they can have those insights if they're like not invited to the event is just to do a readout. So like taking clips of things or collecting themes and then sharing those out live. It's not just like, here's a report, take the report and do what you want to do with it. I'm inviting them then to a follow-up call so we can have a discussion about how we can make improvements together.

Yeah, that follow-up call has been really essential, especially for... I mean, granted, I'm in a different context, more in like a consulting slash agency type of context for a lot of our research projects. We do individual or independent research for our own courses and such. But when we do these...

consulting gigs, one of the key things that we tend to do is that follow-up call. And it does two things. One, we do want to show the findings and explain it, give a little bit more context other than here's a PDF, go read it. But also, we can put in these clips. And there's often this misconception, and it's one of the things where it's like, listen to what stakeholders do

do not what they say. Because I'll often have these conversations with these clients or other stakeholders and they're like, we want access to all the recordings because their intent is to perhaps watch it and to get some benefit out of it. But when you're interviewing or doing usability research with

13 people, that's at least 13 hours, if not more of footage. Like, is someone really going to spend 13 hours of their time other than the researchers, of course, like, is anyone going to really spend 13 hours of their time looking through all these clips? Probably not. Right. So if we actually want to make these findings more accessible and to make them more real to people, sometimes all you need is a few clips. You don't necessarily need to give access to all 13 hours of footage.

So I definitely, you know, agree and second that, you know, having that little clip, that short 30 second to two minute clip, however long you want it to be, but short and sweet and just gets to the point can be really, really helpful.

engaging and can really give that sense of empathy that can otherwise be really hard to come by if I'm just like, here's a quote that someone said. Right. Yeah. Yeah. And hearing it straight from, from that, um, either the customer or the agent that we have is a lot more powerful showing pictures of the environment, um, sharing like, you know, there's all this background noise and here's an example of what that could be. Um,

I would say when you're taking these clips, it's really helpful to theme them as well. So that's synthesizing the research because now instead of having to watch, let's say 13, like you were talking about 13 different video clips and then try to make sense of the three or two, five different topics, then you can say like, well, this is a top theme of their biggest challenges, challenge one, challenge two, challenge three, and everybody has touched on them. Hmm.

Right, right. And it's kind of pre-analyzed for people in a way, because otherwise we're kind of expecting whoever's reading this report or reading these findings to like come up with that themselves. And a lot of people don't have the energy to do that. It's a lot of work, right? I mean, research is a lot of work.

So, yeah. Now, what what comes to mind now is like, obviously, research is a lot of work and there are going to be groups of people that perhaps think, well, it's employees like we really need to do this much work with employees when ultimately our moneymaker or revenue generator would be our end consumers or end customers.

What has been your experience, not just at Asurion, but also like past experiences? What have you seen as far as like the appetite for research with internal staff? I'm a little bit conflicted because of the demand for Gen AI. There's also a need to work really fast, right? So we want to be able to be one of the first people in the market to leverage that and learn from things that are in production. But at the same time, I mean, they flew fast.

10 of us out and the entire design team here to Orlando just for a few days to be able to do some research. So I'd say that the appetite is very high. We just have to be able to make sure that we keep in check some of those balances of like we're getting people off of calls. So there's a lot of buy in, but we have to balance with our to be able to serve the customer.

Right. Yeah. And it's nice to hear that too. And sorry to put you on the spot for that question, but I'm only asking because I know that in some other organizations I've been part of in the past and I won't name them, but

But there have been some organizations where the appetite for research with internal staff is low because the idea is just sort of like, oh, they'll kind of get it or they'll, you know, they know the tool, they have to use it anyway. They'll figure it out. We just onboard them, right? We'll give them a PowerPoint slide of screenshots and they can just figure out based on the screenshots what they need to do. And, you know, on the one hand, I think

I get it because you may not have the immediate return on any investment, right? But on the flip side, there really is a return on investment that if we take some time to think about it, can make that internal research with employees worth it. Like what comes to mind is,

In the amount of time it takes someone to find an onboarding document, read the onboarding document, practice it, learn it, and then finally use this tool that's maybe very poorly designed, then...

In that period of time, we've wasted money because like you're saying, call center, just as an example, there's an employee who's got some task that they're primarily paid for and they're not doing it, right? They're doing this learning and this troubleshooting or button mashing to figure out how to use this tool, right? Yeah.

So in a way, it is costly, but it's important to do that research to make sure that we do save money over time, especially the larger the company gets, the more people have to do this thing and the more money that's essentially costing. Yeah. I mean, we have a fully staffed org just focused on training. Like they...

They create the training materials. They schedule the calls with all of the relevant folks to get them onboarded. And one of our goals is to help streamline that process instead of making it like an eight hour day. How can we do that? So how can we build systems that are so intuitive that they need as little training as possible?

Right, right. And yeah, it saves time too for the training staff. I mean, not to say that they're not doing their job, but they're doing it in a more efficient way and they're able to cover many more topics perhaps. Yeah, and they could focus on different things. Right. And so what comes to mind for me now is like domain expertise, because obviously people are training to get better at their job, not necessarily better at a tool, right? There's like a little bit of both that come together, right? Like to think about a designer, obviously a designer wants to get good at using Figma,

But you also want Figma to just become a tool like a paintbrushes to a painter, right? Where they're not thinking about the bristles so much, but instead they're thinking about what they're doing with it. So a question I have, this is kind of about domain expertise, right? There's often this conflict of like, keep it as simple as possible, but also make it complex because there's domain experts who are using this tool and who want...

the jargon. So how do you kind of negotiate that? And, you know, what do you think of as far as like, do, do you think you've seen teams like buy into the simplification or have you seen teams buy into, well, I think we need the jargon there. Or have you seen a little bit of both?

I think we're, I love this question because we are definitely positioned right now where we want to make everything as simple as possible. We have our frontline agents, our experts have

made, they've managed with the tools that they have right now and they've become really efficient. Like let's talk about efficiency and effectiveness, right? So they've become super efficient at using less than ideal tools and they've been able, they know their shortcuts, they have things bookmarked, they have their process, but is it ideal? Could it be better? A hundred percent, thousand percent. And now with generative AI, that opens up a lot of doors. So to your

initial question, which is, um, is there a buy-in to simplify and streamline processes for domain experts who already kind of have their ways and methods? I'd say that there is a lot of buy-in and we know we can shave off, um, minutes, several minutes off of a session that can allow, um, an agent to be able to, you know, have a little bit of time to think about, um,

process. They have a little bit of time to think about like going in between calls because it's call after call after call. So if they are really going super fast on one thing, it's just a lot of context switching. Yeah, I can imagine that is really difficult, right? Where it's like

It's like good luck trying to improve your process if you have calls one minute after another and you barely have time to get up and use the restroom or, you know, just as an example. I mean, of course, even if it is allowed to do that, that's still mentally something that's very taxing and very difficult. So that's great to hear that, you know, this process has ended up yielding these sorts of outcomes and that research has ultimately really been paying off in that way.

Yeah. Yeah, it is exciting. Yeah. And so as far as like how research, you know, you've talked about doing a lot of research, going to Orlando, working with these call center employees. How is that guided design choices that your team or other teams maybe have made?

So one of the commitments that our team has made is to try to minimize the amount of UI elements that we have, because what we have observed is we've got, let's imagine like a webpage and then on the other screen, there's another, a tool, a client tool, and they're looking back and forth and we've observed a lot of the things that they're ignoring. So we want to remove some of the things that,

at some point along the line had committed to being important, we're realizing that they're not important, that they're not

And how can we remove them, take away some of the cognitive load to look at all of that and simplify the UI? So that's just one kind of takeaway that's big because this is a legacy system. It's kind of baked in and we have reporting built off of this. So when I'm saying like turn off these UI elements, it's a lot of a bigger undertaking. Yeah.

Right. Yeah. It's not like, let's just delete this. Like something we might do in Figma or Sketch. It's like, I can just delete that. Right. And it's like, no, we can't because that exists and there's data associated with this. Yeah. Metrics and events are tied to these systems. So we have to figure out how to get buy-in to adjust the reporting, maybe challenge why some reporting streamlines are even set up and then go from there.

Yeah, that's certainly something that's easy in theory, but hard to do in implementation, right? Maybe takes a while. The other thing I was thinking about is with legacy systems, people kind of have learned patterns as well. It's like, I remember in the past, I did a podcast episode with Paige Lobheimer. And one of the things he said was,

it's, you know, when you, when you organize or change these legacy systems, reorganize them, that it's almost like someone goes into your house and reorganizes your kitchen cabinets. Like there might be things you've learned to avoid and like certain pockets where you pay attention to things like your mugs and your drinking glasses. And if they all of a sudden moved somewhere else, even if it was hypothetically more accessible right now, there's like this additional learning curve. And so sometimes a design change is a lot easier in

in theory, and might be harder to do in implementation. Yeah. So we have a few work streams right now. We have one that is like our North Star where they're looking at our end goal because it's such a large undertaking that they're putting that together. But we have another effort where we're working kind of in place. Like what are the things that we can start removing, toggling off or like, uh,

improving and simplifying so that we can leverage that and take those learnings and maybe utilize the same like Gen AI model and put it in our end goal. So I guess said differently is like, I hear what you're saying, taking somebody and just plopping them into a really nice

effective, like well-designed house would be great, but then you don't know where the forks are. Right. So just really basic things that you need every day. And we're being cognizant of that in the way that we're testing in the current environment, simultaneously building like this big brand new thing.

Yeah. I mean, on the one hand too, sometimes it is better to just start new so that you can just, instead of having to overcome existing habits, right. You're building new ones, but, but yeah, it's, it's a constant challenge of where you maybe do a little bit of testing to see, does it work better in this format or does it work better as like something totally new? So it sounds like you guys are working on some really exciting things and

that this this work that you've been doing with these internal experts has been really helpful not just for your company but even you know maybe more fulfilling in the sense that you're doing these really complex um redesigns and challenges yeah it's been really exciting i think we're still getting to to learn and and come um to understand how to partner in these larger teams because you know i mentioned data science being part of the product team but that like

That's not one person. It's a lot of people and a standup is almost impossible. So we're figuring it out and we're ready to take on those challenges. Awesome. Well, I think we covered a lot of really cool things and hopefully there are others listening to this episode who are

you know, about to embark on this process of doing research with internal tools, or maybe they already are doing research with internal tools. If you could offer any advice to someone who is brand new to this sort of work, or whether that's, you know, maybe they were previously doing more work within customers and now they're finding themselves in this new space. What advice could you give people to make this, you know, I guess advice you would have given yourself if you had to reflect on when you first started?

Hmm. I would just say, you know, when you are running a usability test or for the first time, let's say, always pilot, always pilot because and you can pilot with a friend, you can pilot with a coworker that you already know, as it's likely that you guys are working on different projects. So that gives you some space to reflect and make adjustments and make sure your test questions aren't biasing the answers.

that they're not, they're unbiased and that you're able to manage a dialogue because you're not always going to get direct answers to what you want and how to respond to those things. Because there might be a tendency to try to point something out and say like, but you didn't notice this one thing. And so just practice with your peers and, you know, get those reps in.

Absolutely. And I appreciate what you said, too. There's always that temptation, like, especially for me, a lot of, for example, my family, if they're struggling with using a particular app, I just tell them where to tap. I'm like, go to that button.

And, you know, and sometimes I do let them struggle a little bit, but it's really hard to let people struggle through an answer, whether that's an interview question, answer or a usability testing task. And we see that they're having a hard time.

On the one hand, there is a time and place to interject as a facilitator to maybe get to a particular part of a system that you need to test. Yeah, you have limited time. Yeah, because you're limited on time. We don't have all day. We can't just let them sit there forever. But we do still want to get some semblance of, is this something someone's ever going to find without our help?

So sometimes we need to give a little bit more time than we're comfortable with. And I know for me as a novice, that was something I really struggled with is just letting people go for a while and not feeling this sort of benevolent urge of like, I want to help because you are helping just in a more long-term sort of way. Yeah. I think one of the questions when I was running these interviews for feedback would be like,

someone would ask me, I'd ask them a question. It's like, okay, and can you describe to me what you're looking at? And they would look to me for confirmation. Like, I think that this is X, Y, Z. And, you know, you'd have to bite your tongue and be more like,

can you tell me more about what you think it might be or something like that instead of confirming that or denying that question. Yeah, instead of saying, yes, exactly, that's it, right? We can't like give them the yes, you're correct or no, that's wrong. Or even if it's wrong, we don't want to like make them feel bad either, right? We kind of want to just have that poker face, which is hard. Yeah. Oh, and I have another tip actually. I'll say that in terms of building the rapport, especially for people

testing with internal employees, find ways to connect. Like, you know, if you're, you happen to be in their physical space and I would see somebody with a drawing or a photo, I might ask them about it. And that's a really easy way to just start building a connection, making someone feel like a little bit less anxious and just really aligning to that human element.

Yes, for sure. I think that's a great technique and a good way to like, because there is a challenge at the same time where we want to build rapport. But like you were saying, we don't want to give people this sort of confirmation, like position where it's like people are looking to us for the right answer when that's not why we're here. We're here just to listen. And sometimes that might also mean we can't say much about ourselves or we don't want to cause someone to like doubt their position.

the position or the validity of whatever it is they're saying. And so sometimes we have to say less about ourselves intentionally. And like, for me, I overshare all the time. So that can often be really hard to, you know, if someone says, oh, like I'm from this place and I happen to be from the same place.

Sometimes it's a game time decision. It's like, do I really want to say I'm from there as well to make that connection? Or am I going to kind of let my enthusiasm stew in my chest, but, you know, find another way to connect? It can often be a bit tricky to find the right balance there. Yeah. Maybe like something that's a little closed, like...

One of the experts I worked with had several Rubik's cubes on his desk. And I was just like, oh, that's really cool. And we just connected on that a little bit. I'm not a big Rubik's cube person. And it wasn't like super personal that went into a lot of dialogue.

So I thought that that's just an example. Yeah, but it does give people a chance. It gives people a chance to get excited about something they are excited about. It gives the, it gives you a chance to make a connection based on something they care about, as opposed to something you care about that you're kind of pulling them into. Right. Exactly. Exactly. Well, I think that's great advice. So yeah, thank you for sharing that. And as far as like, if people want to learn more about

running studies or about you do you have any resources or social media or places you can point people to yeah well you can find me um I'm on Instagram and um I have my site angieli a-n-g-i-e-l-i.com um my Instagram is not professional but you can dm me there it's like my life all all things it's not just UX but I'm happy to answer any questions in there too

Awesome. Well, Angie, thank you so much for your time today. This has been so fun. Been a lot of fun geeking out about research, but especially just catching up with you as well. So hope you have a great rest of your day and hopefully we get a chance to talk again soon. Yeah. Thanks for having me. That was Angie Lee. If you want to learn more about her work, check out the links in the show notes.

Also, check out our website where you can find thousands of UX articles, videos, reports about UX design, research strategy, even UX careers. That website is www.nngroup.com. On that note, if you want to stay up to date on our latest research and publications, we do have a weekly email newsletter, which features our latest articles, videos, and upcoming courses as well.

And of course, if you enjoy this show in particular, please follow or subscribe on the podcast platform of your choice, maybe even YouTube. This show was hosted and produced by me, Therese Fessenden. All editing and post-production is by Jacob Swing and the Laramore Production Company. Music is by Tiny Music and Dresden the Flamingo. That's it for today's show. Until next time, remember, keep it simple.