Lyft's Monal Chokshi on lightweight user research

https://www.intercom.com/blog/when-personas-fail-you/

Monal Chokshi, Head of UX Research at Lyft, has spent her career helping tech companies establish a user research culture and get closer to the people hiring their product. She joined Lyft in 2015 as its very first researcher and was tasked with building and leading their research department – something she’d previously done successfully at SoundCloud and Intuit.

Monal joined me on our podcast to share how researchers can prove their value, the way she’s structured her own research teams and why lightweight, guerrilla research – an exercise she likens to dating – is the user research gateway drug for startups.

If you like what you hear check out more episodes of our podcast. You can subscribe on iTunes or grab the RSS feed. What follows is a lightly edited transcript of the interview, but if you’re short on time, here are five key takeaways:

1. A startup’s first researcher hire may need to focus on tactical research in the early days. Use it to make a name for research and show your value, which opens the doors for more strategic initiatives.
2. Guerrilla research is a great tool for quick, lightweight user feedback, but isn’t a substitute for the larger user studies you’ll need to make reliable business or product decisions.
3. There are five keys to gathering lightweight feedback: choose the right audience, let them do the talking, avoid leading questions, pay attention to the non-verbal and show appreciation.
4. Embedding researchers in product teams allows them to become experts on a particular set of user needs, product opportunities and the design required to enable those users to be happy and enjoy the experience.
5. The role of a user researcher is to tell the story of users and help grow empathy across the company. That means finding creative and frequent ways to share their insights with all departments.

Jillian Wells: Monal, welcome to the show. Could you tell us a little bit about your career in research and what you’re doing these days at Lyft?

Monal Chokshi: Lyft is the fastest growing on-demand transportation service in the US, and I feel pretty lucky to have had the career path that I’ve had leading me here. I found my passion for UX research pretty early, when I was at Stanford in the mid-90s. I did my undergraduate degree in symbolic systems, which is an interdisciplinary major consisting of classes from computer science, philosophy, psychology and linguistics. My concentration was in human-computer interaction.

When I started working in the late 90s, it was right in the middle of the Dot-com era. I was what today you might call a UX generalist, so I was doing everything from user research to product design and even some front-end coding. At that time, there was no term like UX or user experience. It’s very early and folks wore a lot of hats, especially at the startups I was working at. After working at a few startups I realized that my true passion was understanding more about the users and their needs in order to inform really good design; so I decided to go back to graduate school at UC San Diego in order to focus solely on moving forward with UX research.

Jillian: Do you feel like your undergrad career also prepared you for research or did you get more into you once you were in the field?

Monal: It definitely prepared me for that. In fact, I did an internship while I was in undergraduate at Boeing and I was working at usability labs. That was my first experience getting to do some user testing and research. I entered the workforce full-time with that experience and those thoughts in mind.

Jillian: How was the transition after earning your graduate degree?

Monal: My graduate degree was in Cognitive Science with a concentration in Human-Computer Interaction. I went to UCSD because there were some research labs focused on using ethnographic research methodology to understand users in real world situations and how their social and cultural environment shape their behaviors. I’d had a little bit of experience doing field research, was really interested in it and thought it was a very impactful way to learn about users and then apply that to design. In fact, my master’s thesis was funded by Nissan and I was doing ethnography of drivers and cars, so now I’ve come full circle.

Showcasing the value of research early

Jillian: What was the state of research when you arrived at Lyft?

Monal: I was hired as the first official researcher at Lyft about two years ago. The company, by then, was very very hungry for insights. Lyft didn’t have researchers on board from the start, but they’ve always invested in some kind of formal studies and lightweight research along the way.

I love telling this story because it’s really indicative of like the culture at Lyft and what our founders care about: In the early days, the entire Lyft team including our founders, Logan and John, would respond to customer support tickets aiming to better understand and address our users’ needs and pain points. They also did, and continue to do, guerrilla research like observing and chatting with Lyft drivers in their cars to better understand how Lyft is doing and how well the product is working for them as they drive. In fact, they’re both Lyft drivers themselves to stay in touch with the driver’s perspective.

Jillian: How has research at Lyft changed since you’ve arrived?

Monal: Since I was the first researcher there, I was building things from scratch, similar to when I was at SoundCloud. There was definitely a need and a desire from the get-go to do user testing. Lyft was a few hundred people at the time, and I worked as part of the design team. My first goal was to setup an ongoing user testing program, and I was lucky enough to have another team member join. We worked together to get that going but we ended up doing a lot of tactical research in the very beginning. It was a lot of, “Hey, let’s prove what we’ve got going on. Let’s make a name for research, show our value and work closely with the design team.”

Once we proved our value we were able to hire more folks and start bringing in more strategic initiatives, answering bigger, more challenging questions. We were really trying to fill some of the foundational gaps we had early on, but it’s a balance of what you use your time for and building the value that the company needs immediately, then coming back and showing them more of what you can do and the value of that as well. The team continues to grow when people see what you can do.

Embracing guerrilla research

Jillian: You mentioned guerrilla research, which is something you gave a recent talk about at Mind the Product’s Product Tank. What’s your definition of guerrilla research?

Monal: Guerrilla research is a way to get impromptu and informal feedback from users. Most often, this entails quick and dirty user testing of your product or a prototype with easily accessible folks. This is more than just showing them some screens and asking if they like it. It’s about observing their use of it and learning where they struggle.

Jillian: When is it most useful?

Monal: Most researchers would be remiss to call guerrilla a real form of research. It has a spur of the moment nature and doesn’t really allow for us to control for the factors that are necessary when conducting real research. Relying solely in guerrilla research is risky because we can’t really use it to make reliable business or product decisions. That all said, guerrilla research is really just a lightweight way to get feedback, and it’s best for teams who are not doing any research at all or who are not talking to their users.

I recall a meeting with the PM and a designer at SoundCloud who’d been debating the design of a new feature. When I found out that their debate had been going on for days, I said, “Let’s just go out there and see how people are reacting to it,” so we went out to a local coffee shop to do some quick and dirty guerrilla research.

I see guerrilla research as being a gateway drug to real user research.

We were able to quickly test a prototype of a feature with a handful of users and it was really eye-opening experience for both the designer and the product manager who had really not done any kind of research before. Much to their surprise, that issue they were debating was not even on the table for these users. A quick round of testing revealed an entirely different issue to consider, and they immediately started taking this new perspective into account. As a result, they saw the value of getting feedback from users, became user research converts and kept asking me to lead more research.

I see guerrilla research as being a gateway drug to real user research. I’ve seen many folks start to understand the value of interacting with users in this way and then want to invest in having user research on board to do real research.

Jillian: You have a really nice metaphor for guerrilla research and the fact that many of us have already used the principles behind it – our dating lives. Walk me through this.

Monal: I have an outgoing personality, but it’s always a bit nerve-racking when you randomly approach someone you don’t know to participate in research. It’s that same kind of feeling when you go out to talk to a stranger in a bar or for dating purposes.

There are five key principles that can be applied to guerrilla research as well as your dating life.

1. Choose your target audience. We do this in our dating lives already. We may look to meet folks who share similar interests or hobbies. Let’s say I’m a rock climber, I might meet someone who I’m really excited to talk to and possibly date at the climbing gym. We know who we want to date, potentially, and we know where to find them. Similar with guerrilla research – you need to know who your product’s target audience is and figure out where to easily access them. You cannot get worthwhile feedback unless you’re talking to the right audience.

2. Let them do the talking. Going back to guerrilla research, you’ve met the person that you want to target and they’ve agreed to participate in a testing session. When you’re getting feedback from your product, your main goal is to listen and to encourage your users to keep talking. You want to ask open-ended questions such as, “Why?” or “Tell me more about that.” You want to encourage them to keep talking so that you can learn what they are thinking while they’re doing the task you’ve given them with your product. Similarly with dating, it’s always a good tactic to show genuine interest in someone by asking questions and actively listening. You want to let them do the talking.

3. Don’t ask leading questions. These could bias a user into answering a question in a certain way, which you don’t want for research. For example, asking, “Isn’t this feature useful?” may lead users to agree with you rather than come up with their own impression. The better question in that case would be, “What do you think of this feature?” With dating let’s say you meet someone. You might not want to assume anything about them or what they’re interested in. Instead of asking leading question about what their interests are, it might be best to leave it open-ended and ask, “What kinds of things do you like to do?”

4. Pay close attention to what they do. People often say one thing but do another; so even if users say, “I think this product is great. It’s so easy to use,” you may have observed some pitfalls they encountered or you might have noticed some confused facial expressions they had as they were using the product. You’re still getting to see where they had issues and trouble. Now with dating, your body language can play a huge part in understanding the difference between what people are saying and what they’re actually doing. They might intend a cold shoulder, but they don’t want to be mean so they might say something nice. Reading body language is a very important.

5. Show appreciation. Buying someone a coffee cannot only entice them to participate in your guerrilla session, but it’s also a great way to show that you valued their time and feedback. When dating, of course, it’s always respectable to thank someone for a great night out.

Building a UX research team

Jillian: You’ve built out the research teams at Lyft and previously at SoundCloud. There are a lot of different paths into this field. What do you look for in a researcher?

Monal: There are definitely many paths, and it’s one of the things I love about our field. We have people entering from so many different perspectives and it creates a really nice diversity.

There’s lots of traits that are important to me when considering who to hire. Obviously, it’s great to have people who have experience with UX research already, but we always look to bring on folks who are more junior and to help them grow as well. I look for three key qualities when considering a candidate.

The first is their ability to empathize with others. As a user researcher, being empathetic is at the core of what we do. We need to be good, active listeners and feel comfortable when trying to truly understand other’s perspectives. Without being able to do that, it’s really hard to collect good data.

Being empathetic
is at the core of what we do.

Then the second quality is analytical thinking. It’s a key component of being able to turn the data that you collect into insights and then recommendations, so having a nice way to point the data that got you to get to the recommendation is really key for teams to buy in to those insights.

Finally, and this might be overlooked, but project management skills are key to the success of a researcher. We’re often managing several things at once, sometimes for several studies. But for every study, we need to manage stakeholders, participants and facilities, and of course all the data needs to be super organized in order for you to be able to find your way through the masses. It’s important that researchers are able to manage their time well, stay organized and really have an eye for detail.

Jillian: How have you gone about structuring your team?

Monal: We started out with more of a service model because we were so small. Now, I’m moving towards a model where there are dedicated researchers for each of what we call our pods. We have passenger-facing products, driver-facing products, enterprise products and products for autonomous vehicles. We have dedicated researchers now for each one of these four pods. My goal is to empower each of the researchers who are working in these pods to be able to become the expert for that product area. They’ll know everything they need to about that user. They’re the expert on the set of users who are using those products, and they can really speak to the user needs, the product opportunities and the design that is required to enable those users to be happy and enjoy the experience.

Jillian: Do you all come together to share that knowledge, so all the researchers are exposed to the same level of insights?

Monal: Definitely. We also have a couple of other research teams beyond UX research. We have a brand and consumer insights team – they do a lot of work with our marketing organization. Then we have policy researchers who work in our government relations team. We have a few different pods of research across the company. I hold a monthly meeting for all of the researchers so we’re able to collaborate, exchange ideas and ensure that we’re not duplicating efforts. At past companies I’ve seen that happen, where you have a market researcher and a UX researcher who are trying to tackle very similar questions and almost doing the same study. As the first researcher, I was in this luxurious position to make sure that doesn’t happen.

It’s been really nice to have this community, and you get to hear what’s going on all across the company as well. We’re there for each other and there to collaborate as well. Just the UX research team, we meet weekly. We all sit together so it’s very easy for us to collaborate.

Jillian: As the head of research, are you still regularly interfacing with customers?

Monal: In the beginning I was in the trenches. I was doing it all. As the team has grown, I’ve moved further and further away from being able to do the hands-on research. It’s very empowering to be able to work towards having research influence the organization, but I do miss the hands-on research and the ability to be out in the field and run sessions. However, I do get some exposure to that through my team, so I will mentor folks and sometimes run a study together with someone. I’m sure as we continue to grow those opportunities won’t be as often, but I do really like to get in there whenever I can.

Research in a dual-sided market

Jillian: Lyft is a dual-sided marketplace – your users are both drivers and riders. What challenges does that present to your team?

Monal: The drivers and passengers are very different users with very different jobs to do, so we look at each of them very differently and consider them separately when mapping their user journeys. What’s really interesting to me is that this is a dual-sided marketplace, but it’s not just digital; the two sets of users actually intersect in the real world at some point along the journey. The user experience converges inside of a contained vehicle, which is moving from one physical space to another. It’s really like interesting.

We recently did a study with our new amp. For those who know Lyft, it’s the new glowstache – a lighted device on the driver’s dashboard. The color matches the color on the passenger’s app so that drivers and passengers can more easily find each other. There’s a whole setup process for drivers, and we wanted to really understand how well it was working for passengers, so we decided to do a user testing study with both drivers and passengers simultaneously.

Myself and Nathan, who is a researcher on my team, ran the study simultaneously, and he had the driver’s start over at our driver hub, setup the amp and then make their way driving towards our Lyft office in SOMA. Meanwhile, I was running a study with our passenger user and hearing all about how they request rides and we had it setup so that the passenger can request that car. We were running the study simultaneously. Nathan, with the driver, would drive up, text me and tell me he was on his way so I knew when to ask the passenger to request a ride. It all worked very seamlessly. They would pull up in their car and then we would get in. We’d have the two researchers in the back and the driver and the passenger together so it all came together in this car.

Then we’d get dropped off and I got to do debrief with the passenger. Meanwhile he drove back to the driver hub and was able to do debrief with the driver. What makes it interesting is coming together in a car. It’s where you see the dual-sided marketplace converge.

Jillian: It’s exciting to think about your digital product like out there in the real world. Lyft just announced 54 new cities in late February. What’s the biggest challenge when it comes to scaling your team’s work along side this?

Lyft recently announced its biggest expansion yet, which will see the service rolled out to 50+ new cities in the US.

Monal: There’s always more work that could be done. That said, I really look to ensure that my team is taking on the most impactful work that we can be doing. Alongside of taking into account our company goals, I work closely with our product and design partners to ensure that we’re tackling the highest priority projects for the company. Each quarter we also aim to get out to various market to conduct research. We’re growing now. We operate in over 300 cities, but we’ll conduct various types of research such as surveys and diary studies to ensure that we’re getting input about the user experience across the various markets.

Jillian: Lyft, as a whole must be hiring like crazy. What has your team done to help democratize access to research across the company amid such a high growth?

Monal: I see a big part of our role as user researchers to not just do the research but to also tell the story of our users to help grow empathy for them across the company. My goal is for us to influence our teams’ ways of thinking so that we’re all focused on creating user-centered products with the right insights in mind.

Every researcher on my team has a goal of presenting their work at a company or a large team meeting each quarter. We work closely with our project teams and direct stakeholders for the research, but then we’re looking to share as much as possible on a widespread level.

Jillian: Do you guys have any other tactics for sharing research insights like creating posters or like pamphlets?

Monal: I have a goal for my team this year to really focus on delivering insights that go beyond the screen and decks. We’re engaging more with doing workshops and finding creative ways to deliver insights that are much more memorable and that will really prompt action on the part of the folks who are in the audience or involved and participating in the workshops. That’s often project managers, engineers and of course our designers.

Jillian: I think we’ll leave it there. Monal thanks so much for coming in.

Monal: Thanks so much for having me, Jillian.