GV's Michael Margolis on user research for startups

Following several years as a user researcher manager for communication products like Gmail, GTalk and Google Voice, Michael became the UX Research Partner at GV in 2010. With more than 300 portfolio companies and 35-40 research projects in a given year, he’s constantly parachuting into new startups, familiarizing himself with their domain, and sprinting after insights.

Michael’s also the mind behind the “start at the end” research framework, a technique that not only transformed how I approach research but also informs how we approach projects at Intercom.

To learn more about how and why to start at the end, the ways in which he quickly develops rapport with new product teams, how to conduct research faster and much more, I hosted Michael on our podcast. If you enjoy the conversation check out more episodes of our podcast. You can subscribe on iTunes or grab the RSS feed in your player of choice.

Below is a lightly edited transcript of the conversation. Short on time? Here are five quick takeaways.

1. When training others on how to do research, be as transparent as possible. For Michael this means running demos with trainees, so he can critique and iterate with them, as well as having them observe his own interviews.
2. A researcher who parachutes into a new team needs to understand not just what the team’s working on and who they’re building for, but also the worries that they have.
3. On each research project, Michael “starts at the end”. That means he first determines the end deliverable and the latest date it will be useful to those who need it, and then works backwards to understand the data he needs, the method he’ll use, and the people he’ll talk to.
4. Speed is a tool Michael uses to overcome resistance he hears to research. Two techniques for getting faster without sacrificing quality: scheduling clumped interviews and handling his own recruiting.
5. A few common UX pitfalls Michael has spotted in startups include reinventing familiar user actions, jargon-heavy copy and seeing onboarding as an afterthought.

Sian: Michael, welcome to the show. I believe that you’re the only researcher at a VC fund. How do you describe your role at GV?

Michael: GV is the venture capital arm of Alphabet, and we’ve invested in more than 300 portfolio companies. That ranges from cyber security, agriculture and life sciences to healthcare, consumer products, you name it. It’s across the board. In addition to having our investing team, we’ve built these operational teams of experts. We’re trying to become indispensable partners, so we have a design team, a talent hiring team, marketing experts, product experts, etc.

I’m part of that design team, and we’re trying to help the companies create successful products and to understand their customers, understand their competitors, and answer those questions that they have. Research ends up being a key tool for that because a startup in many ways is a learning process, and anything we can do as researchers to help accelerate that is key.

Sian: You typically work on around 35 to 40 research projects in a year, which is a huge amount. What does a typical day look like for you?

Michael: I’m doing a lot of hands-on research projects, so in a given week I’m doing one, maybe two actual studies. Often Fridays I’ll be doing interviews. So a portion of the week is then preparing for that, doing interview guides, recruiting, working with the team to understand what I need to do to conduct that and to analyze that.

I’m also doing a lot of training, teaching and coaching of research. More and more of our companies have in-house researchers, but a lot of times I’m working with designers and product managers to help them understand how to think about and incorporate research better. Often they’re talking to a lot of customers, but how do they get more out of that?

I’ll do office hours to talk to people to understand how I can help them. Basically they come to me and say, “We want to do more research,” or, “We have this very particular project and this very particular problem. What do we do? Can you help us or can you help us do it on our own?”.

If I’m being good I’m spending some time writing and developing classes for some of the other trainings and things that I’m doing. The writing is a little more difficult for me to make myself sit down and do, so that’s not something I’m doing all the time, but there’s a mix of all those activities in the course of a week.

Training startups in research

Sian: If you’re also in this role of training people how to do their own research, you must come up against the same kinds of questions time and time again. Do you have a framework that you use to train people in research techniques?

Michael: A lot of it depends on the big questions they have. For me, everything starts with questions. When I’m meeting with a company to understand what they need, often they know they should be doing some research and talking to customers or they have some very particular kind of project. In either of those arrangements, initially I’m spending a lot of time asking them questions to understand what they’ve done before, what they’re trying to accomplish, and more about their business. Based on that, I’m helping them figure out what kind of a study and research we need to do

For me, everything starts with questions.

As we’re organizing that from a training perspective, I’m trying to be very transparent about how I’m planning it, how I’m thinking about the recruiting and the kinds of questions I’m asking them about the types of participants we might include. Being very transparent about all of those parts and demonstrating it through the interview is a very important aspect of this. A lot of times people don’t have a clear picture in their head of what this really looks like when you’re doing these interviews. They talk to a lot of customers and they’re very used to pitching ideas, but when we do these kinds of interviews, it looks very different. So having them see those interviews and see how I approach it is a key piece of that training. Often I’ll do some demo interviews and have them do some interviews and critique and iterate that way.

Sian: That’s interesting. What kind of things do you often notice when you do the demo interviews? For example, do you see people asking lots of leading questions?

Michael: One thing I find myself advising them on over and over and over is building an arc. That’s investing time upfront to build some rapport. You reap the rewards of that in an interview. Don’t just jump right into it like, “Let’s look at our prototype.” That doesn’t work. Think about it like a dinner party. If somebody walks in the door of your house, you don’t just jump into that. You ease into it with some chit chat, and it’s the same in the interview. So encouraging them to take some time to ease into it.

Leading questions is a very common one. Often what I’ll advise is start a question with who, what, where, when, why, or how. It’s much harder to ask a leading question that way, as opposed to asking yes or no questions like, “Did you like it? Is it good?”

One of the easy tips I give people is to smile. Even if it’s a phone interview where I can’t see you, if you just smile and then start the interview, it just changes the way you sound and the way you think. It’s almost like getting into a character.

Parachuting into a new domain

Sian: You have to parachute into startups and very quickly get your bearings on a completely new domain, and figure out how to have some impact. What kind of framework do you use for doing that?

Michael: I’m a researcher, so all I know how to do is ask questions and listen. Explain to me what your business is. I’ll usually have some background, but I want to hear the current version of it because it’s usually changed or evolved. What are the things that you’re worrying about now? What are the things you’re working on now? What’s on your roadmap? What are some of the big decisions or arguments that are ongoing in your company? What are the key business metrics that you’re thinking about and that are really important to you to move? What are the UX metrics, like the HEART framework? Who are your customers?

I try to tease out as much detail as I can about their vision of who these people are that they think they’re building something for. All of these kinds of questions give me good context and a good way to, in the course of 30 minutes to an hour, deeply understand what they are worrying about. I’ll even ask them to imagine they had a magic wand and, I could just bring you back answers to things that would help them. What would those questions be? What’s bothering you? All of those things give me a really good sense for what they need.

What are the key business metrics that you’re thinking about?

Because I’ve been doing 35-40 studies a year since at GV, a lot of it now is pattern matching. There are certain things that come up over and over. So if I can identify the big question you’re asking, I can map that to something else I’ve done over and over and over.

I’ll give you an example from when I was working with Flatiron Health. They’re kind of like big data versus cancer. One of their big initiatives is improving the process for identifying and recruiting eligible patients for clinical trials. They know more about that than I will ever know. They have experts on staff, but the key that they’re trying to figure out is how do we improve that? How do we streamline that? It turns out to be a very difficult problem for a variety of reasons. It’s like finding a needle in a haystack, and you have to do it at just the right time before the patient is put on some other treatment.

When I work with Alex, who is the product manager there and in my view, the client, I’m focused on understanding what he needs. I’m interviewing him like a researcher to understand this domain. What they’re trying to figure out is how to streamline the process of accruing cancer patients to clinical trials. If I distill it down and I can identify that question then I know, okay, so this is a process question. So because of the pattern matching I know, okay, if I have to go figure out a process, then I know what I’m going to need to create at the end.

I have this idea of “start at the end.” I’m going to have to create some kind of a workflow. When I’m all done with this, I’m going to have to create a UX map of some sort that details who’s doing what and when and what’s hard about that through this whole flow.

At the early stages, I don’t know anything about clinical trials. Talk about parachuting into a domain. But I know that’s what I’m going to have to get to. If I can validate that with him, like, “Let’s imagine that we go do this and I come back with something that looks kind of like this and has these explanations and that this description of these friction points that we could address. Would that be a useful thing to you?” If he says yes, then I say, “Okay, now I know that’s the endpoint and I can work backwards from that.” If that’s what I’m going to need to create then what kind of data am I going to need? The kind of data I’ll need is what are the steps and who’s doing what and what are the frictions. If that’s the data, then what’s the method to get that data? For me that would be an interview, and it’s a very particular kind of interview.

This is a process interview, so I’ll develop a full interview guide, but in my head I have this sense of a north star in that interview when I’m sitting and talking to a bunch of clinical research teams. In my head I’m thinking, “What’s the goal and what’s next and what’s next and what’s next?” That’s the guiding north star. Then, who do I need to talk to? That’s going to be clinical research teams. So I’ve been able to work my way backwards to plan out what this activity is going to look like in an pretty efficient, quick way.

The “start at the end” framework

Sian: You were just alluding to “start at the end”, which is a framework you developed almost 10 years ago. I stumbled across it on the intranet at Google when we were both working there as researchers, and I can honestly say it’s changed how I approach every single research project. It’s now an approach that I teach every other researcher, and we bake the questions that you use into our research plans at Intercom. Stepping back a bit, could you explain to people who are completely unfamiliar, what “start at the end” actually is?

Michael: That’s very gratifying to know that somebody’s found it useful. I’m trying to figure out when I’m done, what do I need to create? I’m going to work backwards, so the kinds of questions that I’m asking upfront are about those needs and some of the key aspects of that are being very clear about identifying who is the decider or the decision maker? I focus on that person to understand what do they really need and what they want to get out of this. In that conversation, I’m trying to figure out what you’re going to do with this when I’m done. It’s a way to validate what I think I’m hearing and also validate whether this is actually going to be a useful thing to do, because sometimes people say, “Oh, it’s just going to be really interesting.” I have 30 companies. If it’s just interesting, I’m not going to spend my time doing that. If you’re making some very important decision or product change or something, that’s good.

When are you going to need this done is an important scheduling detail, but if you ask somebody, “When do you need it?”, you hear, “I needed it yesterday.” So often what I’ll do is ask, “What’s the latest date that this is still useful to you?” It changes the way they think about it, because then they back up and they think about their other milestones and where we’re going to fit this in.

Some of those questions help me have that initial conversation with somebody to figure out what am I going to do and what I need. Then as I described, I figure out that end deliverable and then plan backward. So if I know what that deliverable is, if it’s a process diagram or it’s a list of usability issues, then I can work backwards from understanding what data do I need, what method will I need to get that data, and what people will I need to talk to to collect that.

Sian: The reframing of that question that you used is a seemingly small difference. Instead of asking someone, “When do you want this?”, asking, “What’s the latest date that I could deliver this and it would still be useful?”. That has a massive impact on the answers you get and ultimately the impact that the project has.

Michael: It forces people to take a step back and think about their own project and their own timelines in a very different way.

Sian: That’s an amazing tool for prioritization because it helps you filter out the projects that are the nice to know, of which there are always too many to resource, and then the really important questions that need to be answered and that the business will actually make decisions on.

Michael: It also gives me an opening to negotiate what I’m actually going to do, because there are times that something could be very important and they actually need it right away. Maybe there’s a launch coming up or there’s a meeting with a partner. They might need it right away, and that’s okay. It doesn’t mean that when I ask that question that they have to give me some far off date, but then we can negotiate, “This is what I could give you. This is what I could actually get done in that time, and this is what you’d get as a result. If I had more time, I could do more.” Then we can decide whether that’s useful or not. Having that opportunity is also helpful to know what level of rigor or effort to put into something. What do they actually need?

Speed as a research tool

Sian: You bring up a great point, which is the fact people can assume research takes a long time, and that’s really not the case. But in order to have impact, you do need to know what your window of time is.

Michael: That issue of time has been a very important factor for me. Over many years I’ve learned to adapt what I’m doing for speed. I used to work at a product consulting firm where we’d do these very long, very big, very expensive projects over many weeks. Then I went to Walmart.com, where it was just me and I was working very fast, and then I got to Google and I was going a little faster. At GV working with startups, speed is really essential, and I’ve had to figure out how to know enough of which corners to cut. Not in a bad way, but to streamline. What’s the stuff that we really need and how do I adapt some of those older methods to just do it quicker and be more efficient?

At GV working with startups, speed is really essential.

Speed has been something that has allowed me to overcome certain resistance that you might hear to research. “Well, it’s going to slow us down. We’re really in a rush. We’re a startup, and we’re fast.” They might be fast but not going in the right direction, which is the risk. If I can do whatever I’m doing at very high speed, it reduces any resistance to incorporating it into what they’re doing. Let’s assume you’re doing everything right. Wouldn’t you want me to validate just to be sure? You hear, “Well if you could actually do that in a few days, that would be awesome.”

Sian: A lot of our listeners are working at startups and would love to know more about the kind of efficiencies that you’ve figured out so that you can do research efficiently and faster. What tips can you share?

Michael: One of the things that I do is schedule interviews in a way that’s clumped. If I do a study on a Friday, for example, I’ll schedule five one-hour interviews on Friday in a given day. One of the reasons I do that is it’s a lot easier to make sure the team watches because they will block out a day to do it, and I won’t do the study unless the company’s participating. I won’t go off and do it on my own because, and this is part of speeding it up, if I’m more of a tour guide than a reporter, it actually accelerates the communication and the digestion of what I’m doing into the company. They’ve seen it and experienced it together. Then I don’t have to spend as much time doing analysis and reporting and communicating to the team. It’s done at the end of the day. So that speeds it up a lot.

I primarily do my own recruiting, rather than working with a vendor, and that has sped up the turnaround time. Vendors have gotten much faster since I’ve been doing this, but there are a lot now of these services, like respondent.io, that I’ve found myself using more and more. They’ve enabled me to very quickly find the people I need and turn that around. I use a lot of Craigslist, surprisingly. The trick with Craigslist, and with any of these, is being very careful about how you write the recruiting screener to find participants. There’s some art and science to writing those well.

Sian: That sounds intriguing. What do you do in your Craigslist screeners?

Michael: Writing a screener is like any survey in that you have to write it so that nobody can tell what the right answer is. I don’t reveal what I’m looking for. Let’s say I’m working for a company that leases short-term rentals of apartments. I won’t say, “I’m looking for people who recently leased a short-term rental apartment.” I don’t want people to say, “Yes, I would love your $125. I will sign up for this.” I don’t want the incentive to affect how they’re answering my questions, because I’m trying to get the truth. So I’ll try to frame them as much as I can as openly as I can.

A sample Craigslist recruiting ad. Read more here about how Michael and his team craft screeners.

It gets back to the who, what, when, where and why questions. I will try to write them as open-ended as I can and not just say, “Did you rent an apartment in the past three months?” To me that question is the same as, “Would you like me to give you $125?” “Yes, I would.” I have to put questions in there that are intended to obscure what I’m actually looking for.

Sian: Recruitment is so, so important to getting high-quality insights – especially when you’re doing these small qualitative studies. Each person carries a lot of weight, so it’s so important to have these tricks up your sleeve to find precisely the right people and weed out the people who are inappropriate to the study.

Michael: That’s why I spend a fair amount of time with the company I’m working with to get them to detail for me who they think their customers are, which is this interesting thing because people will explain it in a fairly qualitative way. You’ll hear, “We’re looking for people who are like this,” and what I need to do is translate that into something very specific that I can measure or identify through my recruiting screener. So if they say, “We want people who are tech savvy,” I can’t ask, “Are you tech savvy?” I need to ask them, “What does that mean to you guys?” In a very specific context, that could mean they also use certain other tools or other technologies. You’re using certain apps or you’re driving a certain car. It depends on what that thing is, and so I’m looking for this context. What are the specific hints that I can then actually look for?

Universal UX lessons

Sian: You mentioned that a lot of your job at this point becomes pattern matching. After all these hundreds of studies that you’ve done at GV, have you found a way to share those insights across the whole company? Do you have some kind of internal tool that allows you to do that?

Michael: I don’t. The internal tool at this point is probably just my brain.

There are a couple parts to this. One is whatever each of these companies is doing is completely confidential to every other company.I have to be very careful about maintaining that, so I don’t do a study for one company and then share that with somebody else. It’s very specific and confidential.

Don’t reinvent the parts that are familiar and that people will use.

The other aspect is that it’s in the details for a lot of these products, and the details depend on the context very heavily. If I’m discovering something that I’m creating for oncologists when I’m then working for a company that sells tickets online through mobile app, they’re just different interactions, different conventions, different expectations for those users for those contexts. The insights that are common end up being more like best practices. Those would be things like understanding the conventions are in your domain and using those. Don’t break that. If there’s something that you need to do that’s different, do that. Reinvent the parts that matter, but don’t reinvent the parts that are familiar and that people will use. An example of this is if you’re doing some e-commerce site, people know how to use e-commerce sites. Don’t redesign where you put a buy button. Don’t do those things. Just follow the convention and it will just be simpler.

Another best practice is good onboarding, which is something people often forget. They get very focused on their product and they forget that when this first person comes in they have no idea and they have no orientation. Copy is super important too. The kind of thing I have found over and over and over is to look very carefully at your copy. One of the main reason somebody is stumbling is because you’ve written this marketing speak about what your product is or how to use it, and they don’t understand it. Be blunt and be clear and tell somebody what it is, how to use it and what to do. These are best practices that we have seen over and over and over where people could do better.

Sian: I imagine a lot of people ask you for advice about how they can start doing user research themselves. Where do you tend to point people?

Michael: Definitely go and check out all our GV.com library articles. We have a ton of stuff that we’ve written on Medium. It’s a shameless plug, but I’ve written it specifically targeted for startups and have always worried a little bit when researchers would go read it and like, “Oh, this seems maybe a little like I’m cutting too many corners.” It’s really targeted at founders and startups. Some of it is like recipes – here’s what you need to do. Here are the tips. Just follow this and it will work out fine.

There’s also a great set of articles on running research sprints. There’s a video of a class that I used to teach on quick and dirty research. It just goes through the basics, so those are good starting points.

Sian: Michael, thank you so much for talking to us today, and for inspiring my research career.

Michael: Thanks so much. It’s great to get a chance to see you.