Asana's Jackie Bavaro on developing the role of product managers

Fast forward six years to 2017, and Asana is one of the fastest growing-collaboration software tools out there, citing more than 25,000 paying customers. In the meantime, Jackie Bavaro has grown with the company and is now Asana’s Head of Product Management.

From scaling Asana to previously working as a PM at Google and Microsoft, Jackie’s seen the way product teams of all sizes operate, and I recently hosted her on our podcast to dig into her lessons learned. Our chat covers the product development process at Asana, how her team continues to scope small, the value of regular checkpoints she calls “product forums”, balancing data with product intuition, and much more.

If you enjoy the conversation, check out more episodes of our podcast. You can subscribe on iTunes or grab the RSS feed in your player of choice. Below is a lightly edited transcript of the chat.

Brian: Jackie, welcome to the show. One of the fun things about product managers is how varied the backgrounds are. Tells us a bit about your path to becoming a PM.

Jackie: I followed a fairly traditional path to becoming a PM. I double-majored in computer science and economics, then I got a PM internship while I was still in college. So, I consider myself to be one of the lucky ones. I had a friend who had done the job the previous summer, and he straight up told me that I had to apply for it. And if he hadn’t really pushed me into it, I wouldn’t have realized it was a good job for me.

Brian: And by now you’ve worked with a bunch of different PMs. Are there any backgrounds that you find are actually naturally or best suited to becoming one?

The more diversity of perspectives you can get, the better.

Jackie: To be a great PM, it’s really important that you’re a smart problem-solver and you love thinking about product and design problems. I’ve also found that people who are makers, people who have built something or they’ve started something, tend to do really well. Beyond that, the more diversity of perspectives you can get, the better. Right now we’re actually hiring for product management roles and we have no prerequisites, because we believe that having people from many different backgrounds will help us build better products.

Brian: Interesting, so you’re purely looking for aptitude. You’ve also worked in product at Microsoft and Google. Do you think each company has their own distinct slant on what a PM is or should be?

Jackie: A few years ago I actually wrote a book, Cracking the PM Interview, on applying to different companies as PM. I took my own experience and talked to other people, and companies are really varied. But, if I was going to generalize, I’d say that at Google it’s very engineering-driven, and very unstructured. As a PM, they’ll throw you into the deep end and tell you it’s your job to figure out what your team needs. Whereas at Microsoft, it’s very PM-driven and very structured. You’ll have multiple PMs on a team and be learning from other PMs, and PMs are responsible for very structured specs along with all of the product decision in their area.

At Asana we considered what we like from Google and what we like from Microsoft, and we from working with people from Facebook, and because we love clarity, we wrote it out very clearly. PMs are responsible for picking the right problems to go after, defining the goals and determining if the solution meets the goals. PMs aren’t responsible for designing the solutions or deciding how things are built. That’s designers and engineers. This makes the world at Asana a bit more high level and strategic, and that’s true even for brand new PMs.

Defining the role of PMs at Asana

Brian: How much do you want your PMs really in the details of building and delivering the product?

Jackie: A lot of our PMs will meet with their designer every single day. They’ll brainstorm and give lots and lots of feedback. So, it’s not that the PMs won’t get into the details, but there are clear lines of responsibility. The PM has to decide, what are we really going after with this? What problems do we have to solve? And they have to be able to come to an agreement with their designers and let the designers be responsible for the solution. The PM can say, “Hey, I don’t think this is meeting our goal of being easy to use,” but they wouldn’t be going in and saying, “Move this button over here.”

Brian: Is the process of zooming into the details of the solution or the bugs that are happening, and zooming out to a higher level to focus on the problem being solved, or even further with strategy, a struggle in the team’s you’ve seen? We’ve certainly felt it’s a struggle for our product managers.

Jackie: Yeah, it’s something that we try to build into our process. Lately we follow the double diamond model of product development. First you focus on and do some expansive thinking around the problem, making sure that you’re solving the right one. It doesn’t matter how well you design something, or how well you build it, if you’re solving the wrong problem. Once we’ve narrowed on the problem we want to solve, we do another round of expansive thinking about what the possible solutions can be. A lot of times you start off with one idea and you could just follow through and build that, but if you take the time to really generate more ideas, you’ll come up with even better solutions.

Jackie’s product team follows the double diamond model, which was created by Peter Merholz.

Brian: That’s something we were actually just talking about earlier today as a team – that problem definition state and whether it’s just at the start or something that you’re continually iterating on. Do you find the understanding of the problem sometimes changes quite significantly as you’re getting into a project?

Jackie: It can happen. One thing we’re not that formalized about at Asana is exactly how you have to order things, or what things you can overlap. Some teams will do lots and lots of user research upfront, and then go onto designing and building. Whereas other teams will just start with an idea, start designing it, start building it, try to get it into customers’ hands, and do most of their research later. For the teams that are doing more research later, they’ll often find new things and learn new things about the problem as they’re going along. Other teams also can always find surprises. We always try to get our new features into customers’ hands early, so either through a beta program, A/B testing, or through showing people prototypes.

We definitely can learn new things about the product later on, but when we do try to do the research up front, I’d say a lot of times we’re getting maybe 85% of it right. We really try to go with a hypothesis-driven approach. We’ll try to understand what our customers’ needs are, really focus on the parts that we’re sure we need to start off with, and then start to see what needs to be added on top of that.

The practice of scoping small

Brian: That brings us right into scoping. Every product management team in the world says scoping is critical. But I think it’s actually taken for granted. In our experience, we thought we were good at it and then last year we had found we walked backwards into big project mode. We had to be very deliberate to get into scoping small and actually writing down how we go about doing this. Has your team had anything similar where you felt like you needed to codify what it means to scope well?

Jackie: When we’re scoping, we want to scope from goals. We want to start there. When I started as a PM, I knew that scoping was really important. But what I often do is figure out what the big, beautiful, full scope release would be: prioritize all the work, and then cut off all the low priority things. When I did it that way, the low priority things ended up being a lot of polish, like an extra animation. That’s not critical for getting the value. The problem there is you end up with a product that does a lot of different things, but isn’t high quality. So we switched to scoping from the goals. Say there are five different use cases that customers have, we’ll pick one of them at first and try to do that goal really well. That means we’ll have a solution that’ll feel fully baked, then we can run through the next customer use case.

When we’re scoping, you want to scope from goals.

I mentioned the beta programs earlier, and I think that really helps with scoping, because the team, internally, will set up several milestones: The first milestone is when can our team do internal dogfooding of this feature? When is there something that we can turn on for our own small team. The second is when can we turn this on for everybody at the company to dogfood internally? Then they’ll have a milestone for when can we give this to our first friendly customers? They’ll set that date for the first friendly customers, usually like months before the actual launch date. Have that milestone in mind doesn’t answer the question of how to scope, but it makes it very, very clear that you want to scope; you’ll focus on the main things you need to learn from your customers – what does the beta need to have so you can learn that?

Brian: You mentioned with scoping that you focus on one use case and building that to a high quality, not skimping on quality to get broader coverage. How do you decide when you’re done with that problem you’re focused on? How do you decide how many use cases to cover? When have you done enough to move on to something else?

Jackie: Let’s say that for something like custom fields that we launched last year, we had several different use cases. One use case we had in mind was being able to use Asana for bug tracking. Another one was being able to use Asana for agile sprints. And another one was to use Asana for cases where you’d otherwise set up a spreadsheet with lots of columns. We prioritized those use cases in terms of what we could go after, in order, and had hypotheses about what features we would need.

Custom fields, one of Asana’s feature releases from last year

We worked really closely with the user research, marketing and sales teams to (learn) what we would need to see from customers to feel like it was ready to launch. For us, the big questions was sorts – it was going to be more expensive to sort by those custom fields than by a lot of the other things. As a PM, I like to ship early, and I really wanted to see if we should ship without them. We would work together as a cross functional team to look at the feedback we were getting in from customers, and look at the story that we could tell in the marketplace to see if there was a real, valuable use case that we could talk about that didn’t include sorting.

As we were going forward, we were finding that sorting was the number one request from customers. We weren’t finding any customers who were in the beta program saying, “I love this. It works fine without sorting.” Everyone said, “This is good, but I really need sorting for it to be useful.” So that’s how we realized that we really did need to include sorting in our MVP.

The role and value of product forums

Brian: The idea you mentioned there was the product forum, which is interesting. I’ve seen a few companies seem to be having this now, and I’d love to hear more on the practicalities of how this product forum actually works. Who’s on it? How frequently does it meet? Give us a sense of its shape.

Jackie: The people in the product forum are our head of product and co-founder, Justin Rosenstein, and then a representative from PM, user research and design. We have it every week. We have maybe two hours blocked off a week, but if it ever starts to feel rushed, we’ll extend the time. It’s one of the most important processes we have at Asana.

Brian: How would you frame the primary value of this forum? Because it could be viewed as, “Hey, this is the way we do approval for the key steps in a process,” or “This is the way we’re actually trying to get people to learn about how we think about product.” It’s also a way for leadership to stay involved in the details and what’s actually happening.

The main goal is mentorship, because we want to build high quality products in a scalable way.

Jackie: I’ve been at other companies that have a product review, and it really is just a launch gateway. As fast as possible someone comes in and tells you if you can launch or not. They’ll point at a bunch of things and say you’ll have to change all that, and then go and launch it. That’s very much what we did not want to do at Asana. For us, the main goal is mentorship, because we want to build high quality products in a scalable way. We want to make sure that what we’re getting out the door is not only high quality, but that all of the PMs and designers and engineers start to understand what our quality bar is, and why we think one solution is high quality and another one isn’t.

That’s why we put a lot of effort into making sure this doesn’t feel very intimidating. Of course it can feel that way, it’s like your launch review, but we try to make it feel a little bit more casual. We’ll focus on the positives, talk about the things that worked really well, and make it more of a discussion so that people can actually learn from each other.

Brian: Do teams actually request a review, or a slot at the forum? Or are there certain steps in the process where it’s expected?

Jackie: Both. So there’s a few checkpoints that teams should go through. If you’re doing a big launch, you would go at the very beginning when you’re setting those goals. Once you’re actually defining the problem, you would go in and talk about those goals with the team to get everybody on board. You could also go once you have designs and want design reviews or design feedback. And finally, you would go at the end as a launch review. Those are the required steps for a large launch, but there are lots of times teams come in more optionally, just because they have an idea and they want to get early feedback on it, or they want to discuss a tough trade-off they’re making.

Asana’s product development timeline.

Brian: How is it perceived by the teams?

Jackie: For the early check-ins, teams really, really love it. They’ve often come up and said it was great to get directional feedback early. When we’re giving feedback in product review, we’re always trying to do it in terms of principles and frameworks and higher level company strategy. So teams will come back and they’ll say, “I didn’t realize how this could fit into the bigger picture in that way.”

I think teams are still a little bit intimidated by launch review because they’re finally presenting all of their work. Asana has a culture of empowerment, and empowerment is really scary. One of the things that will happen is that at product forum we won’t tell people that they have to change something, or we won’t tell people how to change something. We won’t say take this button and move it over here. Instead, we might say something isn’t up to our quality bar right now. That’s launch blocking feedback you need to improve it before you can launch. That happens a little bit, and it doesn’t happen for every team.

What’s a lot more common is that we’ll have a discussion. Somebody from design will say they have one suggestion they might make, user research might have another, PM might have another and Justin might have another, and sometimes our advice even conflicts. We don’t resolve it for the PMs or the designers in that room. We give them that feedback and they have to decide what to do. That’s a lot of power. That’s a lot of empowerment. It can feel scary because the decision really is on them.

We’re always looking for ways to try to make that feel safer, to not take the optional feedback. One idea is having people go over all the feedback with a PM buddy, who can help reassure them that a lot of times the right way forward is not to do another iteration, but to say it’s important to launch fast, to say, “That was great feedback, but we’re actually going to go ahead.”

When process breaks

Brian: We talk a lot about process here, and process is such a balancing act. In theory, process is only there to make us faster and make us better, but in practice, it often fails at both. Can you share any stories of where you’ve tried to bring in process and it hasn’t worked?

Jackie: I’m one of those people that leans on the side of having too little process rather than too much. I’ve definitely been told by the team that they would rather have process.

One of the places that I’ve found where I’ve had trouble with process is when I’ve glossed over the difference between big and small projects. I wrote a post on the Asana product process, and that’s a good product process for the way we do large launches. But what I found is that on teams that were smaller, for example a team that’s doing lots of growth experiments, they weren’t going to go through all 19 steps every single time. They knew that they had to skip some of them, but because the process was designed for big projects, it wasn’t clear to them which steps they could or should skip. That was really interesting for me to learn, and it’s something that we’re now working on improving.

We’ve done a bunch of changes around that process. For example, instead of going to the product forum, which is once a week on Friday, if you have a small change you can actually just go to your manager and your manager can approve it so it can get out that day.

Working in lockstep with marketing

Brian: I want to try to move a bit out of process, which is always the temptation to talk about indefinitely. On our Inside Intercom tour I talked about how we as a product team at Intercom work with marketing. In the early days, we worked really poorly or absently with marketing, and we really tried to get a lot better at that. It’s like scoping. Everyone knows you should have the story clearly in your head about what you’re building, but in practice it’s actually very hard to do. Is this you’ve done well at Asana, or struggled with?

Jackie: It varies. Sometimes we’ve had really good successes where product and marketing have been in line the whole time. An example of this is when we did the custom fields launch. I mentioned already about how we were trying to decide if we wanted to have sorts, and working cross-functionally with marketing and user research, we realized we needed them. But another thing came up. We do these planning meetings twice a year, and before the big launch of custom fields we were talking about how we were going to market this. What do we need? We realized that the biggest problem with marketing custom fields, which can be used for anything, it that it’s hard for people to understand how they should personally be using it.

So, we did big group brainstorms with marketing, and they suggested that something really effective would be to build templates – pre-built templates that already have the custom fields in use so people could see and preview what it would be like to use custom fields. That was more of a marketing led feature. They really were the ones that convinced the rest of the team that would be great to build. We built it and we saw that, even on their own, those templates really were a big win.

Brian: How do you structure yourselves with marketing? For example, is the product marketing manager going to standups? How do you try to make that more of a genuine team?

Jackie: We don’t usually have marketing at our standups. In our product process we have a few checkpoints where people meet with marketing. At the beginning, before we’ve even defined the problem, we start to work with marketing. PM, marketing and PR will get together and talk about the story of what we’re building, and get on the same page there. What do we think will be important for the success here? What story are we trying to tell? Once we understand that story, does that change what features we need to include or how we need to develop them?

The teams are aligned on what we’re going to build all the way through.

In the ideal case, we have that meeting early, and the teams are really aligned on what we’re going to build all the way through. Then to keep marketing up-to-date we’ll often use status reports inside Asana.

On a large project you know you have to do this, but on a more medium-sized project, sometimes marketing doesn’t get looped in until later. Then, sometimes I’ll see a problem where PMs think too small about their feature. They’ll build something that’s very cool, that I know and marketing would know, could be really important for our customers. It could change the way they work. But, the PM starts to put it forward as, “We built this next thing that was a customer request, and here’s how it works.” Then when marketing comes in at the last minute, they’ll follow the PMs lead. I’ll see the blog post before it’s about to go out, and I’m like wait a second! You’re not making this sound nearly as exciting as it really is, and you’re not pointing out that there’s a deeper customer need we’re solving here.

Weighing evidence against intuition

Brian: You recently wrote a post about developing your product sense. Something we’ve found is that as a company scales there seems to be this inevitable desire for great certainty on product decisions. Analytics that are providing a clearer direction, or qualitative feedback that the whole team can get behind this very quickly. In my experience, there’s never that much certainty. Even though you have more and more information, rarely do you have this clarity that we all seek. When making decisions, how does your team balance product intuition versus actually on real evidence?

Jackie: That product intuition is critical. If you’re just trying to run a million A/B tests and then launch the ones that are win, I don’t think you’ll end up with a great product. At the same time, even the very best PMs have the wrong intuition sometimes, especially on a complex product like Asana. Because it’s such a flexible product, sometimes people are using the product in ways the PMs didn’t expect or didn’t realize. There have definitely been times when even our very best PMs will think that a change will be positive, and when we run the A/B test we see that it was negative. When we dig in to understand why, we start to realize there were side effects of the change that we hadn’t really expected.

The way that we handle this at Asana, which is sort of developing, is to distinguish between something that’s a safety net A/B test and something that’s a real experiment. For most teams at Asana, they’re building things that are based on heavy user research. So we have lots and lots of evidence from our customers,the sales teams and our customer success managers that there’s a real customer need and that we’re building something that will help them. When we’re building something like that, we still want to run an A/B test, but we’re doing it as a safety net A/B test to make sure we didn’t break anything.

Even the very best PMs have the wrong intuition sometimes.

Sometimes we want to measure the size of the win. We know this will be a win, and we just want to see how big. We’re not running the A/B Test to decide if we should launch this or not, because we’re going to make the launch decision based on that product intuition and all that customer research. But, we also have teams that are more experimentally focused. For example, if you’re changing things in the onboarding, there are really counter-intuitive effects that happen. Sometimes getting a user through onboarding as fast as possible is (best), but sometimes it’s actually better to slow them down so that they’ll be set up better for the future. When you’re making those kinds of changes, it’s really important to rely on evidence and understand what the silent majority are doing.

Keeping an ear towards our customers

Brian: Another topic you brought up in the post on how you work was the Voice of the Customer and how rigorous your team is at trying to pull this from all the different sources. I was recently chatting with a PM from Atlassian, and he was talking about a similar setup they have. But he had a big concern that resonated with me. He feared about empathy removal for PMs when you’ve got all this very impactful aggregated customer data. That the more successful the Voice of the Customer program is, ironically could in some ways distance the PM from that direct customer contact that really is where a lot of your conviction comes from. How do you advise your PMs to really stay close to their customers in a meaningful way, in the midst of that kind sea of information?

Jackie: We’re huge on anecdotes and direct customer feedback. We have automated scripts that will send customer tickets, customer PS responses and customer return survey data to the PMs every week so they’ll get to see the actual written words of our customers. Beyond that, when we have the Voice of the Customer process, we’ll generate a top ten list and we’ll form teams around that. Once the teams are formed, they’ll go into research mode and reach out to customers again. They will go sit in on user research sessions, meet directly with the customers, go on customer calls, go on sales calls, and make they’re actually talking directly to customers and getting to see the real customer words.

Brian: So you’re using the data to help prioritize, but ensuring the teams are talking directly to customers to really get the full picture of those problems.

Why don’t we wrap up with an impossible question – how do you think the PM field will evolve in the next five years?

Jackie: The fundamentals of focusing on goals and defining the right problems will be around forever. But if I had to guess, the things that might be easier in the future would be things like gathering customer feedback, usability testing, QA and having cheap ways to validate problems. For example, there’s already things like Google Surveys and UserTesting.com where you can get really quick feedback on any hypothesis that you have. I imagine in the future that will become even more popular, and PMs will be able to make even more informed, customer-focused decisions.

Brian: So the speed at which we’ll get a lot more useful information will increase.

Jackie: And the barrier to entry. There are definitely times today when a PM has an intuition and they’d love to talk to a real customer and validate it, but they had 20 different ideas that they had that day, and they had to prioritize. In the future you might be able to take all 20 ideas and really quickly get feedback. You won’t be having to make as many guesses.

Brian: So PMs in the future will have no excuses for not shipping great product. Let’s wrap up there, Jackie. Thanks a million for coming on the podcast. It was great fun.

Jackie: Thank you so much.