Let’s talk about that trickiest of subjects: Product Judgment.<\/p>\n
Also known as Product Intuition or Product Instinct or Product Taste, it is the idea that you can use your own judgment<\/i> to (1) accurately predict what your customers need, want and value, and (2) design and ship the right solution for them. Here, I will tackle some of the common questions around it:<\/p>\n Product Judgment is a tricky subject because any time someone working in a Product role has their judgment questioned, the natural reaction can often be to take it personally, to infer that the person critiquing is challenging their ability to do a good job. How can you be good at Product if you don\u2019t have strong Product Judgment?<\/i><\/p>\n I have experienced this many times on both sides, as the receiver and the critiquer. The truth is that Product Judgment is a complex topic, and in my opinion, one that is very poorly understood by many. I hope this post allows people and teams to safely talk about Product Judgment.<\/p>\n Before we dive in, we must first acknowledge that this isn\u2019t a hard science. No matter what, there is always some<\/i> subjectivity to Product Judgment. Two people with very strong Product Judgment can disagree. What I hope to achieve here is to remove as much of the subjectivity as possible, so we can all make better product decisions more often.<\/p>\n There is a lot to this, so let\u2019s start to unpack it into discrete themes.<\/p>\n No one starts out with strong Product Judgment. It is not innate. It takes years to build, and therefore ranges from very weak to very strong. It takes deliberate work and dedication to build it. Therefore some people have it, whereas others don\u2019t.<\/p>\n All great Product Managers and all great Designers have strong Product Judgment. I\u2019m nearly 20 years into my career, and I\u2019ve yet to meet a great PM or Designer without<\/i> strong Product Judgment. They all have it because they understand how to get it, and were deliberate in doing so.<\/p>\n Here are examples of Product Judgment:<\/p>\n So if it is not innate, how do you get it?<\/p>\n That is it. It is that simple. The more direct experiences you have with real customers doing real things, the faster your product judgment grows. You need to observe and talk to real customers<\/i>, who are really using your product<\/i>. You therefore control how fast you build Product Judgment by deciding how often you personally observe and talk to your customers.<\/p>\n These experiences, whether it is observing someone use your product, or interviewing them about your product, need to be wide ranging. Full Product Judgment needs to take in your customers’ perceptions and<\/i> behavior. How they use your product, what they think about your product, what they think about competing products, how they think about how your product is positioned and priced. These experiences need to cover both sides of product-market fit: the product, and the market. They also need to include all types of customers, those who are loyal, those who are new, those who have quit, those who are considering you.<\/p>\n Watching and talking to your customers must be direct, and must be wide ranging.<\/p>\n The value of using your Product Judgment is to get the team to make faster, high quality decisions. Because you understand customers so well having spent so much time with them, you can emulate them when looking at design proposals and proposed product decisions. In the same way you can predict how your friends and family will behave, because you have spent so much time with them, you can predict how your customers will behave.<\/p>\n This ability to predict is about shooting for repeatable product success. But time does not stand still. To have repeatable success you must continually evolve your understanding. Because your product changes, your industry evolves, technology evolves, and your competitors get better, you must continue to have direct experiences with your customers. You must keep talking to them about your product, about what you have shipped, about real usage, and how it too is changing.<\/p>\n You cannot obtain Product Judgment from reading summarized lists of feature requests. Nor from reading survey data. Nor from reading research reports from your research team.<\/p>\n Here is a common shortcut for busy people: your research team does the direct watching and talking to customers, and they play video clips of those customers using your product. This is not enough. Observing a recording is certainly better than reading reports or lists, and so it helps a little bit, but that is a very slow path to growing Product Judgment.<\/p>\n Another shortcut is to talk to the people in your company who do most of the talking to customers, notably the Sales and Support team. These people are<\/i> directly talking to customers, and that\u2019s good because they have built some judgment doing that, but you won\u2019t build much personal judgment this way.<\/p>\n The indirect sources of customer insight above are all really important because they can help round out your Product Judgment, but they can\u2019t create it<\/i>.<\/p>\n Product Judgment is attained from direct experiences with customers, but there is more to it than that. It is domain specific. This is good when you can transfer that judgment across domains, but it is often the case that you cannot, and you need to get in front of those customers.<\/p>\n To properly understand a customer when they talk about your product, you must first deeply understand it yourself. You must know how it works, and how it is supposed to work. Without this deep understanding you will get lost when your customer is getting into the weeds, or you will struggle to read between the lines and ask perceptive questions. The ability to ask perceptive questions is critical to running high signal interviews from which you can learn the most. You must know your product deeply to fully understand and interpret what customers say and do when using it.<\/p>\n It may sound obvious that you should use your own product to understand it, but many product designers don\u2019t actually use the product themselves as they are not the target customer. They might play around a little in it, and think<\/i> they are using it, but I am talking about real usage. Really using it for the real thing it was intended to be used for. Really understanding one\u2019s own product only comes from this real usage.<\/p>\n There are ways to do this in any company. For example, at Intercom we build customer communication software. Part of Intercom is a product for Support teams. But our PMs, Designers, Engineers don\u2019t work in Support. So we have a process internally where they periodically work with our Support team, for a day or half day at a time<\/a>. They then experience real usage, which helps them become better at interviewing customers, and also rounds out their own Product Judgment.<\/p>\n Because Product Judgment is based on direct experience, you can\u2019t have strong judgment in areas you haven\u2019t observed directly \u2013 no one has strong Product Judgment for all products. I\u2019ve learned this the hard way. Early on in my Intercom career, I made some bad product and design decisions.<\/p>\n Before Intercom, which is a customer communications product for businesses (in other words it’s a B2B product), I spent most of my career working on consumer products like Gmail and YouTube. Having never worked on B2B products, I had zero product judgment in this domain. But as someone who had designed lots of communication software, and as someone who hadn\u2019t yet figured out this Product Judgment thing, I simply assumed I had the judgment to make the right decisions.<\/p>\n I incorrectly assumed that if I had designed products for people communicating with their family and friends, I could design products for people communicating with businesses. I was wrong, and I made mistakes. I had to go and learn Product Judgment for B2B communication products by observing and talking to Intercom customers. There was no other way. There is no other way<\/i>.<\/p>\n In my early days at Intercom I was missing the B2B piece, and learned the hard way, but I wasn\u2019t starting from scratch. Having worked on social software for much of my career, I had strong Product Judgment in that domain, and could apply it to Intercom. Things like the psychology behind human communication, people\u2019s behavior patterns with different communication channels, and what easy-to-use communication workflows and UI look like.<\/p>\n Knowing the limits of the Product Judgment you have built in the past is critical for knowing how to fast track the new judgment you need to build. This is really important when you change teams to a new domain or change companies. Unless moving to a direct competitor (and honestly even then), you need to get talking to customers asap.<\/p>\n It is obvious that different industries are different domains, for example the difference between a communication product for family and friends to use, and one for businesses and customers to use. Different parts of building software are also different domains.<\/p>\n You can talk to customers about how they perceive value in your product<\/i>, how they think about your competitors<\/i>, and that conversation will improve your Product Judgment on making roadmap decisions. It won\u2019t help you at all in designing workflows and UI.<\/p>\n You can observe and talk to customers as they engage in using your product<\/i> and its workflows, deeply interrogating UI decisions you\u2019ve made, and that will help you build Product Judgment in knowing how to design an easy-to-use product. It won\u2019t help you make roadmap calls.<\/p>\n Building a great product crosses many domains. From what problems the product solves, to how it solves them, to how it is priced and positioned in the market.<\/p>\n To build fully rounded Product Judgment, you need to cover the wide range. This is why when I run customer interviews, I always cover a lot of ground. When working at Google, I ran many hundreds of tactical research interviews on the usability of proposed new social products. But I always covered much more ground than that. I always covered desirability and perceptions, and always spent time understanding what other social products they used. I added time to interviews to ensure we were getting the full picture, to ensure we were maximizing the growth of our Product Judgment.<\/p>\n So there are two critical skills here:<\/p>\n A final thing to understand about Product Judgment and domain is that it is also a moving target.<\/p>\n With experienced leaders and practitioners, there is a danger of giving yesterday’s advice today. To avoid dogma, you must continually ask: what has changed? Things fall into three different buckets.<\/p>\n First, there are many things that never change, and they are rooted in human psychology. Things like our cognitive biases, how we interpret different shapes and forms. You can learn these in many good books (and you should), but to truly internalize them you need to see them in action, you need to see people applying them unconsciously when interacting with your product. But when you understand them, you can easily spot them in any product.<\/p>\n Second, there are things that change slowly. Internet software design patterns is one such area. Like all new technologies, where the early years see wild divergence in design patterns and then extreme convergence, the fundamentals of how internet software is designed hasn\u2019t changed much since that convergence happened about 10 years ago. So this domain knowledge is very transferable. If you have observed hundreds and hundreds of people using internet software, you see the same design patterns work and fail time and again, and so can predict with great accuracy the fundamentals of where a new proposed design will be confusing to use or hard to understand. You can use this judgment to easily shortcut the design process. This is an important skill, and one that all the great design leaders have deliberately acquired. Assuming they are close to the target customer, they can look at a product and dissect what makes it work or not work.<\/p>\n Yet whilst the fundamentals of internet software design remain the same, much is still evolving, albeit slowly. We have new versions of operating systems every year, with new types of devices changing how people use technology. It is critical to avoid dogma.<\/p>\n Finally there are things that change fast. Things that quickly and fundamentally change how people think. Brand new types of products, new innovation from competitors, macro socio-economic changes that alter our perceived reality in the blink of an eye.<\/p>\n Some things don\u2019t need judgment. Judgment comes from experience but for many things there is a right and wrong way to do it. Some of these things are in the first category above – things that don\u2019t change, but there are also things that evolve and what is right and wrong evolves also. Often these evolve from human behavior patterns that set over time, and the consequent solutions people expect. For example, you steer a car with a big giant wheel. The accelerator is to the right hand side of the brake. Same in software.<\/p>\n There is a right and wrong time to use checkboxes and radio buttons. There is a right and wrong way to design buttons and links. There are right and wrong ways to use variables in font like size, colour, contrast, etc. This idea is one of the benefits of a formal education in designing and building software. From Design to Engineering, there are common best practices that don\u2019t require judgment \u2013 they are objectively better or worse \u2013 and education teaches them to you. Of course you can also teach yourself, by reading and practicing, and many great PMs, Designers and Engineers are self-taught.<\/p>\n One could conclude from what they have read here thus far that Researchers, as the people talking directly to customers most often, must have the strongest Product Judgment. This can be true, and I\u2019ve seen it firsthand. When it is true it is an incredible<\/i> asset for a team. As an ex Researcher, I always encourage Researchers to understand their personal domain limitations, and broaden their thinking to become the best version of themselves.<\/p>\n Unfortunately, this breadth of domain knowledge in Researchers is often not the case. Despite being the people talking to customers most often, many Researchers don\u2019t have strong Product Judgment in other domains, and so find it hard to translate direct customer observations into product decisions. It is one thing to talk to customers and build judgment, it is another to apply it<\/i>.<\/p>\n Researchers often stop short of applying it. They hand over the insight, they don\u2019t own the impact of the insight. Perhaps this is for good reason \u2013 maybe nobody asked them to help design<\/i> the product, or invited them to actively participate<\/i> in a scoping workshop. But this is wrong in my opinion \u2013 they should be invited, and if not they should invite themselves and prove their value.<\/p>\n This is why I encourage people and teams to blur their roles and responsibilities, so they can learn how to apply<\/i> what they are learning. For the same reasons I want PMs and Designers talking directly to customers (and not outsourcing it to Researchers), I want Researchers sketching solutions, talking to Engineers about how to build them, and building fully rounded Product Judgment by applying what they have observed.<\/p>\n They can look at a proposal and say \u201cThis is not something I have built judgment for because it is not something I have directly talked to customers about\u201d. They know their own limits. They typically defer to the people in the room who have built that judgment, or<\/i> they recommend that we get talking to customers asap to start to build that judgment.<\/p>\n As I have grown in my career, from working as a Designer, Researcher, and PM to running a full Product org, I find myself in this position more and more. As time goes on, I move into higher levels of strategy, and like almost all other leaders and execs, I talk to customers less and less. This is not good, and something I work on improving periodically. But leaders and execs are desperately time poor, and so while one should never get too far from your customers, it is important to defer to those who do have strong judgment in a domain because \u2013 and only because \u2013 they have<\/i> talked to customers.<\/p>\n They just know<\/i> this is the right option, and they are often correct. This is because their brain is using lessons learned from years of direct customer interaction, synthesizing it all subconsciously, and shortcutting to the recommendation. As has been popularized by Thinking, Fast and Slow<\/a><\/em>, this is how our brain works.<\/p>\n This is the most challenging of situations, as less experienced staff may not understand where this judgment is coming from, and simply see it as uninformed dogma. It is also unfortunately true that leaders with poor Product Judgment exhibit this exact same behavior, but from a place of false confidence and ego, rather than lessons learned from customer interaction.<\/p>\n This is tricky to disambiguate, but you can usually tell the difference by the leaders’ interest in talking to customers, past experience with talking to customers, and insistence that everyone talks to customers. You can often identify strong\/weak Product Judgment by using interest in talking to customers as a proxy.<\/p>\n\n
\n
How to obtain product judgment<\/h2>\n
Product Judgment does exist, and it is learned<\/b><\/h3>\n
\n
Product Judgment is obtained through direct experiences with customers<\/b><\/h3>\n
Growing your Product Judgment is a never-ending journey<\/b><\/h3>\n
You can\u2019t obtain Product Judgment through indirect means<\/b><\/h3>\n
To understand how your customers talk about your product, you must use it yourself<\/b><\/h3>\n
Product Judgment and domain knowledge<\/h2>\n
Product Judgment is domain specific, it is not universal<\/b><\/h3>\n
Some domain knowledge is transferable, but you must know your limits<\/b><\/h3>\n
Different parts of building a product are also different domains, you must go wide<\/b><\/h3>\n
\n
Understanding where and when Product Judgment needs to evolve, versus simply be transferred, is very important<\/b><\/h3>\n
Sometimes it is easy to conflate best practices with judgment<\/b><\/h3>\n
Domain specificity is the reason Researchers don\u2019t always automatically have the strongest Product Judgment<\/b><\/h3>\n
The roles people play<\/h2>\n
People with strong Product Judgment are very good at saying when they don\u2019t have that judgment, because they deeply understand how it is accumulated<\/h3>\n
An experienced leader with deep Product Judgment can often look at something and know what to do, but find it hard to explain<\/b><\/h3>\n
Many people think they have strong Product Judgment but don\u2019t<\/b><\/h3>\n