{"id":11536,"date":"2017-01-17T16:47:30","date_gmt":"2017-01-17T16:47:30","guid":{"rendered":"http:\/\/blog.intercom.com\/?p=11536"},"modified":"2020-07-31T23:11:03","modified_gmt":"2020-07-31T22:11:03","slug":"4-ways-to-deliver-better-product-analytics","status":"publish","type":"post","link":"https:\/\/www.intercom.com\/blog\/4-ways-to-deliver-better-product-analytics\/","title":{"rendered":"Product analytics: 4 ways to deliver better insights for higher quality decisions"},"content":{"rendered":"

Analytics only matters if it drives the right decisions. Here\u2019s how we optimized our process to deliver better product analytics for higher-quality decisions.<\/p>\n

What is product analytics?<\/h2>\n

Product analytics is the process of transforming data into insights and data-driven decisions using the collection, segmentation, and interpretation of product-level data.<\/p>\n

4 ways to deliver better product analytics<\/h2>\n

Over the past year, the product analytics team and I have been working on four key areas that have vastly improved our ability to deliver great analytics across Intercom.<\/p>\n

So whether you’re a seasoned leader hoping to improve the effectiveness of your existing analytics team or an individual just starting out on your own analytics journey, the following might also benefit you.<\/p>\n

1. Foster a culture of open feedback<\/h3>\n

For analytics to inform and drive decisions that directly affect your product, there needs to be absolute trust in the analytics and the insights derived. Great analytics teams create processes to facilitate and increase that trust. Here are two ways you can do that:<\/p>\n

Use blameless postmortems.<\/h4>\n

If one analyst is using one piece of terminology, and an engineer is using another, you\u2019ve got a problem.<\/p>\n

Make mistakes and slip ups an opportunity for learning<\/a>. Understand why and how incidents occur but more importantly, how to prevent similar mistakes from happening again.<\/p>\n

For example, when we notice that something has gone wrong \u2013 a broken dashboard, an issue with tracking or bad data \u2013 we fill in a postmortem doc outlining what happened, how the issue was resolved and what lessons were learned. We then hold a meeting to discuss the postmortem and brainstorm how to prevent something similar happening again.<\/p>\n

These postmortems have resulted in concrete learnings and a number of key action items, such as introducing a new peer review process.<\/p>\n

Create feedback loops.<\/h4>\n

Create ways to provide feedback on the queries\/code of teammates. We use a peer review process based on GitHub. Any technical effort (SQL query, python code) that\u2019s used to produce an analysis that\u2019s shared outside of the team has to be reviewed by someone else on the team.<\/p>\n

After a few months, we now have a central repo of reusable queries, whose quality and efficiency have greatly improved. We also have a weekly \u201cteam understanding\u201d meeting where people are encouraged to present something they are working on in an informal way \u2013 an analysis that\u2019s in flight, or a visualization they want input on, etc.<\/p>\n

Allowing everyone to give open, honest, thoughtful feedback will help your team to collaborate and coach one another, and make everyone\u2019s overall work better in terms of quality and impact.<\/p>\n

2. Develop close partnerships<\/h3>\n

The foundation of a successful product analytics team is close partnerships with the product teams they support. There needs to be a shared understanding and context between these two worlds.<\/p>\n

This has a lot to do with how your analytics team is organized. There are multiple ways to organize and structure an analytics team. Two popular models\u00a0are centralized or embedded.<\/p>\n

    \n
  1. Centralized:<\/strong> All analysts reside in a single group that holds all the company\u2019s analytic expertise. The analysts partner with product teams on a temporary, per project basis.<\/li>\n<\/ol>\n

    <\/p>\n

      \n
    1. Embedded:<\/strong> Individual analysts are embedded within the product teams they support.<\/li>\n<\/ol>\n


      \nFor a long time at Intercom, product analytics was completely centralized within the larger finance and analytics organisation. Analysts partnered with one or two product teams but product analytics was generally seen as more of a resource rather than a partner.<\/p>\n

      So in 2016 we moved towards a hybrid model that sits somewhere between centralized and embedded. We still have a singular product analytics team that work on foundational things together, but analysts are also embedded depending on the projects\/goals at hand.<\/p>\n

      One of the smallest changes we made in this hybrid model is that analysts now attend the standups for the product teams they support. This gives them daily context into what the product teams are working on. Most importantly, it gives each analyst a sense of belonging to the product team and the product teams feel the analyst is a part of their team.<\/p>\n

      3. Use a common language<\/h3>\n

      If one analyst is using one piece of terminology, and an engineer is using another, you\u2019ve got a problem. Great analytics requires a common language so everyone\u2019s on the same page. We did that in two ways:<\/p>\n

      Data and instrumentation<\/h4>\n

      A few months ago, one of the analysts on the team drove an initiative for a new naming convention around the data we collect (\u201cevents\u201d, as we call them). How could we design a convention that would be easily understood by non-analysts and engineers to make instrumentation straightforward, flexible and scalable?<\/p>\n

      We settled on a reusable and structured event naming model<\/a>, where each event has an action, an object, a place and an owner.<\/p>\n

      We\u2019re already seeing the benefits of this consistent language \u2013 engineers are finding the new naming structure easy to use and follow and it\u2019s helping to add consistency and familiarity in not only the data we track but also how people talk about and think about data at Intercom.<\/p>\n

      Core metrics<\/h4>\n

      There are multiple ways you can derive metrics, and many companies tend to take a purely data-driven approach. But to facilitate a shared understanding and a common language it\u2018s better to take a more collaborative approach.<\/p>\n

      For example, we\u2019ve hosted a series of workshops between analytics and the product teams to talk through the core mission of our products, what behaviors they are trying to drive, and what potential signals we can extract from the data to understand if we are driving those behaviors. Using this as input, we then turn to the data to extract and analyze what core activation and engagement metrics we care about. Again this process is leading to more consistency in how people think about and talk about core metrics in Intercom.<\/p>\n

      4. Educate your organisation<\/h3>\n

      Data is most powerful when it\u2019s accessible and understood by everyone in the company. Key to that is ensuring your analytics team educate people around your organisation. There are a few simple ways you can do that. Here are two we\u2019re trying:<\/p>\n

        \n