Closing the gap between data and product development

Our product analytics team are creatures of habit: every morning we go to the same coffee shop.

The barista recently asked us to try a new coffee, but when we tried it, one of us made this face: ?

The information transmitted with that single, involuntary expression was likely worth 10x more than a verbal response.

Our goal is always to make data make sense to everyone in the company.

We often explain Intercom’s value this way: talking to customers on the internet should feel like talking to your local barista in a coffee shop. But moments in a coffee shop aren’t only about conversations. In the moment, humans observe and capture small, non-verbal interactions as they communicate.

That’s what a good analytics event does: it gives us a deeper view of an interaction. If you have a cleverly instrumented product, and data that makes sense, you can interpret why an interaction happened. Then you can decide what to do about it.

What we talk about when we talk about data

In product analytics, we collect data about how people use our product. That data can then become a living link between our users and product development.

To make sure that happens, our goal is always to make data make sense to everyone in the company. A deep understanding of the data helps product teams improve the product.

The data we collect are sent to us as events. As well as letting us send more personalised messages to our users, these events show us patterns of use — in the Intercom app and the Intercom Messenger — that help us improve how we build our product.

Until recently, we had about 350 of these analytics events set up across the Intercom app and Messenger. But with new product shipping all the time, our process for naming those events was pretty fast and loose: you needed deep context to understand them. Plus, with no underlying structure it was impossible to link events together to identify reactive and proactive behaviours.

Here are a few examples of these confusing event names:

keyboard-shortcut-switch-tab-comment
unauthenticated-warning-hovered
conversation-controls-saved-reply-inserted

Clearly, our event names failed a key principle of analytics: they made very little sense to anyone but the analytics team.

Making our data democratic

Our product is purposely built to help people understand customer behaviour, so we’d long been aware of the irony that our own data was hard to understand.

We don’t have to spend ages trying to figure the data out.

We also wanted to do something different with our data: rather than looking at every event independently, we wanted to better show the journey our users took in the product.

That’s why we started to explore a new solution: moving from a flat list of event names to a reusable and structured set of categories.

We thought this would be a technical challenge, but it was an information challenge more than anything. We needed content strategy and analytics to work together to understand these categories.

We ended up with a structure whereby each category had an: Action, Object, Place, and Owner.

• Every action had to affect something (that’s the object).
• The action had to happen somewhere (that’s the place).
• Ownership allowed us to better organize our tracking, and let each team know which events belonged to them.

Our hunch was that this structure would allow us to really see events not just as flat, disconnected moments, but as rich, 360-degree interactions.

Testing this on our new Messenger

Our Intercom Messenger was the perfect place to test these new categories. It looks at both sides of a conversation – our own customers and the customers they speak to – and our new Messenger came with a whole bunch of new features. We wanted to see how often our customers used these new features, and how their customers responded.

Traditionally, getting the full picture of how these features were used would have required about 500 new events. Remember, we had about 350 events covering our entire product up to that point. This would have nearly tripled that number. ?

With our new structured approach to events, we have 1 owner, 8 actions, 9 objects, and 2 places.

Here’s a real example:

{“action”:“received”,
“object”:“message”,
“place”:“in_app”,
“owner”:“team_messenger”
“platform”:“mobile_web”}

Now, we don’t have to anticipate every event, just all the parts and places. So without having to define every interaction, we can combine different objects and actions to get the full picture.

‘This is huge for us’

It’s now been four weeks since we launched. Here’s what we’ve heard so far:

“There is serious power in seeing the flow the user took. The naming structure makes it super easy to follow that flow quickly with a high level of context. This is huge for us: we don’t have to spend ages trying to figure that stuff out.” – Dale, Product Engineer

“Before, you could tell someone did something (e.g. typed a reply), but you had no visibility of the journey they took through the Messenger to get to that point.” – Peter, Product Engineer

We’re confident we’ve made our data democratic. The real win was seeing engineers and product managers going into the raw data and examining it themselves. We’d removed the conceptual block that stood in their way. And by focusing on the journey a user takes, rather than isolated moments, we’d exposed interesting questions.

Those questions are what matter most. We analyze our products so we can keep making them better. In the case of our Messenger, the end goal for us was better communication with the Messenger team. And that all serves a higher goal for our customers: better communication with their customers.

Editor’s note: This is the last of our posts explaining the thinking behind our new Messenger.

Part 1: Reinventing messaging all over again
Part 2: Making messaging human again
Part 3: The right kind of disruption
Part 4: Building a cross-platform product