Maximizing user research across a product's lifecycle

At many companies, researchers are like paratroopers. We parachute in for special projects, with the mission of gaining the required insights, only to be reallocated to another team once the mission has been completed.

It’s a far from ideal approach that limits the impact a researcher can have on a product, because it’s harder to influence a team when you’re not a part of it. It also forces research activities into a “step” in the process, or a box-ticking activity for the start and finish of a project. At Intercom, we’ve opted to do research differently – embedding researchers in the product teams that need them most.

If you’re thinking about hiring your first researcher, or are currently a research paratrooper who wants to be embedded on a product team, you’re probably asking – how do you know which team needs a researcher the most? How do you continue to support other teams that need research done?

At Intercom, we allow product maturity to guide how we work. This enables us to choose the most appropriate way of working – a lean research approach where we are embedded with the team, or an iterative approach where we are connected to the team.

Research is a continuous process

When we’re building a product at Intercom, there are no distinct research “steps”. This is because research takes place across the entire lifecycle of a product.

However, whether you’re designing and building a new product or you’re making improvements to an established one, it’s essential to choose a research approach that complements the maturity of your product.

For a brand new product, benefits and features evolve over time, and prior to launch there is a long feature maturity phase. This is when decisions are made about which features should be built, and how those features will work. For example, at Intercom our designers will “think big” at the start, designing the ideal version of the product, outlining in detail how it could work and which features we could build. Following this, customer insights help the designer iterate on these plans. Next, a product manager will scope the project, and ultimately guide the engineering team on the order in which the features should be built and released.

For a new product, the benefits and features evolve, until the product fully solves the problem.

Contrast this to an established product. The design evolves, but the benefits and features remain the same (when you have a loyal paying customer base, changing features and benefits mid-flight can be risky). Whilst an established product will have small bursts of feature maturity (or small feature additions) throughout its lifetime, when the time comes to redesign these products, visual changes usually take precedence.

For an established product, the design evolves but the benefits and features remain intact.

As a researcher, when you’re starting work with a new team, it’s essential to select a way of working that fits what that team needs. At Intercom we choose the most appropriate working approach, based on how mature the product is. The two approaches are:

1. A lean research approach: The testing of hypotheses in quick, methodical ways. This enables the researcher to shape how a brand new product will work, and requires the researcher to be embedded with the team.
2. An iterative research approach: Repeatedly prototyping and testing with increasingly realized versions of a product. This enables the researcher to shape how an established product looks and feels, without needing to be fully embedded with the team.

A lean research approach

There is a misconception that lean research is primarily about testing minimum viable products, but it’s actually more similar to agile development. The focus is on creating value with limited resources. The limited resource for researchers is time. (Though lack of time shouldn’t stop you gathering reliable insights.)

Lean research is about evaluating ideas in a quick, methodical way and has two main goals:

1. Reduce the risk of building the wrong thing, by ensuring the product being designed will have value and utility for its prospective customers.
2. Evaluate and evolve a product’s features to ensure it’s solving the problem identified.

A researcher can do this by conducting regular, small research projects and collecting continual customer feedback. The methods (e.g interviews, concept testing) will constantly change, because they’ll need to choose the research method most appropriate for the problem being addressed. Often this means conducting smart, scrappy research, such as concept testing with static screens, or conducting short remote interviews online.

Like many Intercom products and features, the inspiration for Smart Campaigns came from feature requests. We knew that customers wanted a way to target and send messages based on how people had interacted (opened, clicked, replied) with a previous message. However, this was a big engineering challenge, and there were multiple possible solutions for how this product might work. As an embedded researcher on the team, I was tasked with testing these solutions, and providing answers for any of the questions that bubbled up during solution design.

Sitting with the team, and attending all of their rituals (e.g stand up and retrospectives) from day one allowed me to keep a pulse check on their confidence levels and progress. Being available to collaborate with the product manager and designer (e.g design reviews) kept me in earshot of their concerns, which helped me identify key moments where new customer insights were required.

We were in a constant state of hypothesis creation and testing, which meant lots of small research projects.

We were in a constant state of hypothesis creation and testing, which meant lots of small research projects were being conducted along the way. Though small, they provided opportunities for research to have a voice during product discussions, which ensured customer insights were helping to drive the decision making process.

Before launch, we had conducted multiple rounds of customer interviews and concept tests, sent an endless stream of research messages, and had three different betas. In total we put six variations of the product in front of customers.

All of this contributed to building a rich understanding of the customer. It enabled us to go deep into whether the product we were building was solving their problems, and, what we had to build next to make it really useful for them.

The lean research approach was most valuable for Smart Campaigns, because it was a new product that had not yet been fully defined, and there was flexibility as to what features would be built. This approach only succeeded because a researcher was embedded with the team, as otherwise it would have been impossible to keep up with the pace of change, which would have meant many missed opportunities to gather insights when they were needed the most.

An iterative research approach

You’re probably familiar with iterative design – developing and refining a product’s design based on feedback and evaluation. I would argue that iterative research is the driving force behind iterative design. Whilst internal feedback and design reviews can help to some degree, putting designs into your customer’s hands can reveal problems even the most seasoned designer or product manager couldn’t anticipate.

Iterative research has two main goals:

1. Evaluate and refine the usability of a product, thus reducing friction, so customers can intuitively and painlessly achieve their goals.
2. Optimize the design for conversions, to increase the stickiness of the product.

A researcher does this by conducting regular, consistent usability studies, which are time intensive and complex to plan and analyze, but yield rich design insights. As you learn what is working, or not working about the design of the product, new iterations emerge. Testing these iterations, at a regular cadence, ensures the design changes being made have actually solved problems, and not created new ones.

When we were redesigning our Messenger last year, we had an ambitious goal of creating a business messenger that was as familiar to end-users as the consumer messengers in their pocket. As the lead researcher on the project I knew that in order for us to achieve that goal we would have to conduct regular rounds of usability research with average internet users. Thus we began bringing members of the public to the office, once every three weeks, during the course of the redesign work.

Iterative research succeeded because there was a human-centered design goal defined at the start.

As I was able to anticipate the type of research, and more or less the dates when we would need to run the studies, I did not need to be embedded within the team. Instead I aligned myself with the lead designer Julien. I was constantly seeking opportunities for collaboration with him. He ensured I was invited to important design reviews and scoping meetings, all of which helped me stay connected to the team. This relationship was core to ensuring I had a deep understanding of the problems the designer was working through, which gave me clear design research objectives, and meant each of our studies was having a big impact.

During the redesign we conducted eight rounds of usability testing in our Dublin office (as well as some further remote studies with international participants), and saw first hand the positive impact of the changes we were making to the product.

One of the designs Julien proposed was a range of new message types, and new ways to preview the content of the message. The inspiration for these came from observing people, almost on reflex, closing our messages as they browsed our test website. People told us about their irritation and frustration when these types of “pop-up” messages took over the screen. When we tested these new message types and preview methods, we saw that people were no longer closing the messages automatically, and were instead opening them and reading them when they were ready.

The more usability studies we conducted, the closer we were getting to achieving our design goal. At the same time, we were gaining immense confidence that we were making the right design decisions.

Continuous usability testing can have a huge impact on how a product looks and feels, and that’s why an iterative approach was most valuable for the Messenger product. This approach succeeded because there was a human-centered design goal defined at the start of the project, which guided the research method. In addition, the relationship forged between the designer and researcher ensured a close connection to the product team, where the researcher could trust the designer to include them on important meetings and communicate critical updates and changes.

As a researcher, or the person deciding where to resource a researcher, trying to figure out which team in your company needs a researcher the most can be a daunting decision. Letting product maturity guide this decision, and choosing a working approach that fits the needs of the team, can help you get the most out of the research, and help the researcher achieve greater impact on the products.