Intercom's product principles: Building in small steps to deliver maximum customer value

Large changes are hard to understand, and harder to debug.

At Intercom, we deliver complex changes in a series of small, controlled, easy-to-understand steps. Small changes are easier to build and faster to review, enabling us to deliver value to customers more quickly.

This is the seventh post in a series exploring our product principles. Here, Aidan discusses our engineering principle “Build in small steps”.

No one gets it right all the time

Mistakes happen on every team, in every company in the world. Once you accept that you’re not going to get it right all of the time, you can adjust in one of two ways:

• Try to correct mistakes before you ship them, taking steps to validate or check your work.
• Allow yourself room to be wrong, learn from the mistake, and adjust quickly to correct it.

If you’ve already sunk weeks into a change, there is often less room to be wrong. This can lead you to rely on validation to avoid surprises when you ship your change. Validation has its place, but it’s a poor substitute for deploying something for real. The more validation you need to carry out before shipping, the longer it takes before you can iterate or move on to the next feature – it ultimately slows you down.

When shipping a change, we aim to control:

• The number of variables affected: The more variables affected during a change, the more difficult it is to figure out which part of the change has caused the problem. By shrinking the size of each step, we tighten the feedback loop and set ourselves up to learn and adjust much faster if we need to.
• The size of the change: By shrinking the size of our changes, we also reduce the blast radius of each change. It’s important to test changes but there’s only so much that upfront validation will catch. Smaller changes let us focus our attention on incrementally reaching your goal, and not get too caught up in ensuring each change is perfect.
• Which customers experience the change: We rely on feature flags to validate changes in production and incrementally release them to customers.

We don’t know for certain whether the feature will solve a customer’s problem until they have it in their hands. This commitment to shipping small iterations quickly ties into another Intercom principle: ship to learn.

Managing complexity

Building within complex systems is challenging. When bugs crop up in large changes, or bloated features miss the mark, it’s hard to pinpoint the specific issue. Small steps make it easier to validate, change, and move on – confident you’re building on solid ground.

Large changes include lots of assumptions:

• External assumptions about how your change will impact your customers’ workflows.
• Internal assumptions about how different parts of your change interact and depend on one another.

While you can do your best to check external assumptions, it’s often faster and more robust to ship your change and validate or adjust. Internal assumptions are where complexity can creep in. When your change gets large enough to encompass several building blocks that all depend on one another, testing them together can be risky. It’s much safer to release these incrementally, building one on top of the other and monitoring the impact as you go.

Durable speed builds momentum

Speed is great but durable, reliable speed is game-changing. Shipping a large change means there’s a lot more riding on it being successful, and a higher risk of surprises compared with a series of small, quick iterations.

“A tight cycle of shipping small changes, learning, and iterating builds strong momentum”

A tight cycle of shipping small changes, learning, and iterating builds strong momentum. It eliminates the need to be right first time, encourages faster decision-making, and reduces the blast radius of mistakes. Moreover, breaking work into smaller units means engineers can progress work in parallel, allowing the team to move faster as a whole.

Building in small steps requires the right team culture

Building in small steps doesn’t happen by accident, it takes deliberate intent and the right environment. Our team culture and infrastructure stack play a crucial role in our ability to ship small changes quickly.

Once teams buy in to the importance of shipping small changes quickly, peer reviews are prioritized and completed faster. Since small changes are easier to understand and review, there’s more chance bugs will be caught at each stage. This team understanding and urgency speeds up the entire process.

“We’ve invested significantly in ensuring that once a change is reviewed and merged to master, it takes less than 15 minutes to hit production, including automated testing and staging validation”

When deployment times slow down, engineers are more inclined to batch changes, resulting in a cycle of larger changes. We’ve invested significantly in ensuring that once a change is reviewed and merged to master, it takes less than 15 minutes to hit production, including automated testing and staging validation. It’s completely hands-off and engineers get an automatic Slack notification once the change is live.

Applying the “build in small steps” principle to Intercom’s Salesforce integration

Last year, we looked to integrate Intercom more deeply with Salesforce, allowing customers to automatically create Salesforce cases from Intercom conversations. We quickly understood the complexity; conversations and cases don’t always map straightforwardly and syncing data across a many-to-many relationship would be challenging from both an engineering and design perspective. Added to this, there was a wide variance in how customers wanted to use this feature and it needed to fit within the existing integration which customers depend heavily on.

After working through the many implications and design tradeoffs, we parked the project in favour of something we felt would deliver more reliable impact. We almost didn’t build it because we approached it as a large change instead of a series of small steps.

We revisited the project with a different approach

It didn’t take long before our Sales team came back to highlight just how important this feature was to our customers – and we decided to give it another look. This time we took a different approach, starting with the smallest possible version and sidestepping some of the complexity until we could learn what customers really needed.

“The customers we worked with really valued the pace at which the team iterated and how the feature evolved on a day-to-day basis, guided by their feedback”

In two weeks, another engineer and I built the most basic version of this feature that we could share with customers. We did this in many small steps ensuring we didn’t break any of the existing workflows customers were already using. This made the feature far more tangible and customers were able to give specific feedback about the product gaps and improvements.

Once customers were using it, the team iterated in small steps, quickly making the feature more flexible. As the flexibility grew, so did the number of customers using it and we rapidly expanded the beta.

It turned out that the many-to-many relationship didn’t block customers from using the feature and we successfully launched it safely and without this extra complexity, something we only discovered by starting small and iterating quickly. The customers we worked with really valued the pace at which the team iterated and how the feature evolved on a day-to-day basis, guided by their feedback.

Building in small steps works for everyone

We build in small steps primarily because it helps us deliver customer value faster in a safer and more durable way. But as well as that, as an engineer, it’s a nicer way to work. It’s far less stressful than shipping big changes where there’s a lot riding on you being right first time – and you get a regular dopamine hit each time you ship what you’re working on to production.

So, if nothing in this blog post convinces you to optimize for risk reduction and delivering incremental value, you should do it because it’s just more fun.

Interested in learning more about the way we work at Intercom? Find out more.