Intercom's Darragh Curran on scaling engineering

Darragh joined Intercom in early 2012 as a product engineer and our second outside hire. Fast forward to today, and he’s our VP of Engineering, has grown our engineering team to more than 90 employees, and Intercom regularly ships code to production more than 100 times a day.

Darragh joins me on the podcast to explain how he’s scaled our engineering team and culture. Our chat covers the values that create his team’s foundation, what he looks for in new hires, working across disciplines, and much more.

If you like what you hear, check out more episodes of our podcast. You can subscribe on iTunes or grab the RSS feed.

What follows is a lightly edited transcript of the interview. Short on time? Here are five key takeaways:

1. In order to grow their team’s capacity and output, engineering managers must work on the unglamorous yet frequent things that will unblock their team.
2. One of Intercom’s key engineering values is to do less better. In Darragh’s words, “A very well polished sliver trumps a big, fat, janky slice” of a project.
3. When hiring, Darragh aims to identify engineers who demonstrate the potential for a steep trajectory over those who appear to have the skills needed today. It’s then on the team to help new hires grow.
4. Engineers, designers and PM’s at Intercom work on teams together, allowing them to share context, have tight feedback loops and iterate towards the right solution.
5. Engineers looking to join an early-stage startup should evaluate three things: Are the people genuinely excited about the problem they’re going to solve? How do they approach their work? And what do they value?

Des Traynor: Today I’m privileged to be joined by our Vice President of Engineering, Darragh. It’s good to have you on the show. How many engineers did Intercom have when you started, and how many do we have today?

Darragh Curran: There was four when I started. It’s at eighty or ninety today.

Des: That’s quite a change. You recently wrote, when you were celebrating your fourth year with us, that you were pretty convinced on day one, after a pull request of all things, that you’d joined the right company. What was that about?

Darragh: Before I’d started I got access to Intercom and was trying things out. I was playing with the product and something annoyed me, a little detail. There was bad spacing on the login form, and I wanted to fix it. At Amazon and previous companies the way in which we’d make changes involved some deliberate peer review – sharing explicitly the change you wanted to make with your peers and getting feedback on that. Before I joined Intercom, while some of that might have happened informally, it wasn’t a deliberate process.

This tiny little detail that annoyed me was an opportunity to propose introducing that workflow to the teams, specifically to Ciaran, Ben and David. The idea behind deliberate code review is that it’s a really great opportunity for us as engineers to improve upon our work, to learn from the experience of others, and to help educate people on the changes we’re making. It represents an environment that appreciates respectful and critical feedback on the work we do. For engineers this is like the core tool channel that happens in.

Darragh’s first pull request at Intercom

The confirmation I got on joining being the right thing was almost a meta-version of that, the fact that I could make a change that introduced this new workflow and it was well received, confirmed to me that there was a healthy, progressive mindset in the people I was joining.

Des: I remember thinking, we worked so hard to recruit you, and your first significant contribution was to remove some white space! Thinking back to those times, you were our first experienced engineering hire, and there’s not a lot of management of engineering to be happening when it’s basically yourself and our CTO, Ciaran. You were working on a lot of features. What stands out from that period?

Darragh: The thing I’m most proud of is the work I did on an internal tool we have called Muster. It’s basically a tool that manages our infrastructure and how we deploy. We were quite a small team, four engineers, but we were working at an incredibly fast pace, making dozens and dozens of changes a day. The simple, small step of pushing those changes to Heroku, which is what we used at the time, was becoming tedious and troublesome. We would not realize we were pushing other people’s changes, and it was a distraction to have to change and wait and type the command to deploy.

It basically took us away from our flow, and our service was getting big enough that Heroku was not really fulfilling our needs. Every time we deployed, it caused little outages, because of the way it handled queuing up requests during that period. We could all anticipate that our needs to move fast as the team scaled would just increase and increase, so we invested in a tool that would enable us to continually deploy. Along with the deliberate peer review, once a peer agreed that this change was good to go, and along with automated testing, we now had automated deployment. We could focus more on building the product than on the tedious work of safely deploying it.

Des: My understanding of Muster, from a distance, is it’s basically the core of why we can ship so often. How often do we roll out these days?

Darragh: It’s typically well over 100 times a day. Changes from all engineers and all teams.

Intercom releases per day

Des: Does Muster still work?

Darragh: Muster still works. It’s hit some challenges along the way, but the basic principle of it is quite simple. When a new change appears, prepare it in a way that’s repeatedly deployable and safely manage that deployment. Along with it comes the ability to safely roll back and to scale up or down different parts of the infrastructure.

Career after code

Des: When I called by your desk earlier, I certainly didn’t see much code on the screen. That’s no surprise with 90 engineers here. How have you dealt with the gradual moving away from writing code to doing things that are incredibly valuable but very different?

Darragh: If I’m honest, there was never one sudden transition. It wasn’t a point in time. In the early days there wasn’t really much management overhead within the team. We all loved engineering, we loved building product, so that’s what we focused on. As the team grew, there was a greater need for leadership and management.

You’re taking on the unglamorous but frequent things that unblock your team.

Progressively over a period of time the work that I did was very much steered towards leading and growing the team and not on contributing as an individual. What that looks like in the middle is that you’re working a lot around the periphery, you’re never taking on the big critical part. Instead you’re taking the often unglamorous or small but frequent things that will unblock your team. You’re investing your time in enabling your team to be effective, enabling them to grow, and therefore grow the capacity of the team and the output it can achieve – rather than what is perhaps at the time more comfortable or easier, to contribute that capacity yourself.

Des: That’s the biggest mistake engineer-turned-managers make: they try to engineer their way out of every management problem. Did you have to train yourself not to do that and not be like, “Just give me the keyboard, and let me do this!”?

Darragh: That default is very hard to avoid. The worst version of that is if you see someone struggling, to just go and do it.

There are some intermediate steps where you’re not fully detached but you’re getting to the outcome you want and using the journey to it as an opportunity to teach someone. It might be tedious at the time, because you feel like it could be quicker if you do it yourself, but the idea would be that you do this two or three times and it starts to pay off.

Des: A near-joke question, but given how distant you are from code, when did you last see a piece of code? Do you have any clue how Intercom works?

Darragh: I still have a very good grasp of how Intercom works, and I’m exposed to a lot of the work. For various different reasons I’ll still end up looking at code, perhaps interesting pull requests or interesting postmortems on problems, so that I can properly appreciate the challenges and know how we solve them.

In terms of writing features, it’s actually been years. The period in between was where, again, I was doing a lot of this unglamorous work around the periphery. Two things I remember in that recent period are trying to close out on an important re-factoring to remove some technical debt, which I may or may not have been responsible for but I was quite familiar with. I also recall an incident that happened with our system where there was an all hands on deck response. I was randomly able to contribute a fix, or a failing test or something, that helped us overcome that. At the time I was like, cool, I’m not too rusty. I don’t know how I’d fare today to be honest.

Engineering core values

Des: You talk a lot about our engineering principles, which are printed all over our walls. There’s people, there’s process, but the commonality you push for is the principles that they work through. What are your main engineering principles?

Darragh: The first one we talk about is to be deliberate. What this really means is to be very thoughtful about how you spend your time and how you make decisions. If you’re not actively doing that the reality is that you’re inadvertently, passively wasting time or making inefficient decisions.

A really important one for me is the idea to think big, and also horizontally. Particularly as we’ve grown, the organization is structured into groups and teams and people are very focused on one part. That’s where they’re going to put most of their energy, but we constantly want to remind people to not constrain themselves to that explicit area of focus. We all have the ability to contribute and improve and have impact beyond the team we’re on. Thinking about it in that way helps us spot opportunities where perhaps our team structure isn’t best supporting the outcomes we want and we have an opportunity to change that. Perhaps a bit of product work or engineering work might be right for the team but wrong for Intercom, and you’ve got to think holistically to figure that out. If people aren’t thinking like that then you rely on process and management to correct all those things.

The one represented in the Muster example we talked about earlier is the idea to move fast but optimize for the long term. Again, this is one of these things that if you’re not deliberate about it, you lose. Your ability to move fast as a team, just by pure physics almost, tends to decrease as the organization grows. It’s the kind of thing, when you lose it, it’s almost impossible to regain. The team has grown, the code has grown, the systems are bigger. The process might be a bit more heavyweight. You can just bow down to that and let that knock you over, or you can try to deliberately defend that ability.

Engineering team values on display in the Intercom office in Dublin.

To us, that’s represented in our ability to ship code and our ability to make decisions. It manifests in a bias towards simplifying things and fighting against complexity. The principle that code is almost a liability, the less we have the better, and the important thing being that you can make the trade off between being fast now and not caring about the future or being fast now because in the past you’ve cared about the future. That’s where we place ourselves.

The next thing I care about is the idea of doing less, better. A very well polished sliver trumps a big, fat, janky slice of a thing. While we do think big and have these grand visions for what a thing should be, we always start small on the most confirmed valuable piece. It gives us opportunities to learn, build momentum and then ultimately we move faster towards the bigger goal.

A very well polished sliver trumps a big, fat, janky slice.

To remind people where we came from, there’s the idea of building great product. While we are software engineers, the outcome success for us is producing great product.

Des: I can image different sides or different types of engineers, some are going to be primarily back-end, some are going to be primarily front-end. Is it important they all care about product? As in, if you’re working on EC2 deployment, versus a user-facing feature that everyone sees, like our messaging form, does the principle apply?

Darragh: It absolutely does. There’s two parts to that. First, the engineers that join Intercom tend to travel around quite a lot within the team. They’re not confined to one problem. Aside from that, even the most back-end-oriented folks still have to make tradeoffs and decisions in how they design their systems. The classic tradeoff might be designing a system that needs to respond in milliseconds versus one that responds in minutes. You’ll build them in totally different ways. They might cost or take more money or time, but both of them are valid answers depending on what the customer need is.

A lot of good behaviors stem from empathy for the customer. Understanding how people use the thing. Not only that, it’s that curiosity and understanding fuels us as people. The thing I did had an outcome and made something better, so for a whole number of reasons the more we keep that connection the better.

Hiring for potential

Des: One thing you talk about a lot is this idea of hiring for the potential of people as opposed to hiring for perfect job fit. It seems like there’s two ways you can go if you’re hiring a certain role. You can hire somebody who can grow into it, or you can hire somebody who’s definitely going to be effective on day one. How do you think about that?

Darragh: One of our explicit principles is to hire for potential and help each other to grow. We think about that very deliberately on the side of trying to identify people who demonstrate the potential for that steep trajectory over the people who appear to have the skills you need today. Obviously the intersection of those two is great, but if you have the latter without some elements of the former and that person turns out not to be good at adapting or learning or growing you’ve got a problem ahead of you.

Des: The risk is the company will outpace them at some point, or just because they’re good for 200 engineers they might not be good at 600 engineers.

Darragh: Or they’ll come under a lot of pressure, they’ll fall behind or people around them will out-deliver them. When new problems emerge you’re not going to have anyone who’s done it before so you want someone you can trust can adapt.

The growth mindset

Des: Some of that to me sounds a lot like this idea of the growth mindset, which I know is something you’re pretty familiar with and considerate of. Talk us through the growth mindset and why it matters to you.

Darragh: I first came across the actual term “growth mindset” a few years ago. I believe it was coined by Carol Dweck in a book called Mindset.

People either have a fixed mindset where they believe that their ability or intelligence is somewhat static, or people have a growth mindset, where they believe intelligence or ability can be developed.

Infographic by Nigel Holmes for Stanford Magazine

All the people you likely want to hire probably demonstrate elements of the growth mindset, which is someone who will embrace challenges, who will persist in the face of roadblocks, who sees effort as the path to mastery, who craves and learns from critical feedback and who finds lessons or inspiration in each success. That’s how that mindset is outlined, and as result of all those things people reach higher levels of achievement and with that comes a greater sense of free will.

Des: An interesting thing to me is a lot of our engineers don’t fall into the older fallacy of basically boxing themselves in and believing that they’re only here to engineer. I always like to see engineers push back on design or push back on product decisions. That comes from wanting to grow outside of their role and understand the full context of what they’re doing. Is that something you’ve deliberately worked on? How is the relationship between product people, designers and engineers different at Intercom?

Darragh: We’re all product people. Our motivations are to create great product, and we’re proud when we do that. I’ll probably be a little bit proud when I create some beautiful code, but it’s really the thing it enables that gives me the fulfillment. I think that’s true for all disciplines. It’s not the fancy Photoshop file; it’s the thing that’s working that’s making people’s lives better in some way.

Not all people think like that, and not all companies structure themselves like that. A lot of engineers are more orientated around the technology, often as a side-effect of the way things are structured. They obsess around the technology. They invest in and speculate about how the technology will evolve, and that’s where all their opinions are directed.

What do we do differently? In my career before Intercom, the thing that was different was that we were still ultimately trying to build a product and all these different disciplines were separated. They were siloed. You might not even know the person that designed the thing that you’re building or who the product manager is and what they actually do.

Des: How does that feel as an engineer? Does a work order arrive in your email and you’re just like, well, better go and do this?

Darragh: Some people, perhaps those more in that fixed mindset, will just see that as, okay, I’ll follow orders. For me, I always constantly found myself in that situation where I want to understand why. It’s all about context really. Being in situations where you’re like, “I think I know what the context is, and if I’m right then I think there’s a better approach here.” If you haven’t been a part of the conversations that would enable that, there’s probably a way to retro-fit that conversation, but it’s a tremendous amount of friction and a slow process.

A combining trend has been how we ship software. Twenty years ago, companies released software on very long cycles, yearly or whatever, and the software processes that existed adapted around that. We could talk at length about waterfall. The premise around all of those to me is some either naïve or accepted ignorance around the idea that we actually know what the right answer is. We’re going to release this thing in a year, we know what we’re doing, so we can serialize everything. Let’s figure out what problem we’re solving, let’s design the thing, and pass that to the engineers.

We’re more often wrong than right. The important thing is iterating towards right.

It sounds perfect, and it probably is if you’re building a bridge or a car or something, but software lends itself to the ability to be wrong and change. It’s such a new medium that we’re more often wrong than right. The important thing is your ability to iterate towards right. How you structure your teams can either help or hinder this. Engineers, designers and PM’s being together means that they’re closer, they can all share the same context and they can communicate freely and, along with delivering software regularly, have tight feedback loops and iterate towards the right solution, rather than guessing.

Des: I presume in the other orgs, they’re not doing it just to be stubborn. What’s the drawback of doing it your way, as in, forming software teams with designers, engineers and PM’s?

Darragh: The classic trade-off for me, which manifests less for engineers than, say, designers, is that in the structure I described there, we might have one designer on the team. That person is now in their discipline quite isolated. Their manager is probably less connected to the work they’re doing, but also they’ve less opportunity to benefit from the input of peers who might have the right context. There are definitely ways that we try to mitigate that.

Lessons learned scaling the growing team

Des: Thinking about the scale of Intercom over the last four years, I presume you’ve learnt a few things about hiring and team management, performance, morale, etc. As we’ve gone from a couple of engineers to more than 90, what are the key repurposable things if you were to do it again? What areas do you need to focus on, and what lessons have you learned the hardest?

Darragh: I joked to myself at one point that I hadn’t transitioned from an engineer to a manager, I’d transitioned from an engineer to a recruiter.

There’s been a couple of distinct hiring phases. You have the early days, where the company is small and it’s a huge risk to join. Although you might be excited the reality is that most startups actually diein those early stages. Your pool of people that you can reach, because no one’s heard of you, and that will take the risk, is small. It tends to be people that can offset some of that risk because they know you and they know that what you’re doing is legit, and they might have the context. So you hire from people you know. That can get you quite far.

Then you get to this point where you’ve tapped that out to some degree, and you still don’t have a brand to rely on. You don’t have much inbound interest, so you go into this phase where you’re basically knocking on doors. You’re trying to sell the thing you’re doing, you’re trying to find people who have that particular fit for the risk and opportunity. That’s the phase I identified as that recruiter phase.

There’s a parallel in the product. In the early days it’s friends and old colleagues using the product. Then you’re knocking on doors.

The third phase is where the company brand establishes and we’ve got local awareness. People start coming to you. The thing that you learn, in parallel with the product and marketing side, is that you’ve got this inbound demand. The analogy might be, wouldn’t it be great if they were all customers that pay you a million bucks a year? Wouldn’t it be great if they were all experienced senior folks rather than interns or whatever it might be? We did quite well in that phase of attracting talented folks. The phase we’re in now is were I feel almost we’re coming back to the deliberate recruiting, knocking on doors phase, where we know a lot more about the specific folks we need.

Darragh’s original Inside Intercom welcome post. Read the whole post here.

Des: If you could talk to a younger Darragh four years ago, or any of our listeners who are the first engineering hire at a startup that they believe is going to grow and scale, what would you advise them? Specifically, are there things you think you could have learned from somebody else along the way?

Darragh: If you’re considering embarking on something like this, look for three things. Possibly the most important thing, that people sometimes skip, particularly if they’re running away from a job they don’t like, they skip the bit of the thing they’re going to work on. Are they genuinely excited about the problem they’re going to solve, do they believe in it, and do they have a passion for it?

Des: I’ve seen that before, the rebound career. They hated one job so they jump onto the next one without really vetting it. They’re just happy to be rid of the one they had.

Darragh: Exactly. That’s probably one of the most important ingredients, but they others are very important as well. One is the people you’re going to join. Are they like-minded, do they value the same things? Are they the types of people you’ll learn from?

Then, the approach and mindset. The way they approach their work, what they care about, what they value, and how they do things. For something to be long-term sustainable, you’ve got to have all three of those. Just as base level advice, seek that out and that’s a good foundation for success.

With something like Intercom or any ambitious ordeal you’ll take on, it’s a long haul. It’s not going to be two years and then retire to a desert island. With that in mind, I think there’s a couple of bits that are really important. You should make sure that you’re enjoying it. It should be challenging, it should be hard, but it should be fun and rewarding. You should be doing things that fulfill you.

The other thing that I’ve invested in maybe better of late is to do things that preserve and increase your energy. If you think of a role like this as something that will be five to ten years, you don’t want to burn out. You can work as hard as you want for a number of years, but if you’re not sustaining yourself you’ll eventually burn out. You’ll get them in work and outside of work.

Des: Looking back four years, any regrets?

Darragh: No, not really. The biggest guiding principle to avoid regrets is to take an approach where it’s okay to be wrong, but to do that in a way that you act fast and learn fast. You can’t regret being wrong if you can adapt from it.

Des: Makes sense. Darragh, thanks very much.

If you’re interested in joining the R&D team to help build Intercom, check out our current openings here.