Main illustration: Luis Mazón
Interview assignments have become a common component of the hiring process for engineering roles.
These technical problems, also known as email screeners or, as we call them at Intercom, take-home tests, are a useful way to initially evaluate the technical ability of candidates applying for engineering positions.
Here, we examine the take-home test and offer some practical advice to candidates looking to put together the best submission they can.
What are interview assignments and why do they matter?
The take-home test is usually a technical problem, or set of problems, that is provided to candidates as part of the interview process. Candidates are asked to solve the problem in their own time and submit the solution to the company, who assess it and use it to screen candidates.
“The core job of the take-home test is to set some kind of bar for candidates before proceeding to the more expensive parts of the interview process”
Every interview assignment has a different type of problem to solve, but ultimately they all serve as some form of technical screener. The core job of the take-home test is to set some kind of bar for candidates before proceeding to the more expensive parts of the interview process, for example, a call with an engineer/product manager or an on-site interview with the whole team.
Take-home tests only provide a single, simple measure of a candidate’s value, but can often feel like a high-pressure hurdle to surmount.
Engaging with a take-home test can be a daunting and time-consuming process for a candidate, so let’s demystify it and offer some practical advice.
Where do take-home tests fit in the interview process?
If you’re interviewing for a job at Intercom, we aim to develop a holistic understanding of the way you think about building product and approach engineering problems. We want to ensure alignment with our company and engineering values (often called “culture contribution”). Each section of the interview loop is designed to collect enough data to confidently say, “Yes, we want to hire you!”
We look to collect a wide breadth of information from a variety of perspectives to help us inform the ultimate decision on a candidate. Interview assignments represent one part of the larger interview process, which looks broadly like this:
- Call with a recruiter 📞
- Take-home test 📝
- Call with an engineering manager 📞
- On-site 🏢
The initial call with a recruiter precedes the take-home test, and only after successfully completing that assignment will a candidate talk with an engineering manager to discuss technical and cultural contribution. After this, we bring the candidate on-site and have a number of sessions to collect data, from different perspectives, on your capabilities.
Through the looking glass – empathy for the reviewer
It’s useful when putting together a take-home test submission to build a little empathy for the reviewer. Reviewers are typically full-time engineers and reviewing take-home tests is typically part of their normal day-to-day responsibilities, often just a small slot in a busy calendar.
For a typical candidate, the take-home test is a significant commitment: it could be two precious hours of your evening after a stressful day at work, time you take over your weekend or one of a number of parallel tasks while pursuing many job opportunities.
“Playing to your strengths is a good strategy when applying for jobs”
Once you’ve made your submission, the people who do the actual review will only have a short, focused period to assess its quality. It’s often surprising to candidates how short a window you have to ensure we can see the quality of your work. You’ve got to make it count.
This might sound daunting, but it shouldn’t be. There are some simple things that you can do to make it easy for us to assess your submission. Here is a short list:
- Provide a README with your submission – where’s the main entry point? Where are the tests? How are they run? At Intercom we allow you to write in any language when submitting as we are problem solvers, not language/framework experts. But it means we won’t always be familiar with your language.
- Reviewers typically won’t execute your code so consider providing some sample output.
- Your reviewer is human. It’s much easier to read idiomatic, well-structured code with clear naming. Need to learn how to do this? There are many resources online, like Practical Object Oriented Design.
- We favor building boring software in Intercom and in our experience, most excellent submissions are boring (although the reverse is not always true).
How do we assess a take-home test?
Unconscious bias is a well-researched problem in our field, and it’s important for reviewers to eliminate as many potential sources of bias as possible. It’s crucial to be clear about what we’re looking for when reviewing an interview assignment, and it’s just as important to know what we’re not looking for. When assessing a technical submission, we don’t care about:
- The candidate’s experience level.
- The position they are applying for.
- Their CV or professional history.
None of these details are relevant at this stage. All we assess is the take-home test itself – we only care about your code!
The technical problem should be understood and addressed in the solution. The technical quality of the submission should be high and the code should be clear. One of our mantras at Intercom is that shipping is our heartbeat and that we expect that new hires will ship code to production early and often.
“We don’t drive without seatbelts, and we don’t ship code to production without tests”
Our expectations for the take-home test align with this – we explicitly ask for and expect production-ready code. This means many things, but the most obvious is that we expect it to be well tested. We don’t drive without seatbelts, and we don’t ship code to production without tests. We would never pass a submission without tests. Tests should be effective – would they fail if something was broken? Would it be obvious what had broken?
But what about the test? What is it?
This might be surprising to discover, but the specifics of the test aren’t important. Some problems will need to be solved, but that’s not the most interesting part of the assignment.
We’ve passed great submissions that contained small bugs – it’s unrealistic to expect engineers to never ship bugs, and we value a disciplined approach to solving a problem more than correctness at this stage. The instructions to the test will tell you what the reviewer is looking for. Language is important. Words like “must” are important, as are words like “should.” As one of our favourite teachers used to say when talking about exams:
- Read the question, understand the problem.
- Read the question again, make sure you understand the problem. Underline what you need to answer.
- Solve the problem.
- Read the question again and check that you actually solved the problem.
To summarize, it cannot be understated that you need to understand the problem when doing take-home tests. If the assignment asks for tests, you should write tests. If it asks for a README, provide one. It’s all too easy to jump into problem-solving, but be sure you provide what you’re being asked for.
Demystifying the take-home test
It’s worth recapping that just like whiteboarding sessions during an on-site interview, take-home tests are imperfect. But just because they’re imperfect doesn’t mean they’re useless.
“We do our best to make them feel more personal with some helpful and constructive feedback”
Last year Intercom had a few hundred applications to a pool of intern positions. How do you possibly give each and every one of those a great personal experience? It’s a difficult scaling problem. When using take-home tests in the interviewing process, we don’t expect it to function as a perfect filter that progresses every good candidate and excludes those who don’t “meet a bar.”
Instead, we have to accept that the process is imperfect and if it’s a “no” on the take-home test then we at least owe a duty to offer some practical feedback to the candidate so that they can improve for their next interview assignment. It’s too easy to make take-home tests feel impersonal, but we do our best to make them feel more personal with some helpful and constructive feedback. We owe candidates that much.
We hope we’ve demystified the take-home-test process and given some good pointers that will make the prospect of tackling these interview assignments less daunting, and given you some confidence when applying for your next position. 😄