I've written before on designing technical interviews. One of the things I focused on in that write-up was the need to validate the skills a role actually requires.
Evaluating the skills properly also gives you a sense of the "level" of candidate you'll need. This is often informed by the existing make up of the team as well as the gaps you're trying to fill.
Suppose you come away from this exercise with the conclusion that you need a relatively senior candidate. Typically this means one of two things.
You're looking for someone who can tackle whatever you need them to and own/drive development with relative autonomy.
You are lacking a subject matter expert in a very specific technology that other people on your team also lack familiarity with.
If you're after number two you can stop reading here. For the rest of you, carry on.
Why senior?
There is a whole question of how we level and determine "senior" status in this industry. But putting that aside, there is typically a difference between senior developers and more junior or mid level people. And typically that difference is based around responsibility.
In most cases, your junior people are your "tool users". They'll have defined tickets given to them to implement. They'll get very familiar with your API signature but might need help accounting for edge cases, might not know how to reproduce a race condition, etc.
These are super valuable members of your team. And they become senior and expert in your tech stack and codebase. They're in it more than anyone else!
In this scenario your senior developers are also familiar with your codebase. They can jump in and solve problems. But you're likely asking them to move around a lot. They may need to debug the build process, test out two potential new tools, run code review and set standards, etc.
What does this mean for interviewing senior candidates? It means it's harder to get right.
Tool user tests
Often, for junior and mid level roles, interviews will consist of a coding exercise. In the ideal case the interviewer is judging based on potential rather than the right answer.
It is a mistake to use this interview for a senior role and adjust your expectations upwards. There are senior candidates who will ace this, but you're testing for the second type of role I mentioned, and you claim to need the first.
Why is this such a mistake? Well, high school me might be a good example.
At the time I sat for the SAT I was enrolled in Calculus. I did fine on my math SAT, not amazing, but fine. My close friend was taking geometry at the time. She'd struggled with math consistently over the years. However, she did far better on the math portion of her SAT than I did. There could be any number of reasons for this, of course. But it was partially due to the fact that she was incredibly familiar with the topics being tested. I was two years rusty in geometry at that point.
Putting senior candidates in a coding exercise and setting your expectations higher is going to play out similarly. You want the senior because they can figure out anything. This type of test won't do that.
So what do you do?
Using the same interview and changing the measuring stick doesn't work well. But it's easy to say what doesn't work, it's harder to design what does.
I've said it before, but you need to match the interview to the job. So what do your senior developers need to do well?
This answer won't be the same for everyone, but let's throw a few things out there.
- Code reviews
- Research or digging into new tech
- Debug tricky situations
- Make sense of new code and provide feedback quickly
- Mentor
- Help define tickets and find edge cases
- Understand or even design your system architecture
- Break down problems into smaller pieces
- Prioritize
This list could go on forever. But you can define interviews for any one of these skills, or all of them.
It's typically around this point that I hear "but what if they can't write code". Somehow this fear is pervasive in our industry. But if they can read code well enough to do a thorough code review then they can write it. If they can debug an unfamiliar project, they can write code.
And if they can't write code? If after all of this they somehow tricked you? That list is a whole lot of value they can seemingly provide based on your interview process.
I'd also argue that it's easier to game a coding exercise than it is to fake your way through a code review.
Signal to Noise
Interviews are all about the signal to noise ratio. Matching your process to the skills you expect a candidate to need on the job means matching the weighting of those skills as well.
Will they be mentoring a lot? Perhaps you should have them help someone through a coding exercise that interviewer hasn't seen before.
Do they need to determine pain points and work towards solutions? Have them talk to people about a real or made up problem and zero-in on the challenges. Then talk about remediation possibilities, pros and cons.
And in the end, look at your entire interview process. Is it focusing on the right skills in the right amounts? If you don't actually care about system design and you have a round on that you may inadvertantly drop the candidate that would have been the perfect mentor, the thing you really needed to hire.
There are so many options that it seems lazy to default to the same interview constructs we use everywhere else. Your senior engineers need to bring more to the table. It's important to your team.