<< The release starts here | Home | Welcome to the waterfall >>

Technical interviews

Executing a decent technical interview is hard. Often you only have an hour or less with a complete stranger to somehow assess technical skills which might take days or even weeks to fully appreciate if you were working with them on project. The urge to make a snap decision then apply post rationalisation can be strong. Its very easy to come up with some excuse and kick a candidate out if you have any doubts. It is better to kick someone out at interview stage rather than hire them then kick them out when they cannot deliver and the damage to both parties is significant. Conversely, every interviewee should be considered an opportunity. You might be talking to the next rising star who helps make your team a success. You should try not to blow it. Candidates should be respected at the very least. They have taken the time and are under going the stress of an interview. Care should be taken not to humiliate them.

I have been dropped into many technical interviews with little or no notice or chance to prepare. I used to find myself falling back into a technique I have been on the receiving end of many times during interviews: Experience roulette. You look at the candidate's CV. You pick a project that sounds similar in technology or problem domain to one you have previously experienced. You feel that you could ask a fairly detailed technical question on this subject that a candidate with good understanding could answer. You ask the question. After a few minutes you realise that the candidates experience is very different to yours for this particular topic. You can't tell whether he is talking garbage or not. All you have proved is that you have different experiences.

Sometimes, and I try very very hard not to fall into this trap, interviews turn into an experiment into ESP. I.e. the interviewer thinks of something really hard, asks a couple of questions and drops a very vague clues. The candidate is expected to read your mind and reel off the exact solution that you thought of. I have seen this done so many times. It is embarrassing if you find yourself paired with a colleague who thinks this is a great way to evaluate another human beings reasoning skills. I have been in situations where fellow interviewers pick up some dusty technical book they have had on their desk for a year (probably as an attempt to convince passers by that they actually read technical books or have any idea about the subject matter). They pick a random page, make a few notes, then wander into the interview and trot out a question based on the summary of the section and expect to get back a verbatim response. E.g. (this really happened), "What is the most important concept in Object Orientated programing?" they will ask. The candidate considers and gives a reasonably considered personal opinion. The interviewer shakes his head sadly and motions to his notes which indicate that polymorphism was considered to be the most important feature in the section of the book he was reading. When this happened I asked the interviewer "Why is that?" and failed to get a coherent answer. No party walked away from that interview with a warm feeling.

I have sat in interviews where the interviewer gets a real kick out of asking the candidate a question they cannot answer or tells them their own solution to the problem and presents it as being far superior. I have noticed that this is most common when someone senior is present. When this has happened to me I just feel like these losers are wasting my time.

Interviews are a two way street. Sure, one of the lanes is much busier than the other but the interviewer should be selling their company and their role, not using the process as an excuse for legitimised bullying.

These days I try and follow a simple formulae which works for me. It might not work for everybody and is certainly not applicable for every technical role in the world, but at least it gives both parties a fighting chance of exchanging some pertinent information.

Before we start, I make sure I know how to pronounce the candidates name, that they have been offered a drink and have had an opportunity to use the bathroom. If at all possible I get a quiet room where we will not be interrupted. If it has a white board all the better. I have interviewed and been interviewed in noisy coffee shops or the like. It looks unprofessional and is not conducive to a good candidate performance if they are self consciously shouting down the rest of the customers.

When we get started firstly I briefly describe the organisational unit or project and my role within it. I do this to give the candidate some context in which to present their experience and so that they know at what level of technical detail they should be pitching their answers. I want to hear concrete, well considered responses not meaningless buzzwords or over inflated project experiences. Letting the candidate know this (without being rude) is sometimes necessary. The temptation is to spend too long setting the context. Often I have seen colleagues (I am sure I have done it myself) spend more than half the allotted time talking about their pet project or favourite rant. I try and keep this section to no more than ten minutes in an hour long interview.

Secondly I ask the candidate to talk about any experience they have that they feel is relevant. I warn them that I may interrupt and ask for clarifications or further explore some interesting aspect of what they are saying. Under normal circumstances I hate interrupting people but when you have a very short time in which to extract the relevant information being polite serves no ones best interests. Sometimes a candidate gets over defensive or dwells too deeply on a point that is not communicating anything of interest to me. I try and gently move the conversation along, sometimes pointing out my objectives for the interview and the time left. There is a risk of experience roulette here but I find just letting candidates talk about what they think is important is often very illuminating.

Finally, and I try and allow enough time for this section, I like to do something practical. Depending on the role this can be a white board based design exercises or a pair programming experience around a laptop or what ever seems appropriate. I have prepared a set of canned exercises which I have refined over many executions.

If interviewing a candidate for the consultancy I work for then usually they will claim to have Java, Test Driven Development, Eclipse and similar sorts of skills. When they do this I feel empowered to sit them down in front of a laptop and have them work through some artificially simple problem. Often I find that if I type and they tell me what to do they do better and get more out of the exercises. It can be a nasty shock to try and use somebody else's development environment with only a couple of minutes preparation time. Role playing a pair programming scenario can be a way to reduce this shock. Often the exercises I set have tests included. Sometimes those tests fail and this gives a clue to one of the possible solutions. I should stress I try and use the exercise as a vehicle for the candidate to demonstrate they way they think and approach the problem rather than expecting them to finish the code. My favourite exercise is based around the Money pattern and allows a candidate to demonstrate good O-O design, TDD and hands on problem solving. I have used it maybe fifteen times. Only one person has actually got the code working and ticked all the possible boxes. Several people have demonstrated excellent skills without getting close to any sort of completion and gone on to be excellent team members. A few people have been thrown completely and failed to make anything of the exercise. This is a shame for both of us as I then have no positive data to base my decision on and they fail to progress. Occasionally it exposes a complete bullshit artist who has blagged their way through to this interview stage. This happened on one memorable occasion where I was doing the third and final interview for a permanent hire. The guy had got through an HR screening interview and impressed a fellow consultant. I had found him hard to pin down in the first sections of the interview but had been generally impressed with his broad experience. We started the practical and it became very clear that he did not have a clue about the tools that he claimed to be expert in and use on a daily basis. After a few minutes fumbling he refused to continue. I terminated the interview and saved myself an hour of my life.

Very often I interview for senior architect roles on behalf of clients where candidates do not have the hands on coding experience and will not be expected to 'get their hands dirty'. In these situations I have a couple of role play scenarios where I describe an ambiguous use case, sketch out some boxes that represent consumers and providers of services and leave a big white space in the middle. I invite the candidate to fill the white space with their solution. I pretend to be a technically vague marketing representative or a shoot from the hip coder (sometimes we interview in pairs and we take one role each). I ask them to fill in the space and inject requirements if I am the marketeer. If I am playing the coder then I try to ask all the questions a coder would ask if they wanted to realise the design the architect is describing. If the candidate is missing something or sometimes if they appear to be having too easy a time of it, I suggest incorrect or over elaborate solutions and wait to be shot down. I keep on going back to the original requirements to make sure they have been fulfilled.

Getting the scenario right for these exercises is tricky and I have to admit to significantly refining the problems over time. Getting the balance between a problem that consumes too much time and a problem that is over simplistic is a challenge. The best problems I have are based on real project experiences where I can add elements to them in order to increase the complexity if the candidate is doing well.

I have found that you can have a run of candidates who make a real hash of these practical problems. You begin to doubt the validity of the exercise and start to reassess candidates you at first dismissed. In my experience usually the run of dross is followed by a few stars who re-affirm your faith in our profession and the exercise. Whether this is just luck of the draw or reflects the HR or agency reacting correctly to feedback on candidates I do not know (it should be the later but I suspect the former in many cases).

I find the practical exercises useful and actually enjoy watching people solve the puzzles, especially if they do so with panache. At the very least doing a practical exercise gives you concrete evidence to support your decision if you are called to justify it.

These experiences are mine and may not work well for everybody. I have found them useful and have consistently hired good people. I find that this approach delivers much better results than quizzes of programming language constructs or ad hoc conversations around project war stories. Then again, maybe I am still post rationalising and have simply constructed a process that favours people who appeal to me...
Tags :

Add a comment Send a TrackBack