Home | December 2008 >>


We are still getting distracted from our main task. Two of the four developers have spent more than 50% of their time looking at a production memory leak. I am spending a lot of time doing interviews and attending meetings. All of it is important but it is impeding progress badly.

Welcome to the waterfall

We have made some progress in the last week of development but it has not been as I envisaged. We have spent a lot of time talking through the use cases and that has helped the dev team understand some of the issues facing us. Those use cases have been typed up using our standard template but they lack structure.

I traveled to overseas for a technical alignment meeting. 05:30 start. Back home at 21:30. Shattered. All for thirty minutes where I stepped through one of the sequence diagrams. It was helpful as for that one use case we have agreement on the design. Half an hour for an extended day (which took me another day to recover from) is not startlingly efficient.

What we have failed to do is produce the product backlog. Instead we have identified some massive issues around complexity and spent days trying to thrash them out. This has also involved presenting back to the stakeholders and trying to educate them on the relative cost in complexity in comparison to the possibly limited returns in functionality. We failed to make any impression and were told that they would reconsider but the the requirements would stand unchanged. Not sure who is doing something wrong here.

We are now slotting back into the feasibility process that the clients financial processes mandate. We are producing very short summaries of the options, impact on required functionality and potential risk (expressed as a percentage). We are doing this for the entire scope of the project (9 months). This does not feel agile at all as we are having to sketch out designs for the entire system. We cannot get round this as completion of the feasibility is a payment milestone.

I am also finding that the feasibility document in its current form has been overloaded with very detailed design narrative. The narrative captures the developers thought process through out the exercise. It makes the classic mistake of documenting parts of the solution we have since discounted because the author has a significant emotional investment in those words and cannot bear to see them all deleted. The detailed design narrative also needs to go (hopefully). I am thinking of it as necessary scaffolding for the developers thought process. It helped them organize their thoughts and was useful. Now it needs to be binned as it is already out of date and has no audience.

Technical interviews

Executing a decent technical interview is hard. Often you only have an hour or less with a complete stranger to somehow assess technical skills which might take days or even weeks to fully appreciate if you were working with them on project. The urge to make a snap decision then apply post rationalisation can be strong. Its very easy to come up with some excuse and kick a candidate out if you have any doubts. It is better to kick someone out at interview stage rather than hire them then kick them out when they cannot deliver and the damage to both parties is significant. Conversely, every interviewee should be considered an opportunity. You might be talking to the next rising star who helps make your team a success. You should try not to blow it. Candidates should be respected at the very least. They have taken the time and are under going the stress of an interview. Care should be taken not to humiliate them.

I have been dropped into many technical interviews with little or no notice or chance to prepare. I used to find myself falling back into a technique I have been on the receiving end of many times during interviews: Experience roulette. You look at the candidate's CV. You pick a project that sounds similar in technology or problem domain to one you have previously experienced. You feel that you could ask a fairly detailed technical question on this subject that a candidate with good understanding could answer. You ask the question. After a few minutes you realise that the candidates experience is very different to yours for this particular topic. You can't tell whether he is talking garbage or not. All you have proved is that you have different experiences.

Sometimes, and I try very very hard not to fall into this trap, interviews turn into an experiment into ESP. I.e. the interviewer thinks of something really hard, asks a couple of questions and drops a very vague clues. The candidate is expected to read your mind and reel off the exact solution that you thought of. I have seen this done so many times. It is embarrassing if you find yourself paired with a colleague who thinks this is a great way to evaluate another human beings reasoning skills. I have been in situations where fellow interviewers pick up some dusty technical book they have had on their desk for a year (probably as an attempt to convince passers by that they actually read technical books or have any idea about the subject matter). They pick a random page, make a few notes, then wander into the interview and trot out a question based on the summary of the section and expect to get back a verbatim response. E.g. (this really happened), "What is the most important concept in Object Orientated programing?" they will ask. The candidate considers and gives a reasonably considered personal opinion. The interviewer shakes his head sadly and motions to his notes which indicate that polymorphism was considered to be the most important feature in the section of the book he was reading. When this happened I asked the interviewer "Why is that?" and failed to get a coherent answer. No party walked away from that interview with a warm feeling.

I have sat in interviews where the interviewer gets a real kick out of asking the candidate a question they cannot answer or tells them their own solution to the problem and presents it as being far superior. I have noticed that this is most common when someone senior is present. When this has happened to me I just feel like these losers are wasting my time.

Interviews are a two way street. Sure, one of the lanes is much busier than the other but the interviewer should be selling their company and their role, not using the process as an excuse for legitimised bullying.

These days I try and follow a simple formulae which works for me. It might not work for everybody and is certainly not applicable for every technical role in the world, but at least it gives both parties a fighting chance of exchanging some pertinent information.

Before we start, I make sure I know how to pronounce the candidates name, that they have been offered a drink and have had an opportunity to use the bathroom. If at all possible I get a quiet room where we will not be interrupted. If it has a white board all the better. I have interviewed and been interviewed in noisy coffee shops or the like. It looks unprofessional and is not conducive to a good candidate performance if they are self consciously shouting down the rest of the customers.

When we get started firstly I briefly describe the organisational unit or project and my role within it. I do this to give the candidate some context in which to present their experience and so that they know at what level of technical detail they should be pitching their answers. I want to hear concrete, well considered responses not meaningless buzzwords or over inflated project experiences. Letting the candidate know this (without being rude) is sometimes necessary. The temptation is to spend too long setting the context. Often I have seen colleagues (I am sure I have done it myself) spend more than half the allotted time talking about their pet project or favourite rant. I try and keep this section to no more than ten minutes in an hour long interview.

Secondly I ask the candidate to talk about any experience they have that they feel is relevant. I warn them that I may interrupt and ask for clarifications or further explore some interesting aspect of what they are saying. Under normal circumstances I hate interrupting people but when you have a very short time in which to extract the relevant information being polite serves no ones best interests. Sometimes a candidate gets over defensive or dwells too deeply on a point that is not communicating anything of interest to me. I try and gently move the conversation along, sometimes pointing out my objectives for the interview and the time left. There is a risk of experience roulette here but I find just letting candidates talk about what they think is important is often very illuminating.

Finally, and I try and allow enough time for this section, I like to do something practical. Depending on the role this can be a white board based design exercises or a pair programming experience around a laptop or what ever seems appropriate. I have prepared a set of canned exercises which I have refined over many executions.

If interviewing a candidate for the consultancy I work for then usually they will claim to have Java, Test Driven Development, Eclipse and similar sorts of skills. When they do this I feel empowered to sit them down in front of a laptop and have them work through some artificially simple problem. Often I find that if I type and they tell me what to do they do better and get more out of the exercises. It can be a nasty shock to try and use somebody else's development environment with only a couple of minutes preparation time. Role playing a pair programming scenario can be a way to reduce this shock. Often the exercises I set have tests included. Sometimes those tests fail and this gives a clue to one of the possible solutions. I should stress I try and use the exercise as a vehicle for the candidate to demonstrate they way they think and approach the problem rather than expecting them to finish the code. My favourite exercise is based around the Money pattern and allows a candidate to demonstrate good O-O design, TDD and hands on problem solving. I have used it maybe fifteen times. Only one person has actually got the code working and ticked all the possible boxes. Several people have demonstrated excellent skills without getting close to any sort of completion and gone on to be excellent team members. A few people have been thrown completely and failed to make anything of the exercise. This is a shame for both of us as I then have no positive data to base my decision on and they fail to progress. Occasionally it exposes a complete bullshit artist who has blagged their way through to this interview stage. This happened on one memorable occasion where I was doing the third and final interview for a permanent hire. The guy had got through an HR screening interview and impressed a fellow consultant. I had found him hard to pin down in the first sections of the interview but had been generally impressed with his broad experience. We started the practical and it became very clear that he did not have a clue about the tools that he claimed to be expert in and use on a daily basis. After a few minutes fumbling he refused to continue. I terminated the interview and saved myself an hour of my life.

Very often I interview for senior architect roles on behalf of clients where candidates do not have the hands on coding experience and will not be expected to 'get their hands dirty'. In these situations I have a couple of role play scenarios where I describe an ambiguous use case, sketch out some boxes that represent consumers and providers of services and leave a big white space in the middle. I invite the candidate to fill the white space with their solution. I pretend to be a technically vague marketing representative or a shoot from the hip coder (sometimes we interview in pairs and we take one role each). I ask them to fill in the space and inject requirements if I am the marketeer. If I am playing the coder then I try to ask all the questions a coder would ask if they wanted to realise the design the architect is describing. If the candidate is missing something or sometimes if they appear to be having too easy a time of it, I suggest incorrect or over elaborate solutions and wait to be shot down. I keep on going back to the original requirements to make sure they have been fulfilled.

Getting the scenario right for these exercises is tricky and I have to admit to significantly refining the problems over time. Getting the balance between a problem that consumes too much time and a problem that is over simplistic is a challenge. The best problems I have are based on real project experiences where I can add elements to them in order to increase the complexity if the candidate is doing well.

I have found that you can have a run of candidates who make a real hash of these practical problems. You begin to doubt the validity of the exercise and start to reassess candidates you at first dismissed. In my experience usually the run of dross is followed by a few stars who re-affirm your faith in our profession and the exercise. Whether this is just luck of the draw or reflects the HR or agency reacting correctly to feedback on candidates I do not know (it should be the later but I suspect the former in many cases).

I find the practical exercises useful and actually enjoy watching people solve the puzzles, especially if they do so with panache. At the very least doing a practical exercise gives you concrete evidence to support your decision if you are called to justify it.

These experiences are mine and may not work well for everybody. I have found them useful and have consistently hired good people. I find that this approach delivers much better results than quizzes of programming language constructs or ad hoc conversations around project war stories. Then again, maybe I am still post rationalising and have simply constructed a process that favours people who appeal to me...
Tags :

The release starts here

I am keeping a diary of the main events during a software release as they happen.

Right now, we are moving (in the client's methodology) from feasibility to detailed design.

For a few weeks now one release has been almost finished and another almost started. For the first time in this cycle I have been able to devote most of my work time to the getting the new release underway. There are several of us working on the new software. Some familiar friends and some new faces. There has been lots of requirements capture by the marketing and analysis teams. There has been a significant amount of thinking during a technical feasibility phase. We have had two workshops with a mobile device vendor who will be building one part of the solution.

The project has been picking up momentum and we need to get started and get started in the right way. It would be very easy to let the waterfall culture kick in at this point and the end of feasibility be swiftly followed by a long detailed design phase. I don't what this to happen. I want this to be the best release we have done to date and I want to deliver it using the best practices at my disposal.

I want to get to a product backlog for this release as soon as possible. I do not want to try and produce a detailed construction plan which requires precise estimates for every deliverable before we start.

We need to supply designs which show our partners and internal clients the responsibilities of the various components which we are in the process of specifying.

While feasibility is coming to conclusion I have started a separate activity with the three other designer / developers on the team. We have this week to produce a set of use cases describing what it is we are delivering and what we expect our partners to deliver. We also are going to produce sequence diagrams showing the interactions between the platforms and components. Hopefully these two activities will drive the definition of the responsibilities, give our solution architect confidence that we are proposing is fit for purpose and enable us to create a prioritiesed backlog.

We spent most of today round the white board elaborating the use cases. Some of the team members have spent a lot of time in feasibility and found this exercise frustrating as they feel they already explored these areas. Others who are newer to the project find it useful. I think it is essential to get thee use cases understood at a high level before continuing. I  think the feasibility stage has produced a mix of very abstract analysis and very low level designs. I do not think he have a holistic view of the solution we are building at any level.