I don't know

If you read something new that is contrary to your current understanding you are faced with several possibilities:

1. The new idea stems from a new context which you haven't experienced. In this new context the old idea that you are familiar with no longer applies.
2. The new idea hasn't been rigorously tested, hence the need to seek corroborating testimonies.
3. The new idea has been corroborated, but the corroborators are involved in a conspiracy of lies in an attempt to deceive.

Now before you jump to option 3, you would rigorously explore options 1 and 2 first, for the simple reason that you may learn something new. Well at least you would think so.

In my experience this is seldom the case with programmers. In most instances when faced with a new idea that challenges current "best practice", I see programmers jumping to option 3.

This blog post and the comments that follow is a clear example.

So why is this? I think it has to do with how programmers process information. As left brained people, they build ever more elaborate models of understanding of what is "best". New information to the contrary challenges that model and rather then see the model fall like a deck of cards there is a strong impulse to reject the new idea.

I have blogged about this before:

Not knowing shouldn't be threatening, a state that should be extinguished quickly, with ever more elaborate models. No, not knowing is just part of the rich fabric of life. In fact there is power in the unknown. In acknowledging that there isn't a simple formula, order can be found in chaos.

Finding order in chaos is one of the most basic tenets of Agile Development. It is this philosophy that eschews any attempt to be deterministic and know in advance what is best. Instead Agile development embraces an empirical approach of trail and error, suck it and see, where not knowing and discovery is a natural part of the process.

This philosophy seems to be a hard sell. Everyone wants a fixed model, kind of like painting by numbers, which you can buy off the shelf and will solve your problem each and every time. In an attempt to fix the model scepticism is deemed as negative. The vice of non-believers who refuse to "get on message" about what is "best".

Keith Braithwaite takes a contrary point of view. Rather then seeing scepticism as always bad, he believes that healthy scepticism is a good thing, a sign that you are actually thinking and processing new ideas on their own merits. I agree. The problem with 'best practice" and a "fixed model" is that for any non-trivial process "best" is always conditional. So in different contexts what is best will differ. So what is right for one team may not be right for you. So there is no ready formula, and you need to discover what is best for each specific context, which means admitting that perhaps you don't know. This is why you can't master Agile development by just reading a book, and most Agile advocates suggest that you commandeer the help of an experienced coach who has experienced several different contexts to help guide you in discovering what is right for you.

Keith makes his point very well in an interview on InfoQ. At around 24 minutes and 49 seconds in to the interview, the interviewer asks "How do you choose the right Agile management tool if you are new to Agile?". To which Keith answers: "I don't know". From this admission you can actually see a process of discovery ensue. Keith then goes on to say that in lieu of knowing he would suggest using the simplest thing that works, an Excel spreadsheet. After using the spreadsheet for a while and writing macros to automate the calculations you needed you would learn what it is your process needed from an Agile project management tool and would then be able to decide which tool if any on the market best suited you.

It is a magical moment, with Keith shacking his finger in excitement as he discovers the answer to a question which he previously didn't have an answer to. The interesting thing for me is that the discovery started off with the statement "I don't know".

How many of us would respond to our boss when asked a question like "What tool should we use?" with "I don't know"? Very few in my experience. What is more likely is a long diatribe based on our prior context and what the book says or what we've read in reviews. As a consequence the opportunity for discovery is lost.

Take the time to watch Keith's presentation and try saying "I don't know" more often. You may find that it accelerates your learning tremendously.

Further Thoughts:

Alan Kay has another explanation for why people with prior context find it difficult to accept new ideas:

"Kay blames this lack of innovation on the fact that most adults employ instrumental reasoning to evaluate and apply new ideas. This means that adults have difficulty evaluating new ideas because they're carrying too many existing goals, and too much context to be able to see the full potential of new ideas. One way to overcome this problem is to work with children. Children won't know that something is difficult or that hundreds of papers have been published explaining the intricacies of some particular problem. They are more likely to experiment where adults would have given up, and to ask, "So what else can I do with this?"

I think this is similar to my "preserve the existing model" idea, where people reject new ideas that don't fit their established understanding. The same issue is also seen in science where breakthroughs happen during moments when some one is brave enough to go against the established orthodoxy. It is interesting that Alan Kay believes that children don't suffer from this. This fits with the proverb "you can't teach an old dog new tricks". So one solution is to try and maintain a child like curiosity and always ask the question Why? in the annoying way children do. I do this and it works well. If you ask why 5 times, it is usually enough to reveal something new.