Post Agile - Beyond Best Practice and towards Professionalism

The Agile community has been going through growing pains of late. InfoQ have been tracking this in a number of good articles referencing blogs posts by various members of the Agile Community.

I'm not going to reference all the blogs, but I think this introspection is a good thing, and I sense the birth of something new as a consequence. Bob Martin describes the ethics and the ethos that underlies this new thing in this presentation. He talks about a consensus arising around favoured software development practices, and he raises the notion of an accepted standard of Craftsmanship, which when tied with the ethical issue of actually delivering to our customers what they payed for, raises the possibility of software development actually becoming a profession. The idea of craftsmanship is further explored in this blog. I like the idea expressed here of "doing it right" rather then merely "getting it done". Andrew Walker talks to the ethical issue by making the point that we can't really in any honesty call ourselves a profession whilst the failure of software projects is still endemic costing our customers millions for no or little return.

Jim Shore has been busy blogging about the future of Agile also, the most infamous is this post which shines light on the problems that arise when people adopt Agile Management practices like Scrum, but choose to skip the more difficult challenge of adopting sound technical practices. The post by Jim I find the most revealing though is this one on Kanban. The reason why I find this post interesting is that it raises the question of what is "Best Practice".

Jim tries to explain why people are choosing to eschew iterations, calling planning meetings waste, and are choosing to shrink their backlog, calling it excess "inventory". You will recognise "kanban", "waste" and "inventory" as terms bored from Lean Manufacturing. Lean ideas are all the fashion at the moment it seems :). Besides the fact that these ideas are being applied out of context, labelling them "Kanban" (which interestingly is misappropriated since the term already has a specific meaning), reveals our desire to label and promote "Best Practice". In the comments Steve Freeman makes this point, and says that after speaking to David Anderson, one of the original people who invented what has become known as "Kanban", he found out that the reason why he did "Kanban" had nothing to do with eliminating waste, but rather it was a pragmatic response, to an incumbent waterfall culture that didn't allow for iterations. David himself later comments and confirms that Steves summary is accurate.

So one persons chosen practice in a given context, is marketed as best practice for a whole industry. Some in the Agile community have been clear about the need for self organisation and doing what works for you, but historically process improvement has always been viewed in the western world as the adoption and the enforcement of "best practices", and old habits die hard. As a consequence, selling "Best Practice" is now an established cultural phenomena in itself. The Agile Community as been as guilty of (mis-)selling bottled snake oil as all the other process improvement fads that have come and gone before it, despite the small print in the Agile Manifesto that speaks to the contrary. This opportunistic selling perhaps explains why the Agile banner lacks credibility in the eyes of many. They've seen it all before. Ethical issues aside, our culture of promoting "Best Practice" is very naive. Process improvement for an activity as complex, difficult and diverse as producing software, cannot readily be bottled and sold; which probably explains why we still aren't a true profession today.

A more recent article on InfoQ refers to a blog post that makes the point that Agile and Lean aren't the same thing. Now this is something that should be self-evident to anyone who as taken the time to read the work of Taiichi Ohno on the Toyota Production System for themselves, and shouldn't need explaining. Even so, I think that recognition that we cannot in all honesty promote Agile, Lean, Kanban or anything else as "Best Practice", devoid of context is a big step forward. The emerging consensus is that what really matters is developing competence in a useful toolbox of favoured practices, leading to a high standard of craftsmanship. This is not news, but it is nice to see this common sense point of view finally getting the top billing it deserves. It also perhaps signals that as an industry we are maturing, and now able to face the difficult questions that arise when you ask what does it take to be reliably successful at producing software. This fundamental question has hung in the air, ever since Fred Brooks first raised it. I'm tempted to go further and suggest that this change in the mood music may also be signaling the embryonic beginnings of professionalism. Or perhaps that would be going too far :)

What are your thoughts?