Popularity and Success are not the same thing

After my last post, I went back and read through Giles blog post again. I missed his main point, and I dismissed what he had to say far too quickly. This video of DHH and Tim ferri, drove the point home for me and is well worth watching. This talk didn't prove as half as successful at Rails Conf, as Bob Martins.

Whilst I still think that Giles point is tangential to what Uncle Bob had to say, I think it is a profound one. That is, that any proclamation that requires free thought is likely to be unpopular. This lack of popularity doesn't equate to failure however, it just means that such ideas are more likely to benefit the few, rather then the many.

Uncles Bob's use of Smalltalk as the epitome of failure, rather cleverly raises the spector of fear, and as we all know fear is the great motivator of the masses. So if as a Ruby programmer, you want to be in a job for the foreseeable future, then make sure and do TDD. It's kind of like a Mom using threats to make sure that the kids eat their greens.

Whilst DHH and Tim and Paul Graham, are fundamentally right, they don't address the psychic of the average mainstream programmer, which is mostly dominated by the fear of not finding a job. So in a way this brings me back to were I started. Sometimes being right isn't enough. Sometimes you need to be effective too, and whilst fear is the worst of motivators, Uncle Bob is addressing the world as it appears to the vast populous, which is an effective tactic.

The world that is Giles reality is a rarefied place and the few people and languages that live there are likely to remain obscure for some time to come yet. The last interesting point is that it appears that DHH has out grown the community he created. This is hardly surprising. Like I say followers don't think for themselves, they rely on others, and as Rails becomes ever more popular, the community will become dominated by followers, just like what happened with Java.

For someone like David, this influx of non-thinkers must be nauseating.

Paul.

What Killed Smalltalk: Sometimes Balls can be Useful

Uncle Bob, has been his usual charismatic and entertaining self in his keynote speech this year at Rails Conf. He presented a dire warning for the Ruby Community: "What Killed Smalltalk Could Kill Ruby, too". So what is this beast of death? Well Uncle Bob blames the ease at which programmers could create a mess in Smalltalk as the thing that killed off the language, quoting no other then Ward Cunningham to back up his argument. Interestingly he makes only brief mention of another VM based OO language with garbage collection that appeared in 1995, and was market mercilessly and given away for free, whilst Smalltalk vendors were still charging £3K plus per pop for Smalltalk licenses (yes that's over £3000 per seat). Hmm...

Anyway as you would expect, the Smalltalk community is up in arms. James Robertson has this post. And Giles Bowkett makes this response entitled "What Killed Smalltalk: My Balls". Giles response, demonstrates IMHO complete ambivalence to what is actually happening in mainstream programming today. Whilst IMHO Uncle Bob clearly has his finger on the pulse. I have never been a full blown member of the Smalltalk community. I've always been one of those people on the side lines, using Smalltalk for fun and hoping to get a Job that would allow me to write Smalltalk commercially, but never being fortunate enough to do so. I think that gives me a more balanced perspective. I am neither a curly bracket or square bracket zealot, having lived most of my programming career with both. Whilst I've always admired the Smalltalk language, I have noticed that the Smalltalk community has a tendency to be insular and inward looking (with no sign of those balls Giles speaks of). Anyway, here is my comment in response to James Robertsons blog post. I thought I'd post it here (with some minor edits), since I know that very few people take the time to look at Smalltalk blogs:

There are two audiences here. One audience are the 20 somethings who were merely a lusty glint in their fathers eye when the clash of C++ and Smalltalk was taking place in the 1980's. The other audience are the people who where actually there.

For the 20 somethings, I think there is a lot in this presentation to get excited about. First of all it makes the point that good ideas can fade. It also makes the point that great languages provide the programmer with great power. With this power comes responsibility. As developers we have a choice. We can allow ourselves to be dumbed down and given dull tools so we won't hurt ourselves, or we can become responsible and disciplined to the extent that we can be trusted to use sharp knives. This message is an important one. The other great message, Smalltalk, which is now being discussed amongst a new young audience. If any of these young kids are half as curious as I was at their age, then we should expect downloads of Squeak to spike this weekend...

As for the other audience. The people that were actually there. It is easy to dismiss this presentation as a pile of BS. Well it mostly is, but to engage with it only on a factual level is to miss the point. This presentation is not for us. Bob Martin is not a Smalltalk programmer and never was. I was on one of his OOD courses in the 1990's and he asked who in here as used Smalltalk? I was the only person to sheepishly raised my hand. He then went one to castigate Smalltalk, saying that you couldn't use it for anything serious because it would lead to a mess once you got to more then 10K lines of code. Back in them days Bob was a curly bracket language zealot.

I'm glad that James has admitted that that back in the 90's the Smalltalk community suffered from arrogance too. I read Giles response and it sounded more like a tirade. The mainstream developer community at the moment is in flux. They trusted in vendors only to be let down. Now they are looking for new leaders. Most of them are followers, if they weren't they would have self educated themselves long ago, and Uncle Bob wouldn't be able to get away with his historical inaccuracies. The Smalltalk community are in a great position to provide leadership. The Agile movement came from Smalltalk, TDD from Smalltalk, Ruby from Smalltalk...

So why isn't the Smalltalk community doing so? Why wasn't a Smalltalk programmer giving this keynote instead? Where is Alan Kay when you need him or Dan Ingalls? Is it because we've all got a reputation of being grumpy old men? It is up to the Smalltalk community to make themselves relevant to these new young kids. Bob is doing his best to tell a story second hand. The Smalltalk community should be telling this story itself. Looking in the mirror at yourself and your own short comings, is much harder then pointing fingers at others.

Paul.

The road to mastery - Practice, Practice, then Practice some more...

The craftsmanship movement has hit the ground running. First a conference in London organised by Jason Gorman. Looking through Jason's Blog, and from the video, it looks like the whole Xtc London crowd were in attendance. Jason is an established member of the Xtc crowd, so I suppose it shouldn't have been surprising. Familiar faces like Ivan Moore and Nat Pryce. It brings me back to my early XP days, when I too attended such conferences and the excitement and delight of it all, not to mention the friendships and the insights. What fond memories.

So after going full circle, the "Agile" movement are now back were they started: Writing Code! Or should I call it the post Agile movement? Or better still lets just call them the folks who like to program, get stuff done, and take a pride in doing it well.

Micah Martin was there, and presented his Langston's Ant Ruby Kata. Now Micah's dad, Uncle bob is the guy responsible for teaching me the fundamentals of OO design. So anyone schooled by Uncle Bob, will probably have something to say that would prick my interest. Watch the video. Its a real expose of the Ruby language and the mechanics of programming in general.

Micah borrowed the word Kata from marshal arts. A Kata is an almost ritualised performance of a set of martial art skills, with the desired aim of re-enforcing those skills through practice. So Kata's are meant to be performed again and again, until the practitioner becomes use to the moves involved. Its a wonderful idea, and is synergistic with Ron Jefferies emphasis on practice and learning through doing. Ron's bowling game example could be considered yet another Kata. Hopefully Micah will develop this idea further and I'm looking forward to a whole catalogue of Kata's that the aspiring programmer can learn and practice. Katas to me seem like a great tool when going through the Shu (hold) phase of learning.

Focusing on Micahs Ruby Kata, what was interesting to me was his interpretation of BDD. I have been doing quite a lot of BDD lately, and I have found the style of BDD I've been learning a tad verbose. Micahs BDD style using Rspec is very similar to my own TDD style, and a lot more appealing as a consequence. Given that Uncle Bob learned TDD from Kent Beck, and I learned TDD from a bunch of XPers (Ivan Moore, Steve Freeman, etc) who presumably picked it up from Kent Beck too, its clear to me that Micah and I share a common BDD/TDD lineage. This raises an interesting point about style.

Micahs software shop is called 8th Light, and they conduct an apprenticeship program. Now I always thought of an apprenticeship as something for novices, but 8th Light take a different slant on the apprenticeship idea. Rather then an apprenticeship being just a means of teaching someone the basics, 8th Light also sees an apprenticeship as a means of teaching someone the 8th Light Style. This is another great idea. From my waterfall days, I have known that great programmers will get the job done, using what ever approach is familiar to them, and there is no one right way, No Best Practice, No one style.

Corey Haines explores the idea of style with Paul Pagel of 8th Light in this video. They also talk a bit about the recent Craftsmanship Manifesto and what motivated it. They suggest that programmers should learn more than one style during their Journeyman phase. Again this is something that is synergistic with my experience. I remember working with Ivan Moore and Erik Deorenenberg on the same project. Both great programmers, but each with a different style. Whilst I found Ivans "Smalltalk/Python" biased style more appealing, Eriks prolific nature and practical bias was extremely informative too, and I vowed to get my keyboard skills up to Eriks levels (which is still something on my to do list :)).

So the Journeyman must be prepared to break with his master and explore alternative styles. This seems to me like the Ha (break) phase of learning. At 8th Light they are hoping to facilitate this by organising exchange programs with other software shops.

So there we have it. People walking the talk, and software as a profession actually being practiced. It is interesting contrasting the Craftsmanship movement with the Kanban movement, both recent off shoots of Agile. One group (Kanban) are busy telling others how to manage software development, and are selling their services as Consultants, whilst the other group (Craftsman) are busy writing software themselves for real customers and sharing their practices and insights for free with anyone interested in learning.

I saw this on Jasons blog:

Q: How do you get to Carnegie Hall?

A. (craftsman): Practice, practice, practice.

A. (consultant): Pay a respected market research company shedloads of money to write a report that concludes you're already at Carnegie Hall, and then make a killing selling maps and directions (even though you've never actually been there)

I think it says it all. All's left for me to do now is to get practicing myself. I'll think I'll start with the Langston's Ant Kata in Newspeak. Now that should be fun!

Post Agile - Beyond Best Practice and towards Professionalism

The Agile community has been going through growing pains of late. InfoQ have been tracking this in a number of good articles referencing blogs posts by various members of the Agile Community.

I'm not going to reference all the blogs, but I think this introspection is a good thing, and I sense the birth of something new as a consequence. Bob Martin describes the ethics and the ethos that underlies this new thing in this presentation. He talks about a consensus arising around favoured software development practices, and he raises the notion of an accepted standard of Craftsmanship, which when tied with the ethical issue of actually delivering to our customers what they payed for, raises the possibility of software development actually becoming a profession. The idea of craftsmanship is further explored in this blog. I like the idea expressed here of "doing it right" rather then merely "getting it done". Andrew Walker talks to the ethical issue by making the point that we can't really in any honesty call ourselves a profession whilst the failure of software projects is still endemic costing our customers millions for no or little return.

Jim Shore has been busy blogging about the future of Agile also, the most infamous is this post which shines light on the problems that arise when people adopt Agile Management practices like Scrum, but choose to skip the more difficult challenge of adopting sound technical practices. The post by Jim I find the most revealing though is this one on Kanban. The reason why I find this post interesting is that it raises the question of what is "Best Practice".

Jim tries to explain why people are choosing to eschew iterations, calling planning meetings waste, and are choosing to shrink their backlog, calling it excess "inventory". You will recognise "kanban", "waste" and "inventory" as terms bored from Lean Manufacturing. Lean ideas are all the fashion at the moment it seems :). Besides the fact that these ideas are being applied out of context, labelling them "Kanban" (which interestingly is misappropriated since the term already has a specific meaning), reveals our desire to label and promote "Best Practice". In the comments Steve Freeman makes this point, and says that after speaking to David Anderson, one of the original people who invented what has become known as "Kanban", he found out that the reason why he did "Kanban" had nothing to do with eliminating waste, but rather it was a pragmatic response, to an incumbent waterfall culture that didn't allow for iterations. David himself later comments and confirms that Steves summary is accurate.

So one persons chosen practice in a given context, is marketed as best practice for a whole industry. Some in the Agile community have been clear about the need for self organisation and doing what works for you, but historically process improvement has always been viewed in the western world as the adoption and the enforcement of "best practices", and old habits die hard. As a consequence, selling "Best Practice" is now an established cultural phenomena in itself. The Agile Community as been as guilty of (mis-)selling bottled snake oil as all the other process improvement fads that have come and gone before it, despite the small print in the Agile Manifesto that speaks to the contrary. This opportunistic selling perhaps explains why the Agile banner lacks credibility in the eyes of many. They've seen it all before. Ethical issues aside, our culture of promoting "Best Practice" is very naive. Process improvement for an activity as complex, difficult and diverse as producing software, cannot readily be bottled and sold; which probably explains why we still aren't a true profession today.

A more recent article on InfoQ refers to a blog post that makes the point that Agile and Lean aren't the same thing. Now this is something that should be self-evident to anyone who as taken the time to read the work of Taiichi Ohno on the Toyota Production System for themselves, and shouldn't need explaining. Even so, I think that recognition that we cannot in all honesty promote Agile, Lean, Kanban or anything else as "Best Practice", devoid of context is a big step forward. The emerging consensus is that what really matters is developing competence in a useful toolbox of favoured practices, leading to a high standard of craftsmanship. This is not news, but it is nice to see this common sense point of view finally getting the top billing it deserves. It also perhaps signals that as an industry we are maturing, and now able to face the difficult questions that arise when you ask what does it take to be reliably successful at producing software. This fundamental question has hung in the air, ever since Fred Brooks first raised it. I'm tempted to go further and suggest that this change in the mood music may also be signaling the embryonic beginnings of professionalism. Or perhaps that would be going too far :)

What are your thoughts?

I don't know

If you read something new that is contrary to your current understanding you are faced with several possibilities:

1. The new idea stems from a new context which you haven't experienced. In this new context the old idea that you are familiar with no longer applies.
2. The new idea hasn't been rigorously tested, hence the need to seek corroborating testimonies.
3. The new idea has been corroborated, but the corroborators are involved in a conspiracy of lies in an attempt to deceive.

Now before you jump to option 3, you would rigorously explore options 1 and 2 first, for the simple reason that you may learn something new. Well at least you would think so.

In my experience this is seldom the case with programmers. In most instances when faced with a new idea that challenges current "best practice", I see programmers jumping to option 3.

This blog post and the comments that follow is a clear example.

So why is this? I think it has to do with how programmers process information. As left brained people, they build ever more elaborate models of understanding of what is "best". New information to the contrary challenges that model and rather then see the model fall like a deck of cards there is a strong impulse to reject the new idea.

I have blogged about this before:

Not knowing shouldn't be threatening, a state that should be extinguished quickly, with ever more elaborate models. No, not knowing is just part of the rich fabric of life. In fact there is power in the unknown. In acknowledging that there isn't a simple formula, order can be found in chaos.

Finding order in chaos is one of the most basic tenets of Agile Development. It is this philosophy that eschews any attempt to be deterministic and know in advance what is best. Instead Agile development embraces an empirical approach of trail and error, suck it and see, where not knowing and discovery is a natural part of the process.

This philosophy seems to be a hard sell. Everyone wants a fixed model, kind of like painting by numbers, which you can buy off the shelf and will solve your problem each and every time. In an attempt to fix the model scepticism is deemed as negative. The vice of non-believers who refuse to "get on message" about what is "best".

Keith Braithwaite takes a contrary point of view. Rather then seeing scepticism as always bad, he believes that healthy scepticism is a good thing, a sign that you are actually thinking and processing new ideas on their own merits. I agree. The problem with 'best practice" and a "fixed model" is that for any non-trivial process "best" is always conditional. So in different contexts what is best will differ. So what is right for one team may not be right for you. So there is no ready formula, and you need to discover what is best for each specific context, which means admitting that perhaps you don't know. This is why you can't master Agile development by just reading a book, and most Agile advocates suggest that you commandeer the help of an experienced coach who has experienced several different contexts to help guide you in discovering what is right for you.

Keith makes his point very well in an interview on InfoQ. At around 24 minutes and 49 seconds in to the interview, the interviewer asks "How do you choose the right Agile management tool if you are new to Agile?". To which Keith answers: "I don't know". From this admission you can actually see a process of discovery ensue. Keith then goes on to say that in lieu of knowing he would suggest using the simplest thing that works, an Excel spreadsheet. After using the spreadsheet for a while and writing macros to automate the calculations you needed you would learn what it is your process needed from an Agile project management tool and would then be able to decide which tool if any on the market best suited you.

It is a magical moment, with Keith shacking his finger in excitement as he discovers the answer to a question which he previously didn't have an answer to. The interesting thing for me is that the discovery started off with the statement "I don't know".

How many of us would respond to our boss when asked a question like "What tool should we use?" with "I don't know"? Very few in my experience. What is more likely is a long diatribe based on our prior context and what the book says or what we've read in reviews. As a consequence the opportunity for discovery is lost.

Take the time to watch Keith's presentation and try saying "I don't know" more often. You may find that it accelerates your learning tremendously.

Further Thoughts:

Alan Kay has another explanation for why people with prior context find it difficult to accept new ideas:

"Kay blames this lack of innovation on the fact that most adults employ instrumental reasoning to evaluate and apply new ideas. This means that adults have difficulty evaluating new ideas because they're carrying too many existing goals, and too much context to be able to see the full potential of new ideas. One way to overcome this problem is to work with children. Children won't know that something is difficult or that hundreds of papers have been published explaining the intricacies of some particular problem. They are more likely to experiment where adults would have given up, and to ask, "So what else can I do with this?"

I think this is similar to my "preserve the existing model" idea, where people reject new ideas that don't fit their established understanding. The same issue is also seen in science where breakthroughs happen during moments when some one is brave enough to go against the established orthodoxy. It is interesting that Alan Kay believes that children don't suffer from this. This fits with the proverb "you can't teach an old dog new tricks". So one solution is to try and maintain a child like curiosity and always ask the question Why? in the annoying way children do. I do this and it works well. If you ask why 5 times, it is usually enough to reveal something new.