Order and Chaos and the Power/Threat of the unknown

Right as you can tell by the title this blog is about a number of ideas that I feel are related. I'll try and keep it short. What got me blogging was a post on the XP discussion group on yahoo. The discussion was about the newest XP practice "Slack" as mention in XPE2 (XP Explained, 2nd Edition). The question in most of the posts was "what is it?"

The answer was that no one really knew. The one person who should of known (Kent Beck) contributed a link to a forum discussing his fore mentioned book (XPE2), but offered no detail explanation of Slack himself.

I pointed out what seemed obvious to me, which was slack (leaving space in the plan) will mean different things to different teams at different times. So trying to elevate this behavior to a defined "Practice" was a futile exercise. Unfortunately only one person explicitly agreed with me. But I'm sure a few others did too, but chose not to be as vocal.


So why is it that we feel the need to define the indefinable? We are always modeling the world and trying to put it in a box. I think for many of us, that is just the way our brain works (left sided thinkers), Mathematicians, scientists and programmers.

Creative people (poets, artists, writers etc), seem to suffer from this need a lot less. To them something that is inherently indefinable is a source of fascination and interest. They tend not to find such things threatening at all. Concepts such as beauty and elegance resist precise definition, yet many communities quite happily share a common understanding of what these things are. This understanding becomes what is known as culture.


A book that had a profound effect on me was "Language in Thought and Action" by Awakawa. The book describes how thought is intimately linked to language, and how our choice of language can limit our thinking. In the book there is the idea that the label is not the thing. The map is not the territory. We use 'labels' in language all the time as synonyms for ideas, concepts, groups, activities etc. The synonym itself takes on it's own meaning and inference yet any instance of a thing that we choose to refer to using a synonym may have it's own unique characteristics which do not neatly fit into the meaning inferred through the synonym .

So the key is to know when your are using a label, and that the real world is a lot more complex and diverse. An example of this that comes to mind and is related to software is the CMM. The CMM is an attempt to model software development organisations. It identifys Key process Areas (KPA's) and their relationships with each other. Each KPA is assigned to a role that must exhibit concrete abilities.

As part of my training as a certified CMM assessor, I was exposed to a computer model of the CMM. It was great you could see the effect of changing abilities within a given KPA on other KPA's and on the process as a whole. Logically the model made a lot of sense. The big missing gap though was that people and organisations often defy logic.

So within the CMM a group of demonstrable KPA's represented an organisation who's output was repeatable on a per team basis (Level 2), an addtional group of KPA's defined an organisation whose output was repeatable and consistent across the organisation (Level 3). The next level of maturity, level4 , contained KPA's that demonstrated that an organisation could quantitatively measure it's performance as a pre-requisite to quantitative process improvement.

Unfortunately, the moment you expose this simplified model to real world situations it immediately begins to fall apart. For instance is it not possible that an organisation may have a single team that have demonstrably delivered repeatedly, using quantitative techniques. In such a scenario, the set of abilities demonsrated may cut across maturity levels, they may even cut through KPA's, leaving the CMM model in taters.

In a sense XP has fallen into the same trap as the CMM. When first introduced to XP I was told that you needed to do all the XP Practices together. I was told that they are all related and inter-dependent on each other. So after 4 years real world experience, XP has been shown to have gaps. Despite doing all the practices, there are still issues and problems that XP doesn't address. Is this surprising? So what is the solution? Well for Kent Beck it was add one more XP value (respect) and an additional practice (Slack).

The model is flawed, so just add to the model. There is another way. That is to remember that the model is just that a model. The real world is a lot more complex and diverse. The label is not the thing. The map is not the territory.

I could blog (rant) some more. But in summary, as developers we need to get better at using the other side of our brain. Not knowing shouldn't be threatening, a state that should be extinguished quickly, with ever more elaborate models. No, not knowing is just part of the rich fabric of life. In fact there is power in the unknown. In acknowledging that there isn't a simple formula, order can be found in chaos. Examples of this exist in philosophy, especially eastern philosophy. So Zen sayings like "Knowing without knowing" really do have meaning. Higher order thinking can help us to gain understanding in an inherently complex and chaotic world. All we need is the courage to embrace the power of the unknown.

XPDAY5 - Getting Customers to Know Themselves

In the same session on "Getting to know your customer" mentioned in my previous post, Steve and Andy went on to describe Innovation Games. The goal of these games is to get your customer to think about their problems in a different way, and to come up with new innovative ideas on how to tackle them.

Interestingly, just talking to customers about what they want isn't always adequate. Often customers do not know what they want and end up telling you what they think you want to hear. I think this could be related to the three basic assumptions challenged by the Cynefin sense making framework (see my previous post). If our customers think that we expect them to be rational, intent-driven and ordered then they will answer our questions in kind.

By game play people are let of the hook. They do not need to pretend to be rational. They are free to get in touch with themselves and the true nature of their problem, after all it's just a game.

One of the games is called Product Box. Customers are encouraged to decorate a box containing the "product". The decoration should highlight the most exciting and sellabe features. The idea is to make the product appealing. Another game is called Speed Boat. Here customers are encouraged to draw a spead boat which is tied to things that are holding it back, things that are stopping them speeding ahead in their work.

These games are similar to the domestic probes used by Bill Gaver and fit in well with the Cynefin probe-sense-respond appoach.

Here is a link to more Innovation Games: http://www.enthiosys.com/innovation.php

The combination of Cynefin sense making, Innovation Games and Bill Gaver's Ludic Technologies really made XPDay for me. I'll never look at innovation and product ideas in the same way again.

XPDAY5 - Knowing your Customer

One of the sessions at XPDay that I attended was about knowing your customer. Steve Freeman and Andy Pols introduced use to the Cynefin sense making Framework. This framework challenges three of the basic assumptions we generally make when trying to understand our customers and their organisations. The three assumptions are:

• The assumption of rational choice - People will do what is rational.
• The assumption of intent - Everything that happens is done intentionally.
• The assumption of order - Organisations are well ordered, rule based entities.

It turns out that if you remove these assumptions, then most organisations turn out to be highly complex and chaotic (sounds familiar?). The framework goes on to suggest two approaches to effecting change in such organisations:

• probe-sense-respond - for complex organisations and,
• act-sense-respond - for chaotic ones.
  
  
  This is very different to the sense-categorize-respond approach that has been traditionally taken (for example Business Process Engineering). They point out that the traditional approach is best suited to ordered organisations.
  
  Here is a link to their paper:
  
  http://www.research.ibm.com/journal/sj/423/kurtz.pdf
  
  A bit of a heavy read. Thankfully the approach they recommend for complex/chaotic organisations, namely probe/act-sense-respond is already widely known as suck-it-and-see :^).

XPDAY5 - Anti-Intellectualism and Dead Fish

Tim Listers keynote at XPDay this year was an impassioned plea for people to wake up and use their brains. An example of "brain-less" practice that he described was a team that continued to provide effort estimates to two decimal places even though their estimates where consistently between 100-200% out. Anyone who knows anything about maths (Tim's father) could clearly see the inconsistency between the estimating accuracy and precision.

So how does this type of thing happen? How do supposedly intelligent people keep making such fndamental mistakes? Organisational culture has a lot to do with it, coupled with the inherent difficulty of accurately predicting software development. The stats for software development failures are abysmal. Most projects fail and those that don't are either late or deliver less functionality then promised. Only a small percentage actually succeed.

Tim spoke about anti-intellectualism. People not wanting to know. I've experienced this too. If you propose an intellectually sound approach to tackling a complex question often the response is negative. It is almost as though people do not want to think. Worst still, it is though they are scared that if they did think things through, that they could come up with answers that aren't palatable.

The consequence of this willful ignorance is what Tim Lister refers to as the dead fish. Dead fish projects are projects where everyone knows that they are going to fail, but no one actually comes out and says it. The dead fish stays there smelling and everyone blistfully ignores it. Tim said that you can sense the mood in Dead fish teams, there is an atmosphere, a stench that is perceivable straight away. People know when they've been set up to fail, and it shows in their body language and general lack of enthusiasm.

Tim did a good job at identifying the problem, but didn't offer solutions. My experience is that telling people that there is a dead fish doesn't always help. Usually by that time, many people feel implicated and vunerable. Hearing the problem described so eloquently though was powerful. It has definately motivated me to do more about dead fish in the future.

XPDAY 5 - "Gathering" Requirements

For me there was a sort of theme to XPDay 5. The theme was Innovation. How do we know that we're building the write things? Both of the Keynote speakers touched on this subject. It is an area that as concerned me too (see my post on software as an art). As developers we have managed to lull ourselves into the belief that product requirements are just out there and all we need to do is "gather" them. In Tim Listers keynote he showed a slide with a picture of a little girl picking a flower with the caption "Oh what a pretty requirement, let me gather it". It had us all laughing, but it made the point well.

Bill Gaver, the second keynote speaker spoke about his work with Ludic technologies. His interest is computational devices for the home. He believes that computers in the home need not be practical, work oriented devices, but instead can enrich the experience of domestic living in other non-practical ways. So the question is what should we build for the home? Well it turns out that the obvious approach isn't necessarily the best. Just asking people what they want in their homes doesn't always lead to new innovative thinking. To tackle this problem, Gaver and his team have come up with a probe pack which they use to probe the minds of potential users and gain an insight into how they use their homes:

Domestic Probes:




Real interesting stuff. One of the products produced as a result of this type of requirements elucidation is the drift table


To find out more about Ludic Technologies visit the Equator project website.