Thursday, April 03, 2008

Pattern Languages and Painting by Numbers

Steve Yegges post on Noobs has really got me thinking about models and the purpose they serve. I think that models help fill a void in the design space. To begin with all we have is a problem and a blank canvas. A skilled artist will look at his subject and start sketching perhaps at first with a pencil. He will then refine and re-work his sketch before moving on with oils.

To the novice artist the blank canvas must be daunting. I have noticed that novice developers with a poor understanding of OO find OO programming from first principles a daunting task too. They are much more comfortable starting out with a template or a framework. A standard sketch. So if the subject is a web page, then the Struts framework which implements the model-2 web application pattern is a comforting place to start. All the novice then has to do is colour in the details. Just like painting by numbers.

When coaching, I have had an uncomfortable feeling about patterns. Novices are great at learning patterns by rote, not so good at knowing when to apply them. And even when they do apply them correctly, few seldom know how to tailor such patterns to create a novel design themselves. Worst still is where someone else has imposed a pattern and the programmer follows it religiously not noticing that the pattern doesn't quite fit for the problem at hand.

Artist do not paint by numbers. Sure they learn a lot of styles (patterns), but ultimately their creations are there own. So how do they create and come up with something novel when faced with a blank canvas? Well they rely on first principles. They understand perspective and how it works. They understand light and how it works too. They understand their medium: oils, canvas, brushes and strokes. They have a deep understanding of their subject aswell. Bone structure, muscles, skin tone etc. They also have a good eye for detail and can see things in their subject of interest, drawing out important nuances, and giving them prominence in their final piece of work.

Sound technical understanding based on first principles, a catalogue of prior work (patterns) for inspiration, a keen eye for significant details, and a well honed instinct for creativity. These are the quality of a good artist. They are also the qualities of a good programmer too.

I might be over ambitious, but I believe that average programmers can obtain this level of skill with coaching and practice, and model obsession as we have come to know it is not needed and can be replaced with something better. In a series of blogs I hope to strip away standard patterns and get back to first principles in an attempt to show the cost we are enduring by painting by numbers. I also want to show that going back to first principles isn't that daunting and can be both beneficial and a lot of fun.


Andrew said...

Paul, I really like the artist analogy, its high time we had some widespread recognition that the profession is as much about art as it is engineering.

I view such tasks such as setting up continuous builds, source control systems, getting code to compile correctly - as closer to engineering - they are very precise and have to be exact to work. These things are very necessary. In fact without the mix, there would be plenty more problems. That's why I think there are so many issues facing software. The best developers have that mix of skills - they recognize they need to be both artistic and yet very technical/engineering based at the same time - this mix is a very rare skill set.

Painting by numbers - as you put it so well - is a very dangerous practice and those who believe software works this way are fooling only themselves. This learn by rote and arrive at a 'best practice' mentality leads to complacency and the belief that a software developers' job is all about construction, being a 'bricklayer' or an assembly worker - just follow the instructions and everything will be fine - NO. Many managers view project management in the same light - this is what a spec looks like, this is how to run a project. These notions is exactly what Taiichi Ohno said were so wrong. The underlying problem is that people don't get it - instead of following everyone else, all it take is thought and experience. In my opinion, the experience part is really only gained by working in a healthy development environment, with others who 'get it'. But therein lies the problem - a development role is not viewed as sexy by many today - they're too quick to want to be a PM or an Architect, Business Analyst etc, but those in the know, realize its actually the other way around.

With design, I like to use some higher level construct/language such as UML when is makes sense to do so. Its usefulness is not only limited to communicating technical solutions but also for business models. Unfortunately with the latter, business people don't know UML and don't really care to - so its really only good for communication within the development team - but it can still have value to help understand the problem domain at a high level of abstraction. On the technical side, a particularly complex idea (sometimes over-complexity, but it can also be genuine) may be harder to communicate to others in code. I like to see the big picture - I want to know that I am building a cathedral and not cutting stone, and models are one way of communicating the bigger picture.

One of the big clues to all this is right under our noses - we call it software 'development' not software 'production' developing anything is highly artistic in nature, designing a new car for instance. However, even car designers often have to choose from some standard palette of parts to reduce the cost of production. This equates to the component paradigm in software development, which hasn't really taken off - because there are just too many variables.

Paul said...

Hi Andy,

The best developers have that mix of skills - they recognize they need to be both artistic and yet very technical/engineering based at the same time - this mix is a very rare skill set.

The best Artists have this mix too. Painting is a very technical discipline. Perspective is a precise science so is light and how it reflects of different surfaces. Knowing how to work oils and apply brush strokes is a technical discipline. Artist need a lot of technical training just go get the basics right.

What makes great art though is when they go beyond the basics and use their creativity. It is the same with Software. Technically there is no difference between a PocketPC Smartphone and the iPhone. Technically they both have the same features and solve the same problem. The difference is their design. With the iPhone the user interaction is much more natural. This better design stems from the innovation and the creativity of the Engineers at Apple.

Interesting that you mention aiichi Ohno. When Toyota were launching the Lexus, a group of Design Engineers went and lived the high life in California for six months. The idea is that they whould get to learn swhat luxury was all about. This was their way of better getting to grips with their problem: How to produce a car that appealled to what Amercians felt was a luxurious lifestyle.

The Japanese understand that new product development is a mixture of sound engineering and art. In our profession we have managed to loose that truth in an attempt to dumb down and centralise decision making into the hands of the few.

The best people to indentify the right solution are those intimately in touch with the problem. this is normally the (design) development team.

Here is a quote you may find interesting:

We will win and you will lose. You cannot do anything because your failure is an internal disease. Your companies are based on Taylor’s principles. Worse, your heads are Taylorized too. You firmly believe that sound management means executives on the one side and workers on the other, on the one side men who think and on the other side men who only work.”

Konusuke Matsushita - Panasonic

Andrew said...

Actually, I should have said, the ability to mix technical prowess with artistic/creative flair is a very rare talent (not skill set).

So I view great developers as those who can be both right and left brained and have the ability to operate on multiple levels of abstraction - from high level concept (I'm building a cathedral) to low level details (how do I make the perfect cut for this piece of stone).

William said...

Great Analogy!

I agree with almost all you say, but the idea of patterns as static recipes. They can be valuable if used correctly. The principles underneath the patterns are good, don’t le the misuse of the to discard the ideas.

BTW, I’m a musician. I can tell you that art is everywhere. For example, if you want to learn piano, you may pay a teacher, but University will not accept you if you are grown up. Simply because the fingering technique must be learned at early ages, if not you are spoiled. And if you see the teaching scores, you will see numbers over the notes: this for the student to follow the fingering and train the hand. Later, the pianist will simply read the notes and the hands will use the correct fingering.

Analogy, analogy….

Brad Wiederholt said...
This comment has been removed by the author.
Brad Wiederholt said...

just a quick dash of a note here -- I am having a bunch of thoughts here, but am thinking that there are these 'special' individuals in a lot of fields. For example, I used to design computer-based training, at one point for medical students, and I met a lot of folks who could take on a 'big picture' view of a system. In fact, a lot of the med students who were entering the program that came from engineering disciplines.

Back in the days of high school, I thought for sure I was going into medicine (psychology) and stumbled on to computer science by accident. In my positions over the years, I've had to straddle art/design, computers, and psychology. At the core of it, I believe there is some sort of holistic viewpoint that is required to do this.

Can't exactly put my finger on the person we are talking about, but I think if you look as someone who can approach a problem in a holistic or systems-oriented approach -- someone who can fuse many senses and experiences into a coherent thought -- then you are probably close to identifying him/her. These are the folks who, because they have an innate sense of the overall 'design', can understand how fiddling with the details can impact the balance of the overall system...

Paul said...

Hi Brad,

I know what you mean. Someone who is happy going up and down the ladder of abstraction. Jumping from specific details up to the big picture and back again.

These people are definitely in short supply for sure. Andrew speaks about them in his blog too.

For me I am not sure whether it comes down to nature or nurture. I do know that as a profession we aren't doing much to nurture such people. Being pigeon holed at one strata of abstraction doesn't help.

Maybe there are more people out there with this latent natural ability then we think?