More thoughts on Open Source and Skateboarding

Just started reading Coase's Penguin - an article trying to figure out why Open Source happens at all.

And the idea just popped into my head - Open Source is the skateboard park of the aether.

Notes from a fry-up lunch

I had a fry up lunch on Wednesday with a friend of mine who's a developer and two of his fellow developers (he can own up in the comments if he wants to) a few things came up in conversation that I want to note down here - I might write longer pieces on them later.

• Quote: "The two projects that we made money on are the two projects where we said [admitted?] that we don't know". The standard case where the customer comes to you and says "How much for X" and you say Y pounds RARELY MAKES MONEY! But the model where the customer come to you and says "How much for X" and you say "Mmm, dunno. Do you want to pay me a little bit of money to have a look?" THAT MAKES MONEY! Maybe you can't get there straight away, maybe you have to do some fixed price stuff to get to that stage, but isn't good to know where you should be heading?


• A compelling reason for becoming involved in open source is that you get the respect of your peers. You also get coached by people who know what they're talking about. This is very different from the kind of feedback that you get from customers which is all about bugs or things not looking the way that they should. Perhaps pair programming could be a kind of mini, in-house open source in this way.


• Related to this - the loneliness of the self-taught programmer. Most programmers are autodidacts - they teach themselves. This means that some areas are very strong and some areas are very weak. When other people look at their code self-taught programmers can feel very foolish. If you're a self-taught programmer and you're trying to work within a framework which is supposedly helpful to development, such as test driven development, or pattern-based design, this can be very traumatic and cause you to really lose self-confidence. Maybe this is why some guru/surgeon programmers are so reluctant to pair program.

The Loneliness of the Self-Taught Programmer

I have a friend who is very well-read in all kinds of fields. He has a PhD in Organic Chemistry from Cambridge University but he is also widely read in Russian and French literature. He bundled through the Penguin Classics well before we left sixth form. He likes all kinds of pop music and he also likes opera. What's more he achieved all this culture and erudition from a relatively humble background - most of his reading was done in the ice-cold back bedroom of a council house in West Yorkshire where he grew up.

Now aren't we all lead to believe that education and culture are a good thing? Shouldn't having read all these books have made my friend more confident and well-received when he did finally get into polite society? Well, it should have done, but there was slight problem. Pronunciation. He'd read Proust, he loved him, but he didn't know that his name was pronounce to rhyme with "roost" and not "roust". He loved Puccini but he didn't know that the word "aria" isn't supposed to rhyme with "Mariah".

And so when he did get into polite society, people laughed at him. People who had never read Proust still felt it was all right to laugh at his pronunciation. He learned the correct pronunciations really quickly of course, but he never forgot the sharp humiliation of those condescending smiles.

I was at lunch yesterday with 3 coders, one of whom I'd never met before. We talked about Agile stuff for about 10 minutes and he didn't say anything. Finally he said that working on a project that used Agile methods - particularly in his case, test driven development - had made him completely miserable. After he'd left the project he'd lost all faith in his abilities to code. It took him several months to regain any confidence.

Learning to program is a back-bedroom, late-night activity for most people. Most people who do it, are autodidacts - they teach themselves. There are university courses that claim to teach it, but most of them miss the mark. Either the course is too prosaic and equates teaching programming to teaching the syntax of a certain language or the course is too theoretical and imagines programming to somehow happen as a natural bi-product of understanding certain branches of mathematics.

Even if you're lucky enough to find a course that does teach you programming instead of computer science or discreet mathematics, in order to actually write anything that works there's a ton of arcane detail that you need to learn and there aren't many ways to do that other than dragging through tutorials, manuals, FAQ's and forums and other people's code by yourself. By the time you've done that it's not likely that your knowledge is going to be rounded, flexible and generally applicable. It's more likely that your knowledge is going to be, at least in some areas, brittle, specific and sketchy. So what are the chances that you're going to want someone to look at your code?

And maybe this is what is behind what seem like stubborn denials, refusals and condemnations of pair programming. When you suggest to someone that you write code with them and their response is "I taught myself to code, why should I be forced to do all the work for some rookie who can't be bothered to stay up nights himself," or even "I don't have to drink a bucket of vomit to know it's a bad idea." You might guess from the emotive reaction that there's fear involved. Perhaps the fear of the autodidact.

How can we persuade people that Agile Methods reduce costs?

I've just read this article by my friend Tim Diggins. I think it makes very clear why he uses Agile methods and what that actually means for him - as he says - what he does do and what he doesn't do. I don't think it's so good at convincing people who don't really know anything about Agile methods that they really lower costs - so here's my attempts, in note form.

Lower costs are what they're interested in so that's what needs to hit them first. It's like the story about James Thurber - he had a job as a crime reporter and he was asked to cover a murder on his first night. He reported how he'd heard about the murder, how the police had been alerted, who'd actually discovered the body. When he gave it to his sub-editor, the sub-editor said "No, no, no! This is all wrong! You have to start with the most important detail, the most important word even, first and then give the details in order of importance rather than chronologically"

So Thurber re-wrote his piece and started it off - "DEAD - that's how they found him!"

So how would this start?

"CHEAPER! That's what your software costs will be when you use iterative development." Even this is better. It's to the point but it's not obvious how to bring in the arguments you'd need to convince the reader.

Another tack is the Minto method:

• Bland statement nobody can argue with
• Sophistication/Complication
• Solution that deals elegantly with the Sophistication/Complication

So for Agile this might be:

Everybody wants to lower the costs of software development.
But this isn't easy because a lot of the costs are hidden.
They come late in the process because:

• That's when the clients and the developers realise they weren't thinking of the same thing
• That's when developers charge elevated prices for requested changes to recoup costs for wasted work
• That's when it becomes obvious that some new technology doesn't perform as well as the hype would lead you to expect
• These costs can actually be so high that the project is abandoned - the customers get nothing for their money - the ultimate high cost.

Iterative development [Here is where you would teach the concepts of iteration and working software] which produces working software throughout the lifetime of a project is aimed directly at reducing these late and hidden costs.

• Every time the client is presented with a new piece of working software, the understanding of the clients and the understanding of the developers about what the software is for and what it should do is tested. The chances of terrible misunderstanding (and big late costs) are reduced.
• Changes can be added throughout the project, because negotiation in an Agile project can be in terms of scope, as well as price, it may be that changes may have little effect on cost
• The emphasis on working software means that trouble with new technologies emerges early in the process. Something can be done early to work round the problem and find and alternative solution.
• Iteration by iteration the customers get closer and closer to actually having the thing that they want. At all stages they have *something* that provides the most important functionality. The chances of having to abandon the project are greatly reduced. Conversely, if a project just isn't a good idea, this will be obvious much earlier than with a conventional project, the costs of cancelling it will be reduced.

Negotiations and Web Sites

After teaching a course last Friday I began to realise how important negotiation skills are to software development, and how powerful the combination of Agile methods and negotiation skills can be.

One of the stories I often tell on our Introduction to Agile course is about the first two projects that I worked on when I started out as a software engineer. Both projects were huge. Both projects were making good money from the company in their maintenance phase, they generated a steady stream of changes and updates which my company charged top rates to provide. Both projects had reached the end of their initial development phase on the verge of disaster. Both projects had got to the point where the customer was threatening legal action. In one case the customer was the British navy. If they had sued it would have probably been the end of the company.

In both cases, just when it looked like the projects were about to end in disaster, something very interesting happened. The projects brought in a negotiator. I think I saw him once on another project that I was working on that was about to reach its "Critical phase". He was short, (even shorter than me) and stocky with a totally bald head. They called him the Bulldog. All he ever did was go from project to project which had reached crisis point and negotiate with the customer. He would turn up to meetings with the customer and let the customer vent their frustration about not having any software, or about having software that could only run for five minutes without falling over or whatever it was. He would then try to get out of them what was the most important thing that the software had to do and also get out of them a little bit of money and a little bit of time in which to do that thing. He would also persuade them that moving the project little by little towards totally working was infinitely preferable to calling in the lawyers. If the lawyers were called in, all work would have to stop. After the dust had settled - in a few years maybe. The customer might get some of their money back, but they still wouldn't have any working software. Whatever the problem the software had been commissioned to solve would still be unsolved.

The theory of negotiation basically says that everyone who's party to a negotiation has something called a BATNA. I know, it's not a very pleasant acronym. It stands for "Base-line Alternative to a Negotiated Agreement". I think Agile methods, and especially the practice of delivering working software all the way through a project are very interesting. In their book Getting to Yes by Roger Fisher and William Ury, the authors recommend that negotiations move from positions to values. This is what happens when a project moves from talking about a specification document to working software.