Testing Times for Testing

Went to the Extreme Tuesday Club (XTC) last night. Talked to a couple of guys from the Big Media Company and then to a couple of guys from uSwitch.com. There was also a bloke from an investment bank (Josh, not his real name).

The bottleneck was the QA guy

Curiously, all the conversations - I don't know whether there had been a talk about it or something before I arrived - were about testing. The Big Media Company guys were doing Kanban and Kanban had shown them that the bottleneck was "the QA guy". And they were wondering how to get round this. The guys from uSwitch.com had taken what sounds like a really radical decision and just got rid of their dedicated testers. But they said that the result was that the developers took responsibility for the code. Of course, you can only really do this if you've got TDD (they were calling it BDD) in place. You write your automated acceptance tests with the business people and then you code until the tests past - a common theme from both the uSwitch.com and Big Media Company guys was that you release stuff when it's ready, you don't wait for the end of and iteration.

Testers focus on finding bugs without regard to value

Why did uSwitch.com get rid of their testers? Well, the testers weren't really on board with Agile, and one thing you have to careful of is anti-Agile diehards who bring progress to a halt, but the testers were also focussing on bugs without regard to value. The developers at uSwitch.com keep an eye on the error logs from the live site and they keep an eye on the conversion rate - how many people who visit the site actually make a purchase. If something causes the error logs to spike, or the conversion rate to drop, they're on it. The focus is on delivering value to the site. Another knock-on effect of not having any testers is that the business people, the product owners, have to "be their own domain experts". Again, another theme that emerged from last nights discussions is that there is a temptation to leave final responsibility for the product with the testers. Testers can be expected to have a detailed knowledge of how the system should work, but they're unlikely to have a detailed understanding of which bits of the system are of value to the business.

Offshore testers, are "paid by the bug"

Josh worked for a merchant bank that was still clinging to traditional waterfall models of development. He'd had some success on a few projects in introducing some Agile practices, but again, testing was a problem. In order to make sure that everything worked as it was specified, testing was done offshore and the offshore testing company were pretty much paid to find bugs. It would be very hard to expect this company to know, or to care about the business objectives of the company. So the list of bugs that was sent back by this offshore company had no prioritisation in terms of business value.

Josh also raised the problem of time with the traders being very valuable, that all the time that a trader was talking to them, he was losing money. We talked about whether the amount of time the trader spent on the software was going to be the same in an Agile project and in a conventional waterfall project. The guy from uSwitch.com told a story about some work he'd done at a previous company, where he'd traded in his desktop machine for a laptop and just gone and sat with the guys who needed the software. He'd watch what they were doing and occasionally ask them questions, he'd also get to show them the software he was writing as he was writing it.

Quick - we need a crisis!

We talked about there being no way of persuading people to go for this without actually giving them the experience of doing it. You could try to explain the benefits of an Agile/Lean approach until you were blue in the face, and it probably still would do any difference. One thing that we thought might make even a merchant bank take such an approach was if it was seen and understood that their competitors were doing something similar. We agreed that very few people take such radical approaches when they're already making money. Even the Japanese (and then only some Japanese) only tried these methods when their country had been defeated in a war and economically ruined. We discussed some ways of artificially creating these crises. The second time I'd had this discussion in a week.

For further information, contact Mark@agilelab.co.uk (07736 807 604)