Exploration without Tools is Weak and Slow

A rant, which I should have written years ago. Now's as good a time as any. Consider this unfinished until I've wrestled with it more: v0.2.

This is a core idea in the way that I practice and teach software testing.

It seems obvious to me. So obvious that it hardly bears remarking. And I need to say it in pretty much every workshop, and to pretty much every client, at some point. So I'm the one who doesn't get it. And therefore, I find it hard to explain.

Two fingers to the simple metaphors of exploration.

Let's consider exploratory testing.

You are the tester. Not the tool. Your speed and throughput of testing is dictated by your limits as a human.

Is that human limit to do with being slow at typing? No. So get good at triggering your systems at the speed at which they can be triggered.

Is that human limit to do with being slow at designing? No. Test design takes time, but can be BIG. A single obvious test can have thousands of interlinked moving parts. So get good at building tests which cover, rather than peck. Even for those tests whch are designed in response to what you've just seen.

Is that human limit to do with being slow at reading the output? No. You can design a visualisation which can show you the results of millions of tests at once, use your extraordinary predator brain to pick out the patterns, then use the visualisation and your brain together to slice through the data and build an evidenced model to verify or refute your intuition.

I watch some people exploring, and their clear limiting factor is the one-by-one experiment. Which is often a joy to do: in the best moments, it works like a precise karate move to the vulnerables, followed by a satisfying folding-up of the system under test.

But that one move typically comes after several puzzling and unedifying ones.

Your human limits are the limits to your imagination, your judgement, your cunning. For all that other stuff, there's a tool.

If you're going to systematically push through some iteration over the real behaviours of a system, you need a tool to work at the speed of that system. You need a tool to set up every condition in a neat and reliable way. See it as a bonus that the tool will mean that you don't grind your mind to soup with the tedium. You need a tool to let you look through thousands of pages of aggregated crap like the omnipotent consiousness you are (by comparison with the machine holding the data).

Get off your arse, wiggle those magic fingers, and use a fucking tool already.

And, hold up, not one of those tools that does what a user does, but stupider. You need a tool you can drive, not watch. You need a data generator, a comparator, a grapher, a validator, a parser, plumbing and adaptors, data and data furklers, collections of rotten files, schedulers, sequencers and triggers. It's not going to be one tool (unless, at a pinch, it's Excel) but a quivver of tools, an arsenal, a pack. Each one a slice of superpower – and you're the one to weld all these things together into a cobbled-up one-off chimera to set up you, you tester, for that one slick move that leaves the system under test neat filleted and showing off the next flaw to be fixed.

Go on, get to it. The days of hand-cranked exploration are done.