Exploratory Testing as Exploration with Judgement

rather unfinished, saved to allow me to move on, re-published because @SimonTomes saw it while it was briefly up!

I'll get back to work on it ... soon?

Let's dive briefly into: Testing requires comparison, and using our judgement to value that difference.

For a comparison, we need to hold two things in mind; typically when we test we're comparing something novel and observed (and therefore external) against something known to us (and therefore internal).

To judge, we need a framework to help us distinguish something we value. If we're judging Good against Poor, we'll need an aesthetic. If we're trying to tell Good from Evil, we might need ethics. If we're judging Acceptable from Unacceptable, we might need law.

A simple framework might be a list – you want to identify everything on a list as "mine", and everything else as "not-mine". Or "Dangerous", and everything else as "not-Dangerous". It's easy to slip from this into "mine" and "yours", or "dangerous" and "safe". If you fall into that trap, then you've wrapped the infinite with a label.

You might use two opposing lists – and wonder about what falls into neither. You might have a scale. You might have several scales.

You might make lots of observations, then find patterns and groups within those, then – with a classification framework in mind – see if that framework implies anything that you've not seen, or not looked for. You might look for alternerate frameworks, some complementary or some conflicting. You might want to look at your observations, and see what falls outside the frameworks.

The framework might help us see who's missing. Or it might help us see what looks weird.

Let's think of Exploratory Testing as exploration with judgement against a model.

Unlike my example above, we don't necessarily know what the thing we're looking for will look like. Example: let's go to a different club, looking for new music and new people. That can be uncomfortable – we're out of our familiar place, we don't know what we're looking for, we'll probably make some rotten choices as we find out. ¿If we hang out with the people we know, perhaps we won't meet anyone new –or if we do, we'll meet them by chance – and have an easier time talking withthem?

Often, we're expected to restrict our judgement to situations where we're working through a set of expectations, judging whether the system has a behaviour that matches. Testers can be uncomfortable with exploring to find out what a system does, because they feel they should start from a point of exploring what it is.

We can work without requirements. We can work through a system, judging what it actually is against expectations – and doing that, we hope to find behaviours and qualities we don't expect.

I try to help people to explore something to find out what it does, and from there to describe what it is. We're not testing; we're making a model. We're not particularly judging the system, but we are judging the model we're building to gauge how well it describes the system.

Here's :more on exploring without requirements