Wrangling, Debugging and Testing (conference talk abstract)

Superseded by

Wrangling, Debugging and Testing

Do we spend months getting our systems to a point where we can test? We do. Here’s why, and what we can do about it.

Workroom ProductionsJames Lyndsay

Abstract for a 40-min talk. Seems a bit unlilkely, to me, that I can fit this content into that time. So it needs refining.

Before a system can be verified with even the simplest automation, it must work, and work right. Getting to that point can involve testers in months of system wrangling and debugging. Yet our industry pays less attention to this necessary task than it does to more controllable or satisfying testing work. This talk seeks to redress the balance with stories from integration projects testing systems critical to large businesses and to national infrastructure.

Exploring wrangling (getting the system to work), we will look at situations where the system under test could not and would not work in any test-accessible environment, and at situations where testers could not gain adequate access to critical elements of the system.

We will cover problems such as getting the plumbing right between system components, orchestrating authorisation for tools and testers, working with configuration data and managing interlocking shared environments. We will look into ways that testers can gain technical savvy and leverage political and expert champions to unlock problems.

Focussing on debugging (getting the system to work well-enough to start testing), we will look into situations where subject-matter experts and technical experts rely on trial and error persuade the working system to process data, and where the earliest actions on the system revealed fundamental problems in performance, security, and usability.

We will cover designing bulk experiments to take the slog out of trial and error, creating configuration data and consumable data in enough variety (and in enough volume), and will talk about ways to get access to logs and transaction information. We will cover ways to bring organisationally-distant colleagues together to resolve trouble with cross-disciplinary insights.

Throughout the session, the speaker will give information about what worked to mitigate some of these issues. We will look briefly at how time spent wrangling and debugging can impact an organisation expected test activities  and metrics. We will not cover ways to disguise debugging and wrangling in project plans, and the speaker will resist the urge to offer platitudes and simple metrics. We will not cover how to plan testing to include this often unanticipated work in a way that doesn’t scare the project board.

Use at work

Some may look at their work of wrangling systems in order to test them, and think "We can do better, and this is how"

Takeaways

• Stories from real systems integration projects
• Common pitfalls when testing several interlinked systems
• Some worthwhile approaches to testing several interlinked systems