Rene is a senior test adviser and test coach with over 17 years of experience in software testing. He advises organisations on Lean Quality, test management, test change management, structured testing, and manage test (change) projects. This year he will be the chairman of Romanian Testing Conferences and we had the pleasure to take this interview.
1. In an interview for Professional Tester magazine (Getting warmer - http://www.professionaltester.com/magazine/backissue/10/ProfessionalTester-August2011-Tuinhout-and-Janssen.pdf) you made an interesting affirmation about viewing testing as a risk mitigated rather than failure prevented. Could you please elaborate this idea for our magazine readers?
The idea behind the statement you quoted from Richards and my article is quite simple: We found that discussing the benefit of testing was quite complicated, and in most cases ended in a “let’s agree to disagree”-situation, when the discussion was about failures found (and fixed). We, as testers, would plead that a specific failure found was quite similar to a failure not detected in another project which caused a loss of X euro’s when it was finally discovered in real life use. So, we would say, we saved the business X euro’s. However, the business would plead the failure found (and fixed) was quite dissimilar to that previously not-detected failure, and therefore the comparison could not be made.
Leaving us with no grounds to demonstrate the benefits of testing in a language the business understands: money.
However, this changed when we switched from discussing the benefits of testing based on failures prevented, to discussing the benefits of testing based on risks mitigated. In the article you mentioned, Richard and myself present a simple method which allows business managers (supported by testers) to map business risk types for their specific business. Next, concrete risks envisaged in the change/new product the business wants are collected from business staff (users, operators, dba etc.). Combining the business staff’s and the business management’s inputs, all concrete risks identified by the business staff can then be mapped onto a risk thermometer, using the business risks provided by the business managers.
This will lead to a model represented by a thermometer, showing concrete risks, categorized in a way with which the business management agrees.
This has three advantages:
Business staff can bring their expertise about concrete risks to the table;
Business management can bring their expertise about risks for the business to the table, and can decide what risk types (and related risks) should be thoroughly tested and which risk types (and related risks) are acceptable;
This latter point deserves some elaboration:
When discussing the risk types with business management, it usually turns out management wants to express risks in monetary values, i.e. “what would it cost when this risk occurs”?
If this doesn’t emerge during the meeting with business management, business management can be suggested that expressing the risk types into monetary values might be a good idea.
As a consequence, all risks now carry a monetary value (usually expressed as “it will cost at least Y and at most Z”).
Later on, when risks have been classified into the categories indicated by management, tests will be designed to cover the (most important) risks.
This automatically means that a failure detected when testing to cover a risk would have cost the business “at least Y and at most Z”, enabling testers and business management to discuss the benefits of testing based on a common ground: money.
2. Deciding what should be tested or not based on the thermometer approach looks like a simple and powerful tool for making people better understand the importance of testing. How is this working in practice for real projects?
In one word: Great! The article you mentioned was published in 2011. Even before that, the thermometer was used in some projects, and after publication of the article it has been used in many more. It has proven a powerful tool to:
Help business management make management decisions about what risk types to accept and what risk types not to accept (i.e.: what must, should, could and won’t be tested on a conceptual level);
Help business, project and continuity –management make decisions about (re-)prioritization of effort spent, based on business risks;
Help business staff map real life risks onto a prioritization framework, showing them how their risks relate to the real life risks other business staff have mapped onto the model;
Help testers to identify which tests have which (must, should, could, won’t) priority;
The model is mostly used as the thermometer-model. However, in some cases it has evolved to a two-dimensional matrix, which provides business management with a bit more detail (basically, risk is presented on the thermometer as one value, whereas in the two-dimensional matrix, risk is split up in impact and likelihood).
3. There are projects that are delegating testing tasks to software engineers by making unit testing and integration testing. Also, we saw projects where the testing is basically done by the product owner. How do you see this approach?
Although I value these developments, I also see value in involving professional testers in testing efforts. Software engineers and product owners should definitely be involved in testing. Some developments, like Agile, now pave the way for software engineers and product owners to be involved in testing. Something many professional testers have asked for years: it results in software delivered to testers being of better quality (because it has been low level tested by software engineers), and it results in product owners being involved with product development and testing much more.
However, excluding testers from product development means ignoring a very important aspect of the tester´s craftsmanship: constructive destructivity.
System engineers as well as product owners are focused on making things. They tend to focus development and testing efforts on those aspects of the product under test that this product is supposed to do.
Testers have the ability, skills and toolset to discover and focus on those aspects of the product under test that this product is not or very infrequently supposed to do, as well as on non-functional aspects not easily tested (for example security, performance, usability, maintainability), thus discovering potential failures system engineers and product owners are less likely to find.
Therefore, I feel the testing efforts of system engineers, product owners and testers are supplemental to one another.
(By the way, I think a great benefit of all these disciplines being involved in testing is the possibility to learn from one another: The system developers learn a bit more constructive destructivity, the product owners gain more insight into the technical difficulties of making a product and its associated risks, and the testers learn quite a bit about programming and the risks the product owner needs to have covered.)
4. If tomorrow someone asks you to create a testing university, what will the study materials look like?
A great deal of effort has been made by a Dutch TestNet (Dutch association of testers) workgroup, developing an academic/polytechnic curriculum. It would be too elaborate to discuss the curriculum that has been created in detail in this article, but the curriculum focuses on tutoring the students about user interaction, business processes, software, infrastructure, hardware interfacing and the skills analyzing, advising, designing, developing and maintaining. All materials about this are in Dutch, but if you’re interested, send me an email and we can have a chat/mailsession about it.
5. Congratulations for getting “People’s Choice Award” at RTC in 2014. This year you will be the chairman at RTC. What interesting topics are prepared for this year?
Thank you! I was very honored and humbled to have won the award, and being ask to be this year’s chairman. There are many interesting topics and (seasoned as well as novice) speakers on this year’s RTC.
For an overview of the entire program, please visit www.romaniatesting.ro. Personally, I think the Romanian Testing Conference 2015 has many interesting speakers (a.o. Peter Varhol, Markus Gärtner, Team Army Ants (Software Testing World Cup)) from a lot of interesting companies (a.o. Adobe, Cisco, Facebook) and focusing on interesting subjects that are hot today (a.o. cloud, privacy laws, test automation, test data quality, internet of things).