Misconceptions about software testing that I come across (and fight with)
TESTING
I have been a tester since 2004. I have taken many roles in 13 years but have always considered myself a tester. I have not seen it all, but I have seen a lot. I have been ecstatic when I found bugs that were on the verge of an insane check and terribly ashamed when I missed something that made using the product difficult or impossible in production. I have hidden away in my projects and company believing that I am a good tester I, and I have been on stage, in front of hundreds of people, expecting the one question that will make me realize how stupid my assumptions are.
I have also come across a lot of misconceptions about software testing. I am sure that you will agree with some of them, maybe dislike the way I say it, but I believe the testing craft is about sharing ideas and experiences and this is what I am doing. So here they are:
- Everyone can test. Saying that everyone can test is like saying everyone can drive. Remember this next time in traffic. If you are firm in your beliefs just have a look over TheTestingMap.org. On this map I put everything I believe a tester should know.
- Testing can be done under any circumstances. It is true, but the results won't be the best. You need to remember that for testing to take place you need to have testability. What does testability mean? To remember it easily use the mnemonic CHAOS UI ( Controllability, Heterogeneity, Automatability, Observability, Separation of concerns, Understandability, Isolateability)
- Understanding requirements means reading them. In order to understand something, it is never enough to read it. You need to understand it. For this, there are techniques that you can use. Techniques like: Critical Thinking Questions, Sashimi, Simulate and Model, Cubin, Drawing Diagrams and Flow….
- Automated testing. This is a unicorn. Everyone talks about it, but no one can see it. There is automated testing in as much as there is automated development. The idea that there can be automaton doing a testing job is something for the future. The correct term is test automation. You define some test first, then based on a return of investment you choose to automate some checks. Because in the end, there isn't such a thing as test automation either. It is check automation. You instruct a program to check for certain things. The program will never say it's odd… I'll check this also. It will do, and only do the things you instruct it to do.
- Manual vs. automated testing. First of all, see the point above. There is no such thing as automated testing. Second, all testing is manual. It is done with the hands in the same way writing programs is done with the hands. Manual is not the proper word that illustrates all the brainstorming and thinking activities. As Michael Bolton puts it: "we help to cheapen and demean the craft" by using the manual next to testing (if the first thing that comes to your mind is how can a singer talk about testing, then you are thinking of the wrong Michael Bolton and you should spend some time listening to the great minds in software testing). Manual has a "not so smart" appeal to it.
- Testers are unsuccessful developers. If you are a good tester that knows testing techniques, uses critical thing, and knows how software is developed, you will become a great developer if you want to. All you need to do is put some intention into it and be prepared to be a junior in a new field. I have seldom seen testers become developers. More often, they become Project Managers, Scrum Masters, Business Analysts. From the initial testing team that I started to work with in 2004, I am the only one that still has testing at his heart. I believe testers move to other position for two reasons: they get bored (not really understand the craft) or they become more valuable for the company in these new positions. Being a tester means that you get a lot of experience regarding what works, how to deal with people, foresee expectation….
- Testers with 5 years' experience are senior. Let's not confuse 5 years' experience with 5 times one 1-year experience. Experience comes from dealing with different applications, technologies, different teams, different ways of working. What makes a tester a senior is the capacity to shine in any testing-related context not only on the application that you worked on for the past 5 years.
- Quality assurance is testing. Let's start with the truth: quality assurance is not testing. Quality assurance is defining a way to ensure that the end product is of quality. Quality does not just happen: it has to be stated and defined. A tester (quality assurance engineer as it is often called) is not responsible for the code quality, for other team members respecting the process or not. If you want to know more about quality assurance, please check number Today Software Magazine where I describe in detail what quality assurance means.
- Testers have to find bugs. Bugs are just one type of information. Testers should deliver relevant information about the factors that stakeholders consider relevant.
- Testers break software. As Michael Bolton beautifully put it "testers do not break software, the software is already broken".
- Counting test cases/execution is a good thing. Counting test cases/execution is the same as counting lines of code per developer. It does not give you any relevant information. It's a waste of time and resources. Stop thinking in quantity but in quality. For instance, for how many failed automated checks do we find a bug? This is a good metric to measure your automation success and quality.
- Testers just have to be smart. Of course if you are really smart you can be good in many areas but for software testing you need to know at least the testing techniques and approaches: what they are, how they are used and what results you should expect. Testing also needs a lot of learning and introspection.
- I am a good tester. You might be. But do you really feel like belonging to a craft? Do you contribute to your community? Do you even belong to a community? Same as in your family, in your project, you might be a good tester (person). But the real measure of your goodness, I believe, comes from sharing and getting feedback in regards to what you do and how you do it from your fellow testers, in local communities and at conferences. In my experience,when attending a meetup or a conference one of two things might happen: you learn what great stuff others are doing and get inspired to do it as well, or, as Ion Creanga says it: "I know I am stupid, but when I look around I get courage". Moreover, if you start to believe that you can do things better, then you should definitely start doing them.
- Testers are the bearers of bad news. At first glance it might be true, but a tester also gives you a glimpse in the future. Testers should say: "You are not done yet!" There are still things we have to do. Knowing what you have to do to get there is not a bad thing!
- Testing is an analytical activity. This is a really tough one. It was the thing I believed at the beginning. It is simple right: you have some requirements (regardless of how they look), you have some testing techniques and approaches and you can produce valuable tests. Right? Well it turns out that even for the simplest of crafts knowing the techniques is not enough. How to use them, when to use them, how to get the best results in the least amount of time comes only with experience. And since testing is craft, you should always look for a master, a mentor or maybe mentors to guide your way. If you still believe testing is analytical activity go to testingchahallenges.thetestingma.org and see if you can find all the 18 checks required for testing a single text field.
- It's good & ok to test at the end of the Sprint, when working with Scrum Let's look at the Scrum guide "Scrum Teams deliver products iteratively and incrementally, maximizing opportunities for feedback". It does not sound like much of maximizing opportunities for feedback if you test at the end, right? Let's look also at the 7th principle from the Manifesto for agile software development "Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.". What kind of pace is it if you test at the end? More of a leap. Testing at the end will never give you the opportunity to maintain a constant pace indefinitely.
If you see any misconception about testing around you, do not stand aside. Fight it. Dispute it. Challenge it. As long as we say automated testing or use the term manual testing, we will be the ones to be blamed for letting this misconception become common understanding and knowledge.
