Imagine you need a paint job for your beloved collection car, or new house. You hire the best contractors, instruct them to use the best supplies on the market, give them the best tools and pay them a little more than the market average. They finish the job on time, within the budget. Everyone is happy and you may throw in a little party. But, within a month or two, small rust stains become visible or little cracks start to develop. What went wrong? The best team you hired did not allow for the foundation layer to dry properly, according to the supplier instructions, and started to apply the paint after 4 hours, instead of 6.
What is missing from this scenario is a proper work process defined and monitored. It was not about skipping an entire phase of the process, something that could have been spotted on time for fixing before the delivery of the final product. It was just a small "shortcut" that may look like an improvement for the moment but, on the long term, proved to be more expensive than the small delay that would have happened otherwise.
"Quality Assurance" is a set of activities often overlooked in SCRUM teams. The reasons/excuses vary, from tight schedule and the need for a fast delivery, the quality of the test activities or, worse, not understanding the concept at all! Yet, this is not something that should be so easily put on hold, or even postponed until the time is right, as it may never happen. And the consequences could be significant, ranging from time spent fixing (on your expense) after the delivery to losing the trust of your customer. Furthermore, Quality Assurance is not to be confused with "Quality Control" - the latter being the final step in the QA process, as it needs a functional, potentially shippable, product to be sure about its effectiveness.
Let's start with the definition of quality: quality of a product (either an airplane, a hotdog, a service or a piece of software) is a distinctive attribute of that product, when compared to another, similar product! Sometimes we say that a product A is a "quality product", but that makes sense only when compared against other similar, available, products. More often we use expressions like "product X has a better quality than product Y" - and that makes more sense according to the definition above. In the end, quality is delivering according to the customer's expectations. Not less (for obvious reasons), but not too much either, as you may not get paid for the extra effort.
The quality of a product or a service is the sum of the various quality criteria, either clearly defined by all parties involved at the beginning of the product lifecycle or (sometimes) implied without written agreements. Such attributes are (or should be) the result of a rigorous analysis (and understanding) of the user needs. These needs will be expressed as "quality factors" that will be later built into the final product as the "quality criteria" mentioned above. Those quality criteria have to be measurable and translated into quality metrics that will be used later on to assess the quality of the product (during its development lifecycle and at the moment of delivery) and also as a baseline for further improvement.
The quality of a product is a mix of everyone and everything that contributes to the development of the aforementioned product: the people doing the work, the technology (tools) that are used and the processes that are followed for the entire product development lifecycle. Quality Assurance aims to find and implement a perfect balance between those 3 "ingredients" mixed for a successful result.
Fig. 1: Role of the QA in a project landscape
Ultimately, quality is what differentiates your product when compared to other similar products. Most likely there are other organizations out there that can provide the same level of expertise on a certain field; some of them are developing a very similar product, some of them cheaper, some faster (or both). For the customer of the year 2015 there's nothing that can stop him from grabbing your competitor's product, so the quality built into your product can make the difference. It does not have to be perfect -there's no such thing as zero-defect product- just to fit perfectly to your customer's needs and be a little better than other's.
Peter Leeson -CMMI and Process Improvement Lead Appraiser and Instructor- summarize this in his article "Understanding Quality"1:
Understanding quality is the most critical aspect of your job, whatever it is. Quality is what differentiates your products and services from the others. If your sole focus - as reflected by measurements is quick and cheap, you will lose the battle: there will always be someone cheaper than you.
Quality Assurance is the set of activities aiming to build quality into the final product. It involves every participant in the lifecycle of a product's development, at any level (regardless of seniority or the level he/she is in the organizational structure). As long as someone can influence the development of a product and/or service, he/she is responsible for the quality of the result.
Quality assurance activities can be split into two categories. First aims to prevent the defects that would arise before the development cycle ends, by defining a standard way of working, checkpoints and reviews. Measurements of various KPIs, analysis of the results, taking corrective and improvement actions as well as regular assessments of the way of working (at all levels) are part of this process. The second category focuses on defect detection into the completed product and is better known as Quality Control.
Of course, all this comes with a cost. In the early phases of a project one needs to plan every QA-related activity: identify customer needs, translate them into quality objectives, define measurements, plan risk mitigation and contingency, plan to monitor the progress, define/identify the best work process for the product to be developed and so on. During the project lifecycle there should be regular monitoring of the performance and quality currently under development: checkpoints planned; plan work product and document review frequency; assess the progress (or the regress) for the period since the last checkpoint and take the appropriate corrective actions if necessary; assess the rapport of the costs vs. value of the activities taken (i.e. cost of review vs. cost of fixing a defect discovered after the product is launched) and adjust the work process to best fit the needs of the project.
Furthermore, the results of the various measurements collected throughout the project lifecycle, together with the proven (!) effective practices are inestimable assets of your organization. These can be used as a starting point for development of yet another successful product or when organizing a new team. Having a set of good practices, validated in time, will help you integrate new capacity into the existing environment, either a new team member or a new tool or a new process.
Bottom line, the Quality Assurance activities are vital components of any product development process. It can help your team to stay on the agreed track and, based on measurements, guide back if necessary. It is similar to the GPS device that everyone uses these days. It gives you a route to follow and estimates the time and the cost of the journey. Still, it will not force you on the road, but it will suggest you alternate routes or provide guidance if you wish to get back on a previous trail.
The first statement of the Agile Manifesto states that those who embrace this philosophy values "Individuals and interactions over Processes and tools". So, adding a new process into an Agile environment does not seems like the right thing to do. Right? Let's see!
Further reading the Agile Manifesto, it is clearly stated: "That is, while there
is value in the items on
the right, we value the items on the left more." Or go to the "History" page of
the Agile Manifesto, written by Jim Highsmith2:
The Agile movement is not anti-methodology; in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes.
Check the Twelve Agile principles3: nothing "stops" you to define a work process to best suit your needs and make use of what you've learnt. I assume nobody ever said he values collaboration over quality. And, if someone ever said that, he is most likely out of business by now.
Could it be that Quality management processes do not go against the Agile principles? Is it possible to define work processes for different stages of the project lifecycle and different roles involved in the product that is developed, so that these processes will actually HELP a Scrum team in its day-to-day activities?
Scrum (most used Agile 'flavour' these days) is a model, a framework for the software development process. It emphasizes the collaboration between team members and between the team and the customer, aiming to deliver faster and have better response to new or changed requirements. Ideally, the team working by the Scrum model is self-organized and mature. Every role is filled, everyone knows his/her responsibilities and they know what they have to do (and they do it without any hesitation) plus they adapt quickly to the changes in their surrounding landscape. This makes me think of a small Special Forces military unit instead of a group of computer nerds (irony intended). Unfortunately the maturity level required for a Scrum team to be effective is not something that builds overnight. Your team must have a minimum understanding of the business they're addressing to (so the product they build will solve a problem for that business) and your team members must genuinely trust each other. They must be allowed to try and fail in order to come up with the best solution for the issue they are currently facing.
As it was described above, everyone in the team plays an important role in building quality into the final product. The common pitfall is that the responsibility is diluted among the team and is often taken as granted ("we're experienced professionals; we know what we have to do"). Also it is human nature to focus on what you're good at / paid for. Developers develop and testers test
Quality Assurance can help Scrum teams to clarify the goal and stay on track to actually reach that goal without too much deviation. It also helps clarify the responsibilities of every role involved into the product development lifecycle. Defining a clear way of doing things, no matter it is about development, handling issues or setting up proper monitor and control mechanisms will reduce the risk of making the same mistakes again later on or when approaching a new project. When there's a clear, measurable, path to the goal you're trying to achieve it's easier to observe when things are not going as planned. Any deviation will be noticed before it turns into an issue and improvements or counter-measures can be taken on time if necessary. It is a failsafe mechanism, a fuse box that can prevent or drastically limit the damage that could happen in some cases. A perfect design or system or development process is yet to be invented. Indeed, sometimes the cost of implementing a quality management system is higher than the benefits of having it (think of a really small project, that can be done quickly and the cost of the eventual defects can be easily supported). In this scenario the organization can rely on the experience of the team, but let's make this the exception, not the rule we're living by.
By constant (or at least periodical) review of every aspect of the product and project lifecycle, your work process will become a living organism that evolves in time. Scrum methodology focuses on the software development process, but that's not sufficient for a successful product.
As someone gets to know both worlds (Agile/Scrum and process-driven methodologies) there's an "a-ha!" moment when one realizes that those worlds are not mutually exclusive! You can define a work process that is "alive", that "evolves" in response to changes, based on Agile principles. Or, if you choose Scrum as you "primary weapon" for development, you still have to make sure that you have a good quality management process or that the requirements completely covers the business needs. This mixed approach is not new and not difficult to implement either! The most difficult challenge is to change the mindset of everyone involved, make them see beyond their current task or short-term goal. To make everyone aware that the ultimate goal is not to complete all tasks in time for demo but to build a quality product that fulfill a business need and can be remembered in a couple of years. And make sure the experience gained during the journey is not lost and can be used for your next quest.
Let's hear from Peter Leeson again4:
Over the years, different terminologies have come into existence, which are considered as the new way of doing things. In theory, it means that people have identified the weaknesses of the way they are working and are therefore trying to find a new, more successful approach. In reality, it appears that the weaknesses due to misunderstanding and misapplication of basic principles have led to results which are very different from what was originally expected. We then get a group of people who believe that the new approach is the solution to all their previous problems and start following it with religious fervour, throwing out anything which does not correspond to the new vocabulary and focusing only on applying what they have understood from what they have read in a book - soon they are producing the same mistakes as previous generations and it becomes time for someone to re-create the basic ideas...
In conclusion, as in most aspects of everyday life, the truth is somewhere in the middle. There's no absolute truth and no group that can claim having the right answer to any question. There's no such thing as a "silver bullet" or universal cure to every challenge we're facing in our daily routine.
ISDC answered the processes versus Agile dispute by following the middle path described above. More than five years ago a workgroup of several enthusiasts was created, with full support from the management. Their goal was to develop quality and continuously improve our way of working.
Every successful project was analyzed and a set of good practices was extracted from the experience gained through the years. These practices were refined and grouped into specific areas, covering all aspects of a project's lifecycle, from pre-sale to project closure. Today there are 24 process definitions, with approximately 300 tasks described in detail (who will do it? how should it be done? when? plus input and output criteria). They defined KPI's and measurements procedures and used these measurements to adjust the process definitions. Needless to say, the group is still working today, analyzing measurements, answering questions and implementing improvement suggestions received from the organization.
The processes defined became an internal standard within our organization and work began on disseminating the previous experience (synthesized into the process definitions) back into the organization.
As there's impossible to define a one-size-fit-all process for any work areas in all projects that come in, a set of guidelines for tailoring was created, to adjust the process definition to the specific project's needs. These guidelines are also under continuous improvement and constant review, making them and the process definition documents better and better every day.
A new team was created: the QA Officers team, led by a Quality Manager - to support the project team in defining the quality management plan, set up the quality objectives for their product and define a standardized way of working throughout the project lifecycle. QA Officers observe the everyday work processes within a project and suggest improvements (based on organization's process definitions) or promote good practices to the organizational standard definitions, so anyone can benefit from that experience. QA Officers are an independent group, thus ensuring the objectivity of the assessment of the performed work processes in every project. In addition to this evaluation, the QA Officers Team identifies and documents non-compliances, provide feedback to the project team and management on quality and performed work processes and it makes sure the non-conformities are addressed in a timely manner. As QA Officers are working close to the project teams they are the first line promoting the internal standards to the project teams and also help collecting feedback for future improvements.
This internal standard was built following the "Capability Maturity Model Integration for Development" (CMMI-DEV v1.3) created by the Software Engineering Institute, a non-profit research center at Carnegie Mellon University (www.sei.cmi.edu). Our internal practices were appraised and certified at Capability Maturity Level 3 by the SEI standards - ISDC being one of the very few organizations in Romania to achieve this level.
Quality Assurance is a constant concern in ISDC. Beside continuous improvement of the defined processes and tools there are dedicated sessions on "Continuous Improvement" activities for employees and QA Officers are having regular (formal and informal) discussions with everyone having questions on this area. Our external consultant visits us at least once a year and training sessions are organized for anyone interested.
Although the main development methodology is Scrum there are processes defined and used for the entire project lifecycle! To name a few: project planning; risk management; requirements development and management, release and configuration management, quality management and so on. By doing this we reduce the probability of small cracks to appear in our final product by skipping an apparently minor task that backfire in the worst possible moment. That probability still exists - but it is up to us to make sure we did as much as possible to keep it at the lowest level.
CMMI® for Development, version 1.3 - Improving processes for developing better products and services, November 2010,Technical report
http://www.qpit.net/blog.html - a series of articles by Peter Leeson, CMMI and Process Improvement Lead Appraiser and Instructor at Q:PIT Ltd.(http://www.qpit.net/contact-us.html)
www.qpit.net/blog/understanding-quality.html ↩
http://www.qpit.net/blog/getting-started-101-process-agile-or-lean.html ↩
by Diana Muntea
