In a general, as a widely accepted definition, quality of a product is the main characteristic that differentiates that aforementioned product in relation to other, similar products available on the market. Quality implies meeting the customer's expectations, at a cost that the customer affords (or: agrees) to pay, so that we, as a provider of goods or services, will be able to do this the next day and the day after that. A bug-free product is not necessarily a good-quality product.
Quality is as good as its perception by the end-users, as good as meeting the end-users' expectations. Of course, a bug-free product is more likely to be perceived as a good one, but that's just part of the whole picture. This is an important part, no one could argue about, but just one part of the picture. "Quality Assurance" is the set of activities aiming to build quality into our final product. It consist of preventive and corrective actions that ensure us and our customers that the product we deliver meets their expectations. Customers' expectations derive from the intended use of our product and from various legal or industry standards.
Challenges encountered in automotive embedded software development are different (to say the least) than the challenges faced in different areas of software development: in the automotive industry hardware and software go hand in hand, one cannot exist without the other. In most cases, the hardware is different from one vendor to another; hardware resources are also very limited; the developed systems are required to work in real time; updates or patches are not-so-easy to deploy, while the life expectancy (in conditions of difficult -almost close to zero- maintenance capabilities) is counted in tens of years. While a malfunction in a (let's say) online retail backend system can cause some orders to be lost, a malfunction in the ESP, Engine control unit or the airbag in a speeding car can lead to injury or worse. Communication between the different departments, which are building such products, is essential for a project to go in the right direction, so it is very important for all contributors to speak the same language. Furthermore, a project in the automotive industry is more than software development: the hardware and the mechanical parts of the system (be it a speedometer, ABS or a docking system for your smartphone) are parts of the same system, sold as one product. Teams developing different parts of a system are really distributed across different continents, although they're sometimes in the same group.
Building quality into automotive products is a big challenge. It means aligning customer's requirements to the teams spread literally across the world, while meeting legal or industry (mandatory!) standards. Luckily we, software engineers, can benefit from the experience gained by the car manufacturing industry in its 100-year old history - that's almost 4 times more than the history of the software industry (since it "escaped" from the university laboratories and gradually became a modern commodity). Car manufacturers used production processes for the first time and they were among the pioneers of automation. In addition, car manufacturers faced legal regulations or safety requirements long before the software industry took off!
After this rather long introduction on what quality is and why quality should be built (embedded) into our product, let's see how this could be done. Let's see how automotive embedded software developers can benefit from the experience of the automobile industry, as it was gathered throughout the last century.
Quality Management systems in automotive projects follow a process-based approach. There are three industry standards that stay at the foundation of this:
ISO/TS 16949 -Quality management systems—Particular requirements for the application of ISO 9001:2008 for automotive production and relevant service part organizations;
CMMI® -Capability Maturity Model Integration;
Based on these three pillars and taking into consideration other industry standards such as ISO/IEC 9001 or V-Model and countless of formalized customers' (or: group of customers') requirements, an organization trying to enter this market should develop an internal quality management system that will provide strong arguments, when competing against other organizations in a bid for a new project. Having a formalized framework for performing daily routines (in all business areas) will create the opportunity to learn from the past, will improve estimations and speed-up the learning process, thus decreasing the overall costs of a project, and will provide a competitive advantage over others. Everyone will know the tasks that should be performed (at the exact moment in the project lifecycle), will know exactly what information or input should be available to perform the tasks, and what the expected outcome of their activity is. Measurements can be derived and then used as a baseline for further improvement. Evidence can be presented to an existing or possible customer, showing them tangible information that their money is/will be well invested and that we, as the service provider, have a mindset which is oriented towards quality from the early stages of the product lifecycle, in all areas that cooperate to bring an idea to reality.
Both SPICE and CMMI models group the daily activities within an organization into process areas such an engineering processes (development, requirements engineering, change management), management processes (project management, performance measurement) and organizational/support processes (process management, organizational training management, supplier management). Each process is described in detail, with the expected input and outcome. When it comes to processes, an important characteristic is maturity, describing how well a process is performed, based on how well (or not, that is) various process attributes are met. There are 6 levels of process maturity defined in the models (from level 0 - incomplete- to level 3 -defined, established processes and going to level 5 - innovative / optimized processes).
Compliance with CMMI, ISO models and the maturity level can be certified by specialized organizations or persons, agreed by the body that maintains the model (SEI or ISO), following a certification process. The certification should be renewed at regular intervals. On the other hand, the A-SPICE model does not have an organization behind it, although it is a standard used by most of the suppliers in automobile industry. A certification of compliance of an internal quality management system against A-SPICE model could not be obtained! The practice in this industry is that every car manufacturer requires, by contract, a certain level of maturity for every product they buy. An organization that aims to qualify as a supplier in the automobile industry must be capable to prove to its client, ad-hoc or at regular intervals, that it is performing an A-SPICE compliant process (in all process areas) at the maturity level required by the contract.
Of course, a production process is not to be frozen and used mindlessly over and over again. It should be a living organism that will constantly evolve as more and more experience is gained performing it. Any organization should invest in developing such production processes and create the environment that will allow the production process to grow.
Having a set of work processes defined at the organizational level is only half of a Quality Management System. These guidelines are to be put into practice at the project level, where the work of building a product is taking place on a daily basis. If not properly done, this approach will sometimes backfire and will impede the project as a whole, by unnecessary delays or by creating unnecessary tensions within the team, between the team and the organization or (in the worst case scenario) between the team and the customer. The project team must be aware that investing a small amount of time or other resources in preemptive actions, from the early stages of a project lifecycle, is better than fixing bugs found by the customer in the product, upon completing or delivering the product. A day spent making sure the customer's requirements are clear will save two or more days testing or repairing a bug under the pressure of a deadline. The same goes for documenting a decision that was debated for two hours over a call with the customer or making sure the developer understands the architect's design or making sure the tester understands the functionality to be tested. And the list could go on.
Defining and maintaining a process-based quality management system is not an easy endeavor. Feedback should be collected from projects that are actually using the defined process and compliance or conformance checks should be made from time to time so that process definition can be improved. Furthermore, project teams need support in instantiating the organizational process definitions; they need support in establishing feasible quality goals and tailoring the organizational processes to their needs in a manner that will guarantee both the conformance (to a proven successful way of doing things) and the freedom to innovate or find a better way to do it. No compromises should be made in terms of the quality of the final product.
The person which stands in the middle of this situation is the Software Quality Engineer. Shortly SWQE. They know the process' definitions and the daily routine of a team. They are responsible for replicating the organization's defined processes in the project's ecosystem. Yet, they are not members of the team! There's a different branch in the organizational chart for the persons performing this role and this ensures their independence in and from the project. However, this also limits their capability to stop some bad practices / habits within the team. I am not saying that this is not possible, it all depends on their communication skills - a team-lead or project manager can be convinced or persuaded to change something within a project, assuming that the right arguments are provided, backed-up by numbers or well-chosen examples. Another responsibility is to perform various measurements required by the quality management system and report to the project and quality manager, and, sometimes, if it is agreed, to the customer as well. SWQEs' work covers all aspects of the project, as there are processes defined for every member of a team, from project manager to discipline managers, requirement engineers, developers, testers, integrators and system testers, at any seniority level. The SWQE is not a police officer, but an advisor that helps the project team to keep the focus on delivering quality in every stage of the project lifecycle. The SWQE is also in the first line when the customer asks for proof that the team performs its work in compliance to the agreed level of maturity of the A-SPICE model. This person is the first person responsible, who is also asked for solutions, when non-conformities are identified.
Quality Management (and quality focus) from the very early stages of product development is not to be overlooked, not in today's world. The software industry in Cluj evolved from the early days of outsourcing to taking full responsibility of an entire project (management, requirements engineering, full testing at all levels). Embedded software development followed the same path. We proved ourselves to be valuable partners to major industry players and this enabled us to learn from the best. Our customers are demanding top quality and we should be able to prove it. More and more organizations define internal quality management systems and get certified against one model or the other. Organizations which are active in the embedded software industry should not be different. This also holds true since, in this case, customers auditing individual projects or teams represents the rule, not the exception!
On a personal level, should you hear about a job opening in Quality Assurance, you are now prepared to ask the right questions while talking to a recruiter. Is there a Quality Management System defined within the organization? What standard or model was/will be used to define it? How early in the project lifecycle is quality taken into consideration? And this will break the ice, I can guarantee it!
Peter Leeson, http://www.qpit.net/blog.html
www.automotivespice.com/fileadmin/software.../Automotive_SPICE_PAM_30.pdf