Training for Quality Management
5.3 Two Examples
The most difficult training aspect for quality management is how best to approach the training for quality evaluations. Because the scope, audience, and functions of quality evaluation cover a considerable range of activities, we offer two examples at the ends of this range.
5.3.1 Evaluation of Adherence to Process (PPQA)
An example of quality evaluation comes from the Process and Product Quality Assurance (PPQA) process area from the CMMI® [1]. This process area focuses on evaluating work products for adherence to standards, and evaluations of the
performance of the various process areas within the model for compliance with their process descriptions. The requirement to evaluate a process area for compli- ance with its process descriptions is specified in Generic Practice (GP) 2.9 for each process area. For purposes of this discussion, suffice it to say that the PPQA process area is the umbrella set of requirements for performing quality evaluations, and what evaluations to perform are defined in GP 2.9 for each process area. See Chap- ter 11 for a further discussion of the PPQA process area. (For those who are inter- ested in a more detailed discussion of the CMMI® structure, a discussion of the structure of the model in terms of process areas, specific and generic practices, and the like, can be found in [1].)
Within the PPQA process area, there is another generic practice, GP 2.2, which calls for a plan for performing the process and product quality evaluations. In our discussion of the needed skills and knowledge, we pointed out the necessity for training in writing plans. We also noted the importance for training in the appropri- ate application domains. Both are important for producing a good plan for the PPQA process. The process areas of the CMMI®are diverse, running the gamut from project management activities, to engineering, to process management, and various support functions such as configuration management, measurement and analysis, and so on. Knowing how to write a plan that adequately addresses the ability to perform process and product quality evaluations for such a diverse set of activities can only be accomplished by training the lead evaluators in writing plans of this nature.
Training in domain knowledge comes into play here in that establishing the pro- cedures for determining compliance with the process descriptions for the process areas in question require at least a high level knowledge of the domain to determine if the process execution is in compliance. For example, for the Technical Solution process area, if agile methods are the process being implemented, the quality evalua- tor must have some knowledge of these methods; otherwise, the quality evaluation becomes a checklist activity reliant on asking the process performers if they are fol- lowing the process. As one can imagine, the ability to be fooled is quite high and the value added from such an evaluation is quite low. Obviously, training in how to write both plans and procedures for quality evaluation is necessary, along with the need for domain-specific training. Any constraints or limitations imposed on the process in question to be evaluated must be known as well, otherwise the evaluations will yield incorrect results.
Training in how to do the process evaluations is also important. It is one thing to be trained in writing quality evaluation plans and procedures, but if one does not know how to implement the procedures, then the value of the training in writing plans and procedures is diminished. How to do process evaluations will be a func- tion of the kind of process evaluated. The evaluation of a peer review process for compliance is different from the evaluation of a testing process. This, in turn, is dif- ferent from the evaluation of a design process that will clearly depend on the design methodology being implemented. In software, evaluations of a design process being implemented using agile methods will require different knowledge than evalua- tion of a design process using object-oriented techniques. Clearly, there is an inter- action with domain knowledge and basic engineering methodology skills. One
might easily (and correctly) assume that certain evaluations should be performed by personnel trained in the skills necessary to perform the processes in question.
Superimposed on these training needs is the need to be trained in goals and objectives of the project and the organization. These provide a context for the evalu- ations to be performed. In addition to the constraints and limitations imposed by cli- ent and industry standards and regulations, the organization’s and project’s business needs provide an additional set of constraints and limitations.
Finally, the ability to specify quality evaluation criteria also comes into play. If these are incorrectly specified, processes that do not comply with governing process descriptions may be evaluated as adequate when they are not. One can clearly see the importance of training in this area.
5.3.2 Evaluation of Product Quality
As noted earlier, if we use the CMMI®as a reference point, the PPQA process area addresses both process and product quality evaluations. In the previous example, we discussed the need for training in writing plans and procedures, and the need for training in the domains of interest. We also discussed the need to be trained in the constraints and limitations applicable to the domain of interest. These needs apply as well for training in product evaluation.
In the CMMI®PPQA process area, evaluations of product quality are defined as evaluations for compliance with governing standards. One can easily conclude from this that the focus is on compliance with specified templates and formats. In the defense industry, this conceivably could be considered compliance with Data Item Descriptions (DID) imposed by contract. In performing such evaluations, the evalu- ator looks to see if all the specified content has been provided and if the format of the document is correct. Knowing if the proper content has been provided requires at least some domain-specific training. On the other hand, knowing if the work prod- uct is in the appropriate format requires very little training if the product is a docu- ment. A template, or a DID, along with a bona fide sample from a project in the Process Asset Library, provides sufficient information for performing a format com- pliance evaluation.
On the other hand, if the work product is an item of software, and format requirements (e.g., coding standards) have been specified, the quality evaluator must know how to read source code in the specified coding language, requiring either that such training be provided or that an evaluator familiar with that coding language be used for performing the evaluation. While this can often be accom- plished by software tools, there are times when organizations specify unique coding standards. The quality evaluator should at least be able to determine if the output of the software tool (if one is used) will adequately determination violations of the coding standards.
There are some training needs that are unique to the evaluation of products. Pre- viously, we discussed the need for training in writing requirements. If a quality eval- uator is responsible for evaluating the content of a requirements specification, clearly, that person must know how to write requirements for the various types of requirements that are to be included in the specification, the subject of training we have previously discussed. In the case of evaluations of requirements specifications,
two separate evaluations may take place: one for the proper statement of the requirements and another for format and content of the requirements specification. These requirements become the requirements for performing validation of the prod- uct. Being able to assess if the requirements are testable, as written, requires training in writing requirements, as we previously discussed. Training in the ability to write test plans, test cases, and test procedures follows from this.
5.4
Summary
In setting the context for quality assurance training we considered the entire quality evaluation aspect of a Quality Program. The rationale is that quality evaluation is broader in scope and typically covers more functions than what have been perceived as the “traditional” quality assurance functions, depending on the organizations concept of quality assurance. A further analysis of this idea is covered in Chapter 13.
Too frequently, the role of quality evaluation has been perceived as not requir- ing much in the way of training. Mentoring has often been used as a method of training. Mentoring, in many cases, has meant pairing a new quality evaluator with an experienced one. In many cases, mentoring may be an adequate method of train- ing; in other cases, not. All too often, the so-called experienced quality evaluator is lacking in the knowledge to perform the role properly, and he or she passes down information that is inadequate to the new quality evaluator. In other cases, the men- tor may have biases that should not be passed on to the new evaluator.
Other mistaken approaches have often been applied. Training in how to do peer reviews has sometimes relied on an implicit requirement for the participants to read the procedures and then know how to perform their roles in peer reviews on the basis of their reading. In addition, many companies do not provide training in how to do testing or how to construct a test program. While information on how to con- duct various aspects of a testing program can be found from public seminars, instruction on how to construct a total, comprehensive test program, say, beginning with bench testing and ending in a full-blown qualification or acceptance test, is lacking.
In many cases, the situation existing in quality evaluation has resulted from a lack of understanding or appreciation for the importance of the quality evaluation role. In this chapter, we have provided some context to the requirements for and scope of an effective quality evaluation program.
Reference
[1] Chrissis, M. B., M. Konrad, and S. Shrum, CMMI®
: Guidelines for Process Integration and Product Improvement, 2nd ed., Reading, MA: Addison-Wesley, 2006.
C H A P T E R 6