Page 227 DETERMINING WHETHER OUTCOMES ARE APPROPRIATE An important issue in choosing outcomes is to determine whether they are appropriate. That is, are these outcomes the best ones to measure to determine whether the training program is effective? Appropriate training outcomes need to be relevant, reliable, discriminative, and practical.22 Relevance Criteria relevance refers to the extent to which training outcomes are related to the learned capabilities emphasized in the training program. The learned capabilities required to succeed in the training program should be the same as those required to be successful on the job. The outcomes collected in training should be as similar as possible to what trainees learned in the program. That is, the outcomes need to be valid measures of learning. One way to ensure the relevancy of the outcomes is to choose outcomes based on the learning objectives for the program. Recall from Chapter 4 that the learning objectives show the expected action, the conditions under which the trainee is to perform, and the level or standard of performance. Figure 6.2 shows two ways that training outcomes may lack relevance. Criterion contamination refers to the extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions. For example, if managers’ evaluations of job performance are used as a training outcome, trainees may receive higher ratings of job performance simply because the managers know they attended the training program, believe the program is valuable, and therefore give high ratings to ensure that the training looks like it positively affects performance. Criteria may also be contaminated if the conditions under which the outcomes are measured vary from the learning environment. That is, trainees may be asked to perform their learned capabilities using equipment, time constraints, or physical working conditions that are not similar to those in the learning environment. For example, trainees may be asked to demonstrate spreadsheet skills using a newer version of spreadsheet software than they used in the training program. This demonstration likely will result in no changes in their spreadsheet skills from pretraining levels. In this case, poor-quality training is not the cause for the lack of change in their spreadsheet skills. Trainees may have learned the necessary spreadsheet skills, but the environment for the evaluation differs substantially from the learning environment, so no change in skill level is observed. Page 228
FIGURE 6.2 Criterion Deficiency, Relevance, and Contamination Criteria may also be deficient. Criterion deficiency refers to the failure to measure training outcomes that were emphasized in the training objectives. For example, the objectives of a spreadsheet skills training program emphasize that trainees both understand the commands available on the spreadsheet (e.g., compute) and use the spreadsheet to calculate statistics using a data set. An evaluation design that uses
You've reached the end of your free preview.
Want to read all 189 pages?
- Spring '14
- Human Resource Management, Virtual learning environment