Under various engineer titles, veteran engineer Demetri worked for Expert Engineering, Inc., for almost 15 years. The firm’s performance evaluation history is both unique and long. He has recently been promoted to the position of Principal at the engineering firm. All principals are involved in evaluating engineers because the founders of the firm believed in multiple source evaluation and feedback to prevent favoritism and promote a merit-based culture. At the same time, the firm has a long history of using quality performance appraisal forms and review meetings to better ensure accu- rate performance evaluations. Several months ago, however, the firm initiated a big hiring initiative of a dozen new engi- neers, nine of whom turned out to be graduates from Purdue
University, which is the same university from which Demetri graduated. Indeed, Demetri was active in moving forward the hiring initiative. There is tension and discontent among the other principals, who fear that a time of unchecked favoritism, biased performance ratings, and unfair promotion decisions is on the rise.
1. Provide a detailed discussion of the intentional and unintentional rating distortion factors that may come into play in this situation.
2. Evaluate the kinds of training programs that could minimize the factors you have described. What do you recommend and why? !
End Notes
1. Tannenbaum, S. I. (2006). Applied perform- ance measurement: Practical issues and challenges. In W. Bennett, C. E. Lance, & D. J. Woehr (Eds.), Performance measure-
ment: Current perspectives and future challenges (pp. 297–318). Mahwah, NJ: Lawrence Erlbaum.
2. Adapted from Grote, D. (1996). The complete
guide to performance appraisal. New York: AMACOM.
3. Grote, D. (1996). The complete guide to per-
formance appraisal.New York: AMACOM. 4. Adapted from Workforce Research Center
(2003, April). “Busch Performance Evalua- tions,” Workforce Online. Available online at http://www.workforce.com/archive/article/ 23/42/03.php. Retrieval date: May 1, 2011. 5. Ganzach, Y., Kluger, A. N., & Klayman, N.
(2000). Making decisions from an interview: Expert measurement and mechanical com- bination. Personnel Psychology, 53, 1–20. 6. Kraiger, K., & Aguinis, H. (2001). Training
effectiveness: Assessing training needs, motivation, and accomplishments. In M. London (Ed.), How people evaluate others in
organizations (pp. 203–220). Mahwah, NJ: Lawrence Erlbaum.
7. Brutus, S. (2010). Words versus numbers: A theoretical exploration of giving and receiving narrative comments in perform- ance appraisal. Human Resource Management
Review, 20, 144–157.
8. Short, J. C., Broberg, J. C., Cogliser, C. C., & Brigham, K. (2010). Construct validation using computer-aided text analysis (CATA): An illustration using entrepreneurial orientation.
Organizational Research Methods, 13, 320–347. 9. Milliman, J. F., Zawacki, R. A., Schulz, B., Wiggins, S., & Norman, C. A. (1995). Customer service drives 360-degree goal setting.
Personnel Journal, 74, 136–142.
10. Workforce Research Center. ED: April 29, 2001 The new thinking in performance appraisals: Writing effective co-worker comments.
Workforce Online. Available online at http:/ /www.workforce.com/archive/feature/22/ 28/68/223579.php. Retrieval date: May 1, 2011.
11. Adapted from Grote, D. (1996). The complete
guide to performance appraisal. New York: AMACOM.
12. Talbott, S. P. (1994). Peer review drives com- pensation at Johnsonville. Personnel Journal,
13. Courter, E. (2006). Measuring up. Credit
Union Management, 29(6), 30–33.
14. The discussion of the various sources of performance information is based, in part, on Cascio, W. F., and Aguinis, H. (2005). Applied
psychology in human resources management, (6th ed.). Upper Saddle River, NJ: Prentice Hall. 15. Greguras, G. J. (2005). Managerial experi-
ence and the measurement equivalence of performance ratings. Journal of Business and
Psychology, 19, 383–397.
16. Abu-Doleh, J., & Weir, D. (2007). Dimensions of performance appraisal sys- tems in Jordanian private and public organ- izations. International Journal of Human
Resource Management, 18, 75–84.
17. Hogan, R., & Shelton, D. (1998). A socioana- lytic perspective on job performance.
Human Performance, 11, 129–144.
18. Workforce Research Center. Dear workforce: ED: March 6, 2002. How are peer reviews used for compensation? Workforce Online. Available online at http://www.workforce. com/archive/article/22/13/94.php.
Retrieval date: May 1, 2011.
19. Ramsey, W., & Owen, C. (2006). Is there a role for peer review in performance appraisal of medical students? Medical Education, 40(2), 95–96.
20. Dierdoff, E. C., & Surface, E. A. (2007). Placing peer ratings in context: Systematic influences beyond rate performance.
Personnel Psychology, 60, 93–126.
21. Antonioni, D. (1994). The effects of feedback accountability on upward appraisal ratings.
Personnel Psychology, 47, 349–356.
22. Jelley, R. B., & Goffin, R. D. (2001). Can per- formance-feedback accuracy be improved? Effects of rater priming and rating-scale for- mat on rating accuracy. Journal of Applied
Psychology, 86, 134–145.
23. Useem, J. (2005). America’s most admired companies. Fortune, 151(5), 66–70.
24. Serwer, A. (2005). The education of Michael Dell. Fortune, 151(5), 76.
25. Greguras, G. J., Robie, C., Schleicher, D. J., & Goff, M. (2003). A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, 56, 1–21. 26. Atkins, P. W. B., & Wood, R. E. (2002). Self-
versus others’ ratings as predictors of
assessment center ratings: Validation evi- dence for 360-degree feedback programs.
Personnel Psychology, 55, 871–904.
27. van Hooft, E. A. J., van der Flier, H., & Minne, M. R. (2006). Construct validity of multi-source performance ratings: An exami- nation of the relationship of self-, supervisor-, and peer-ratings with cognitive and person- ality measures. International Journal of
Selection & Assessment, 14, 67–81.
28. Wells, B., & Spinks, N. (1990). How companies are using employee self-evaluation forms.
Journal of Compensation & Benefits, 6, 42–47. 29. Simmons, J., & Lovegrove, I. (2005). Bridging
the conceptual divide: Lessons from stake- holder analysis. Journal of Organizational
Change Management, 18, 495–513.
30. Milliman, J. F., Zawacki, R. A., Schulz, B., Wiggins, S., & Norman, C. A. (1995). Customer service drives 360-degree goal setting. Personnel Journal, 74, 136–142. 31. Franke, G. R., Murphy, J. H., & Nadler, S.
S. (2003). Appraising account executive performance appraisals: Current practices and managerial implications. Journal of
Current Issues & Research in Advertising, 25, 1–11.
32. Facteau, J. D., & Craig, S. B. (2001). Are per- formance appraisal ratings from different rating sources comparable? Journal of
Applied Psychology, 86, 215–227.
33. Brutus, S., Petosa, S., & Aucoin, E. (2005). Who will evaluate me? Rater selection in multi- source assessment contexts. International
Journal of Selection and Assessment, 13, 129–138. 34. Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2002). The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct- level convergence and rating difficulty.
Journal of Applied Psychology, 87, 345–354. 35. Jaramillo, F., Carrillat, F. A., & Locander, W. B.
(2005). A meta-analytic comparison of mana- gerial ratings and self-evaluations. Journal of
Personal Selling & Sales Management, 25, 315–328.
36. Murphy, K. R., & Cleveland, J. N. (2005).
Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
Chapter 6 • Gathering Performance Information 167 37. Longenecker, C. O., Sims, H. P., & Gioia, D. A.
(1987). Behind the mask: The politics of em- ployee appraisal. Academy of Management
Executive, 1, 183–193.
38. Murphy, K. R. (2008). Perspectives on the relationship between job performance and ratings of job performance. Industrial and
Organizational Psychology, 1, 197–205. 39. Spence, J. R., & Keeping, L. M. (2010). The
impact of non-performance information on ratings of job performance: A policy-captur- ing approach. Journal of Organizational
Behavior, 31, 587–608.
40. Fletcher, C. (2001). Performance appraisal and management: The developing research agenda. Journal of Occupational and Organi-
zational Psychology, 74, 473–487.
41. Murphy, K. R., Cleveland, J. N., Skattebo, A. L., & Kinney, T. B. (2004). Raters who pursue different goals give different ratings. Journal
of Applied Psychology, 89, 158–164.
42. Murphy, K. R. (2008). Explaining the weak relationship between job perform- ance and ratings of job performance.
Industrial and Organizational Psychology, 1, 148–160.
43. Curtis, A. B., Harvey, R. D., & Ravden, D. (2005). Sources of political distortions in per- formance appraisals: Appraisal purpose and rater accountability. Group & Organization
Management, 30, 42–60.
44. Mero, N. P., Guidice, R. M., & Brownlee, A. L. (2007). Accountability in a performance appraisal context: The effect of audience and form of accounting on rater response and behavior. Journal of Management, 33, 223–252.
45. Wang, X. M., Wong, K. F. E., & Kwong, J. Y. Y. (2010). The roles of rater goals and ratee per- formance levels in the distortion of perform- ance ratings. Journal of Applied Psychology, 95, 546–561.
168