360 BY DESIGN Facilitator s Guide

(1)

360 BY DESIGN

^®

Facilitator’s Guide

Center for Creative Leadership, its logo, 360 BY DESIGN®, Benchmarks®, SKILLSCOPE®, LeaderLab®, and Leadership Development Program (LDP)® are registered trademarks owned by the Center for Creative Leadership. Prospector® is a registered trademark owned by Morgan W. McCall, Jr., Gretchen M. Spreitzer, and Joan Mahoney.

(2)

(3)

________________________________________________________________________

Section 1: Purposes and Uses of 360 BY DESIGN

^®

Introduction

Purpose of This Guide

What 360 BY DESIGN Measures Uses of 360 BY DESIGN

Benefits of 360 BY DESIGN Overview of Content

Overview of Process Terms Used

1.1 Introduction

The Center for Creative Leadership^® (CCL^®) is a pioneer in using 360-degree feedback to enhance leadership development. Each year, CCL processes 360-degree instruments for more than 50,000 leaders around the world. Through CCL’s extensive assessment-for- development research, as well as direct feedback from participants in its leadership

development programs, it has been confirmed time and time again that this type of feedback has a powerful impact on individual leaders. 360 BY DESIGN is an innovative assessment tool developed by CCL to help organizations customize an Internet-based 360-degree- feedback instrument. Using this tool, organizations can select from 94 competencies and 5 derailment factors—a total of more than 600 questions--drawn from existing validated CCL instruments. 360 BY DESIGN enables an organization to provide individuals with feedback on areas they believe are most important to success. The resulting feedback is intended to support managers in their professional development.

1.2 Purpose of This Guide

The purpose of this Guide is to provide you with the information and materials needed to administer a 360 BY DESIGN assessment, lead a successful Feedback Workshop, and facilitate one-to-one feedback sessions with participants. The Guide is not designed to comprehensively teach a facilitator how to give feedback on any psychometric instrument. It is designed to be easy to use and includes the pertinent information about the research and psychometric basis for the instrument. The Guide consists of eight sections. Section 2 covers the design and management of your feedback process, including contracting and services from CCL, building your instrument, electronic administration, and program components that will ensure a successful delivery. The remaining sections cover how to deliver 360 BY DESIGN feedback (Section 3), how to assist participants in using their feedback data in planning for development (Section 4), and the research background for the development of the 360 BY DESIGN (Section 5).

Readings and references are listed in Section 6. Section 7 includes frequently asked questions about the survey, the process, the feedback data, and how the data will be used. The final

(6)

________________________________________________________________________

CCL conducts ongoing research for all of its assessment instruments, including 360 BY DESIGN.

This Facilitator’s Guide will be updated periodically as new research is released. All revised materials will be made available to certified 360 BY DESIGN facilitators.

1.3 What 360 BY DESIGN Measures

The areas assessed by 360 BY DESIGN are leadership skills, perspectives, and capacities of effectiveness seen as important in organizations. Because each organization selects the specific competencies to be included in its survey, the resulting data are highly relevant and useful, both to individuals and to the overall organization.

360 BY DESIGN provides meaningful feedback that participants can use to create a personal development plan. Specifically, it:

• provides perspectives on how others view the participant and how the participant’s behavior impacts those other individuals

• can increase the participant’s self-awareness, which is a cornerstone of development

• can help the participant confirm known strengths and identify unrecognized strengths

• can uncover development needs, including ones of which the participant may not be aware.

360 BY DESIGN is not a tool for measuring basic skills, job knowledge, or intellectual ability. For this reason, it is not appropriate for use in making selection decisions or as a part of performance appraisals or compensation reviews

1.4 Uses of 360 BY DESIGN

360 BY DESIGN may be used for two general purposes:

• for individuals, as a confidential assessment instrument that identifies strengths and areas for development

• for groups, as a profiling tool for a work group to target development needs and to shape environments that support individual and group development.

(7)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 6 With 360 BY DESIGN, as with all of CCL’s psychometric instruments, confidentiality of individual feedback data must be ensured and safeguarded. The anonymity of individual respondents must be maintained, except where specifically noted and agreed to at the time the instrument is completed (e.g., Boss, Superior, Other data).

Although the instrument may be introduced in a group setting, we recommend that the participant discuss feedback and development planning in a one-to-one feedback session.

Individual participants own their feedback data and must be given exclusive possession of all copies of their feedback reports, unless explicitly requested or agreed to differently by the individual.

1.5 Benefits of 360 BY DESIGN

One of the most unique benefits of using 360 BY DESIGN is the ability to deliver a customized assessment and feedback process. Many organizations today are developing and using custom competency models of effective leadership. By choosing the areas for assessment, organizations can measure what is important to them. Individuals have the opportunity to receive feedback that is highly relevant to their current and future work challenges.

Because the client organization creates the instrument, the survey:

• may be used with both individuals and groups

• may be used for any level of manager or executive

• may be taken at any point in an individual’s career

• can be built to fit the specific needs of that organization.

Moreover, customization is achieved without sacrificing the research that assures valid and reliable competencies on which feedback will be provided. There are more opportunities for customizing a 360 BY DESIGN survey. In addition to selecting competencies to be included in the survey, choosing one or both standard open-ended questions, and adding custom questions, the organization may choose to customize one or more of the following elements:

• names of the standard set of rater categories

• standard set of email notifications sent to participants and raters

Another key benefit of 360 BY DESIGN is that it is an entirely Internet-based process. Not only is turnaround time for the receipt, scoring, and delivery of data shortened, but also both participants and their raters can provide input from virtually anywhere in the world, at any time of day or night.

In addition to being highly flexible and time-efficient, 360 BY DESIGN is also a secure process with respect to the handling and reporting of data. CCL provides a secure server, firewall technology, passcode access, and encrypted transmissions to ensure the security and anonymity of raters and the confidentiality of individual results.

(8)

________________________________________________________________________

360 BY DESIGN features an extensive library of reliable and valid competencies. These 94 research-grounded competencies and 5 derailment factors were developed by the Center for Creative Leadership and are used extensively in leadership development training programs and related coaching activities. The number of items (questions) for each competency is displayed in parentheses below. The competencies are grouped into conceptual categories for purposes of clarity and organization. “Problems That Can Stall a Career” (derailment factors) are reported separately in the feedback report.

Competencies

LEADING OTHERS

Managing Effective Teams and Work Groups

• Brings Out the Best in People (5)

• Forging Synergy (6)

Building and Maintaining Relationships

• Managing Conflict; Negotiation (3)

• Relationships (10)

• Building and Mending Relationships (11)

• Putting People at Ease (4)

• Compassion and Sensitivity (7)

Valuing Diversity and Difference

• Differences Matter (6)

• Leveraging Differences (5)

• Global Awareness (5)

• Valuing Diversity (6)

• Adapts to Cultural Differences (6)

Developing Others

• Confronting Problem Employees (6)

• Leading Employees (14)

• Inspiring Commitment (5)

• Employee Development (4)

(9)

________________________________________________________________________

• Selecting, Developing, Accepting People (7)

• Developing and Empowering (6)

• Delegating (4)

• Motivating Others (4)

Communicating Effectively

• Communicating Information, Ideas (5)

• Communicating Effectively (6)

• Listening (4)

• Participative Management (10)

• Communication (5)

Problems That Can Stall a Career

• Problems with Interpersonal Relationships (10)

• Difficulty Building and Leading a Team (8)

LEADING THE ORGANIZATION Managing Change

• Leading Change (5)

• Change Management (9)

Solving Problems and Making Decisions

• Insightful: Sees Things from New Angles (4)

• Getting Information, Making Sense of It; Problem Identification (7)

• Sound Judgment (6)

• Problem Solving/Decision Making (4)

• Decisiveness (4)

• Recognizing Trade-Offs (7)

• Taking Action, Making Decisions, Following Through (5)

• Resourcefulness (10)

Managing Politics and Influencing Others

• Influencing, Leadership, Power (9)

Taking Risks and Innovating

(10)

________________________________________________________________________

• Has the Courage to Take Risks (4)

• Risk-taking, Innovation (5)

• Courage (5)

Setting Vision and Strategy

• Planning and Goal Setting (4)

• Strategic Planning (7)

• Vision (4)

Managing the Work

• Administrative/Organizational Ability (9)

• Being a Quick Study (4)

• Results Orientation (6)

• Business Perspective (5)

• Organizing (4)

• Knowledge of Job, Business (6)

• Knowledge: Trade and Business (4)

• Seeks Broad Business Knowledge (4)

Enhancing Business Skills & Knowledge

• Customer/Vendor Relations (4)

• Financial Management (6)

• Marketing (4)

• Human Resources (4)

• Sales (4)

Understanding and Navigating the Organization

• Acting Systemically (5)

• Too Narrow a Functional Orientation (5)

LEADING YOURSELF Developing Adaptability

• Interpersonal Savvy (7)

(11)

________________________________________________________________________

• Openness to Influence; Flexibility (9)

• Adaptability (5)

• Embracing Flexibility (6)

Increasing Self-Awareness

• Self-Awareness (4)

• Seeks and Uses Feedback (5)

• Self-Management, Self-Insight, Self-Development (7)

• Open to Criticism (3)

Managing Yourself

• Career Management (9)

• Balance Between Personal Life and Work (4)

• Handling Disequilibrium (9)

• Time Management (4)

• Coping with Pressure and Adversity; Integrity (8)

• Straightforwardness and Composure (4)

Increasing Your Capacity to Learn

• Seeks Opportunities to Learn (5)

• Learning through Others (11)

• Learns from Mistakes (5)

• Learning from Experience (5)

Exhibiting Leadership Stature

• Executive Image (5)

• Leadership Stature (4)

Displaying Drive and Purpose

• Energy, Drive, Ambition (4)

• Motivating Self (5)

• Committed to Making a Difference (4)

• Leading with Purpose (10)

• Doing Whatever It Takes (9)

(12)

________________________________________________________________________

Demonstrating Ethics and Integrity

• Credibility (8)

• Ethics/Culture (4)

• Acts with Integrity (4)

• Relationship Building (4)

• Failure to Meet Business Objectives (7)

• Difficulty Changing or Adapting (10)

GLOBAL COMPETENCIES

• Decision Maker (11)

• Negotiator (7)

• Leader (16)

• Business Knowledge (8)

• Coping (4)

• International Business (7)

• Perspective Taking (4)

• Innovator (6)

• Cultural Adaptability (5)

1.7 Overview of Process

The 360 BY DESIGN assessment process begins when an organization decides to use 360- degree feedback for development. Once the decision to use a 360-degree assessment process has been made, the organization then determines the competencies of leadership

effectiveness held as integral to the success of managers and executives within the organization.

The implementation of 360 BY DESIGN involves straightforward steps that take you from survey construction through the delivery of the final feedback reports and development planning guides.

Step I: Begin by confirming with your Information Technology staff that all participants and raters meet the technical requirements.

Step II: Contract with CCL and complete the set-up order form, which includes: identifying the competencies necessary for high performance, providing participant information,

(13)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 12 scheduling key dates, and optionally writing custom survey questions, customizing rater category labels, and customizing email notifications. CCL can provide a competency mapping session and brief stakeholders to help obtain buy-in for the 360 process.

Step III: Upon receipt of contract and set-up template, CCL will construct the customized 360 BY DESIGN system.

Step IV: Participants access the system and submit their rater list. The system sends rater invitations and access to surveys via email. CCL can deliver an orientation workshop immediately prior to implementing this step.

Step V: Participants and their raters complete the survey online. Client administrators and participants view status online of completion by rater category. The 360 BY DESIGN system sends automated reminders to keep the process on track.

Step VI: Certified feedback facilitators receive printed feedback reports from CCL to review with participants. At a later scheduled date, participants have access to view and/or print feedback reports online. CCL can provide feedback workshops and facilitator training sessions.

Step VII: Participants use their feedback results to create targeted development plans using the 360 BY DESIGN development planning guide. CCL also delivers development

planning workshops.

CCL has developed a number of tools and samples to help an organization move through the various steps. In addition, CCL provides the support of skilled Administrators

throughout implementation, to answer questions and maintain the integrity of the process with respect to accuracy and confidentiality (the steps are outlined in detail in Section 2.6 below)

For more on how an organization can assess whether or not 360-degree feedback is appropriate, please see the References and Related Readings in Section 6.

1.8 Terms Used

CCL Administrator - An individual at the Center for Creative Leadership who is identified as the client’s contact during the implementation of a survey.

Client Administrator - An individual in the client’s organization who has the primary responsibility for guiding the assessment process within the organization and monitoring the status of completed surveys.

Competency - A group of items that have logical and empirical coherence. A competency may sometimes be referred to as a dimension, scale, skill, perspective, or capacity. Each competency in 360 BY DESIGN contains between 3 and 16 items.

(14)

________________________________________________________________________

Item – An individual question in 360 BY DESIGN that describes an aspect of behavior, performance, skill, or personal characteristics.

Item Level – Referring to each question individually.

General Norms – The average scores of all people in CCL’s database.

Local Norms – The average scores of all individuals in your organization who have completed the identical survey.

Participant – Individual receiving feedback from a 360-degree assessment instrument Rater – Individual providing response on an instrument. For your survey, raters include individuals identified by the following relationships: Boss, Superiors, Peers, Direct Reports and Others. The “All Observers” group includes all the rater responses except those from the participant.

Rater Groups – Categories of raters providing responses to items on a survey. For

example, all the respondents who are your peers will be grouped into a single rater group.

The 360 BY DESIGN rater groups are Boss, Superiors, Peers, Direct Reports, and Others.

(15)

________________________________________________________________________

Section 2: Process Design and Management

Getting Ready

Program Components

Purchase, Pricing, and Billing

Building Your Organization’s Survey Use of Group Profile Feedback Implementing Your Survey Process

Completing a Survey – as a Participant or Rater Contact Information for Questions

2.1 Getting Ready

The Center’s 360 BY DESIGN assessment is positioned to meet the needs of organizations and human resource professionals who are experienced users of multi-source feedback instrumentation. Although not all of the descriptors here may characterize each client organization, the fit of the product for a client’s needs may be assessed by how many of these characteristics are present.

• You are an executive, manager, or consultant looking for a multi-rater instrument that offers attributes of strong research and psychometric foundations, the

opportunity to customize, and, at the same time, offers the efficiency of an off-the- shelf product.

• You have completed or are prepared to complete an analysis of key skills,

perspectives, behaviors, or competencies that executives in your organization want to reinforce and develop in others – things that make a difference in the leadership effectiveness of your target group.

• Most of the participants with whom you are working have received multi-source feedback previously, although it may have been through use of a paper-and-pencil survey.

• You want to create an instrument that will generate meaningful and usable data in a feedback report for each participant. You want to use the survey as a vehicle from which participants receive feedback and then create development plans that will enable them to improve their performance.

• You either have or can secure the staff and technology resources outlined in this section so that you have a successful electronic survey administration and a feedback process that delivers developmental impact and results for participants.

(16)

________________________________________________________________________

Typically, one person in the client organization assumes responsibility for managing the overall project. This person serves as a liaison to CCL on all matters related to the

distribution, collection, and scoring of the surveys and the return of the completed feedback reports to the appropriate individuals.

Requirements to Interpret and Give Feedback

Because 360 BY DESIGN is a sophisticated assessment tool and because effective

facilitation is critical to its successful use, we require that it be administered only by certified professionals. The certification for 360 BY DESIGN is incorporated into CCL’s two-day Product Certification Workshop. Individuals can register for an open-enrollment

certification workshop at CCL or organizations can arrange for a custom program to be conducted on your site. By attending a workshop, you will:

• Receive an overview of the research origins of CCL assessment tools and related ongoing research in the field of 360-degree feedback;

• Learn to read and analyze data in the 360 BY DESIGN and Benchmarks^® feedback reports and explain the data to participants;

• Practice facilitating a feedback session with other participants in the workshop;

• Learn to assist others in the developmental planning process;

• Receive a comprehensive Facilitator’s Manual and supplementary materials relating to 360 BY DESIGN and Benchmarks.

Technology/Privacy Requirements

Since 360 BY DESIGN is Internet-based, all participants and their raters need to have a private email account and access to the World Wide Web. Internet Explorer 4.0 or higher or Netscape 4.51 or higher browsers are required to complete this survey. 360 BY DESIGN uses email as the communication method for delivering instructions, with a hyper link to the opening Web page. As a result, user identifications and passwords are delivered via email.

The goal is for each rater to answer the questions honestly. This is enhanced if the rater can sit quietly in a private space to complete the on-line survey. A kiosk rather than a private office or cubicle is acceptable, if it is in a private location. Please see CCL’s On-line Privacy Policy statement (in Section 8) for more information.

Information Technology Contact

We advise that you involve your organization’s Information Technology department prior to using 360 BY DESIGN. Your Information Technology department may receive requests for assistance from participants and raters. Many questions are easily answered by referencing the Frequently Asked Questions in Section 7 of this Guide.

(17)

________________________________________________________________________

As you develop your strategy for using a multi-rater feedback instrument, you need to plan for and deliver a number of program elements. Designing and administering an effective evaluation strategy is of great importance. Selecting the specific survey content from the 360 BY DESIGN Library of Competencies is a key task. The delivery of feedback workshops, facilitator training, and ongoing support for participants who are implementing development plans require management and staff resources that may or may not be in place at the present time. CCL is able to offer additional services in these areas if you do not have internal or external staff already in place and sufficient to meet the needs. Some of these additional services are outlined below.

An evaluation strategy will measure program effectiveness, ongoing learning and

development, behavioral change, and targeted organizational impact over time. When your multi-rater feedback instrument is part of a larger organizational curriculum, CCL is able to offer a customized evaluation strategy that links your development outcomes with business objectives. Evaluation methods may consist of electronic and traditional surveys, in addition to interviews and collection of relevant organizational data. Once the evaluation strategy has been determined and implemented, CCL provides you with data analysis, data integration, reporting, and suggestions for future action planning.

CCL is able to work with you and your other educational providers so that a well-rounded, comprehensive evaluation strategy is implemented. We feel that all of the components of your curriculum should be evaluated in tandem in order to provide a link between your competencies and business objectives.

In the content selection for a 360 BY DESIGN instrument, you may experience an iterative process of mapping your organization’s competencies to the ones in CCL’s library. With expertise gained in working with past clients, CCL staff can assist in moving this process along more quickly.

CCL staff are available to deliver workshops and additional services at your site, on a CCL campus, or by telephone. These include competency mapping, feedback workshops for executives and senior managers, facilitator certification, and one-to-one feedback and coaching services delivered to a client group of executives or managers.

2.3 Purchase, Pricing, and Billing

To initiate the purchase of 360 BY DESIGN for your organization, please contact:

Client Services

Center for Creative Leadership P. O. Box 26300

Greensboro, North Carolina 27438

(18)

________________________________________________________________________

336-282-3284(fax) [email protected]

Client Services can answer your questions and direct you to experienced CCL staff who can advise you about the design of the instrument and how your choices influence the cost and use. Once you decide to proceed, CCL sends you a contract by email to initiate the process.

There are a few things you need to decide, which will be included in this contract:

• number of unique surveys you want to create and administer

• if you want to include optional features, adding custom questions, customizing names of rater categories, and/or customizing notification emails

• number of participants

• number of group profiles

You return two signed copies of the contract. CCL signs and returns one copy to you. We issue an invoice for services, which is due within thirty (30) days. Additional costs will be determined according to the pricing schedule in the contract or by a supplemental contract letter. If you cancel or reschedule the 360 BY DESIGN assessment process as described in your contract, you agree to pay the full amount of the survey set-up fee and other fees specified in the pricing schedule.

(19)

________________________________________________________________________

There are two basic fees that comprise the price for 360 BY DESIGN: a one-time fee to set-up the assessment survey and a participant fee.

Set-up Fee, which includes:

• Customized survey selected from 94 CCL research-based competencies and 5 derailment factors

• Choice of 1 or 2 standard open-ended questions (or none)

• Standard email notification set

• Standard rater categories

• Organization norms (when 30 or more participants have completed the survey; general norms for less than 30)

$2000

Additional options at set-up to:

• Customize standard email notification set

• Customize standard set of rater categories

• Customize questions scored independently from CCL competencies (open-ended or 1-5 response scale)

$1000

$2800

Participant Fee, which includes:

• Registering participants

• Tracking status and sending notification emails

• Scoring

• Printing 1 color feedback report

• Printing 1 b/w feedback report for facilitator notes

• Printing 1 development guide

• Providing access to electronic (pdf) feedback report and development guide for participant and facilitator

Number of Participants 1-25 26-50 51-100 101-250 251-500 501-1000 1000+

$225

$200

$175

$150

$135

$120

contact CCL

Group Profiles

• If customizing standard set of rater categories, add the following fee for set-up.

• If customizing questions (open-ended or 1-5 response scale), add the following fee for set-up.

$300

$900

(20)

________________________________________________________________________

All prices quoted in contracts are valid for 12 months. When CCL receives a signed contract and completed order form, we will issue an invoice for set-up and participant fees payable in 30 days of receipt of invoice.

The set-up fee is charged only once, unless changes are made to the assessment survey or associated set-up options.

Quantity discounts for the participant fee are also available with payment upfront for the total number of participants to be administered in groups over a 12-month period. CCL cannot issue refunds for unused balances.

Additional Fees

Shipping and Handling Reports Exception Fees

Exception Fees

Holding/Transferring Participants

Participant fee is invoiced at registration. No additional fees to move to another active or future administration group.

Canceling Participants

A partial fee of 60% of the participant fee applies for requests to cancel participants after CCL receives a completed order form. If a participant is registered again at a later date, full participant fees apply. No survey data will carry over from initial cancellation.

Rush to score/ship reports before scheduled deadline (participant fee also applies)

$80 per instance

Re-process a report after scheduled deadline:

• to include late data

• to reproduce with additional data

• to transfer data from one rater category to another (only with rater approval) $80 per report

Additional copies of individual feedback report

• Color: $50 per report

• Black/white: $20 per report

Changing due dates in the timeline after participant invitations are sent

• $30 per participant per request

(21)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 20 In most cases, billing for 360 BY DESIGN will take place in two steps. The client will be invoiced for the survey set-up and per-participant fees upon signing the contract. In some cases, a client will request certain exceptions as outlined above. In this case the client may be billed on a separate invoice for those fees, along with charges for additional participants and shipping feedback reports.

2.4 Building Your Organization’s Instrument

A 360 BY DESIGN survey can measure a wide range of behaviors or competencies important to the success of your organization. To customize the survey for your specific needs, you can:

Choose from 94 competencies and 5 derailment factors from across the 360 BY DESIGN library. Each competency has been tested for validity and reliability, and contains between 3 and 16 individual questions. Competencies must be used in their entirety. As a result, you can design an instrument that best addresses the behaviors and competencies you want to assess. It can be short or long, focused or general. There is a list of competencies in Section 5 of this Guide.

Include up to 10 custom questions specific to your organization. These items are listed in random order in the survey and evaluated on the same 1-5 rating system used with questions from the 360 BY DESIGN competency library.

Choose one or both standard open-ended questions. Create up to three open-ended questions. On-screen instructions remind raters that their responses to these questions are reported exactly as typed in the participant’s feedback report.

As you make decisions about survey content, we recommend that you keep in mind the look and feel of the feedback report data that will be generated from the survey. The feedback report is created to give complete, personalized feedback regarding the skills, perspectives, and capacities of effectiveness chosen by your organization. Using feedback on these dimensions, a participant can target areas of leadership and management that are likely to have the greatest positive impact on his or her career, work group, and organization. The information provided in the 360 BY DESIGN Feedback Report is highly relevant to the individual’s current and future career success.

When asking participants to tailor their behavior with feedback from the instrument, it is important that your organization pick behaviors that are relevant to the success of the organization.

To help in this process, we offer the following suggestions:

• Identify key competencies for the future. Gather information from executives, customers, and other key stakeholders on the specific behaviors they want their leaders to demonstrate now and in the future.

(22)

________________________________________________________________________

• Gain final endorsement. It is important that executives in your organization feel that this is their questionnaire, so work with them to revise, refine, and gain final approval or “buy-in.”

Selecting Competencies

After identifying key competencies for the success of your organization, the next step is selecting competencies from 360 BY DESIGN. Consider the impact of the following when building your customized questionnaire.

Consideration One: The Length and Time to Complete

There is a tension that exists when measuring human behavior – you need plenty of

examples to focus adequate attention on important competencies, but too much information can be overwhelming. Lengthy surveys take more time to complete and can be a barrier to receiving feedback. Some individuals may complete the instrument superficially just to finish it or may not return it at all if they feel it is too long.

Consider the length and time it will take your participants and their observers to complete the survey. For example, a 100-item survey will take approximately 40-50 minutes to complete whereas 50 questions may take 20-30 minutes.

There is no absolute rule on how long or short a survey should be, but the trend with our clients nowadays is for shorter (fewer than 100 items) and more focused assessments. If you feel that the survey may be too long, consider more thoroughly assessing a smaller number of topics or a fewer number of competencies. In addition, you may want to avoid

competencies that assess overlapping areas. For example, more than one CCL competency focuses on Integrity. Do the questions in one competency more precisely assess behaviors important in your organization than do those in a similar competency?

Consideration Two: Measuring Competencies Important to Your Organization

You have the option of selecting from similar competencies within 360 BY DESIGN dimensions. For example, one key managerial area, Building and Maintaining Relationships, has 5 competencies from which to choose (Managing Conflict; Compassion and Sensitivity;

Building and Mending Relationships; Putting People at Ease; Relationships). How can you best determine which to select? Answers to the following questions will help guide your choices:

• Are there questions on one competency more suitable for the level of the individuals who are participating in this initiative? Make sure to read the individual items in each competency, not just the competency descriptions.

• How does choosing one competency over another impact the length of the survey?

• Are the behaviors described in the items specific to the development needs of individuals in my organization?

(23)

________________________________________________________________________

• Are there competencies that have higher internal consistency with fewer items? (See Table 1: Reliability Estimates for 360 BY DESIGN Competencies, in Section 5.3.) Writing Items

You have the option to include additional questions in your customized survey. In order to write good items, it is important to fully comprehend the topic you wish to assess. A well- crafted question will help provide meaningful information, while a poorly crafted one will provide useless information or, worse, lead participants to inaccurate conclusions and interpretations.

We suggest the following strategy. Before you begin the item-writing process, clearly identify what dimensions you want to assess and what you want to do with the results.

Catalog representative behaviors, skills, and competencies and determine the best way to capture the information you need. This process will help you clarify your goals and make item writing easier and more beneficial.

Things You Should Do

• Be as concise as possible without losing the meaning behind the item.

• Use specific, everyday language with common meaning and interpretation.

• Create several items to measure complex phenomena.

• Write specific open-ended questions.

Things You Should Not Do

Use colloquialisms, metaphors, similes, figures of speech, foreign phrases, scientific words, or jargon.

• Be unnecessarily verbose.

• Use double negatives.

• Use inflammatory or derogatory language.

• Attempt to measure several ideas in a single question.

Relevant Tables/Charts, included in Section 5.5 of this Guide:

• 360 BY DESIGN Library of Competencies

• 360 BY DESIGN Library of Competencies with Brief Descriptions

2.5 Use of Group Profile Feedback

When you purchase 360 BY DESIGN, you may order one or more Group Profile reports.

The group report may serve as a profiling tool, used to analyze development needs and from which to develop training or team and organization development responses. A group profile

(24)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 23 summarizes the data from a group of individual feedback reports, which you specify. A minimum of five participants is required to produce a Group Profile report. Breakout groups for group profiles are defined at set-up by assigning participants to specific groups.

Additional fees apply to define breakout groups after set-up.

The group profile presents a detailed summary of the information compiled from the feedback reports of the individuals. It provides a snapshot of perceptions related to skills, perspectives, and capacities in your assessment.

The report format is similar to the 360 BY DESIGN individual feedback report in that it presents Self data in comparison to data from Others. In the Group Profile report, “Self”

represents the average (mean) of all self-report data for the group. Scores for each rater group (e.g., boss, superiors, peers, direct reports, others, all observers) are the averages (means) of the ratings from each rater group category.

To ensure proper use of a group profile, you should be aware of the following:

• The report is a snapshot of the group from which it was derived. Its meaning or interpretation should not be generalized beyond that group to other groups or to larger segments of the organization. For a combined profile report to be generalized beyond that group, the sample size must be large and must be a random sample of the population to which you want to generalize. (For example, a combined profile report of 30 people from the marketing group cannot be interpreted as having meaning beyond that group of 30 people, unless the 30 comprise the vast majority or a sizable and representative portion of the marketing group.)

• These data are most appropriate for instructional use or for comparison to an individual feedback report. In order to deliver feedback on this combined profile report, you must be certified or trained on the individual use of the instrument from which data are being aggregated.

• The Center for Creative Leadership has made every effort to protect and guarantee the confidentiality and anonymity of the individuals who comprise the group. We expect users of this report to take all necessary precautions to protect the anonymity of individuals and groups and the confidentiality of these data.

• This report should not be shared with other groups (e.g., other business units) without the expressed permission of the group or unless the group identities have been made anonymous. It is conceivable that if results are negative, the information could be used to discriminate or take negative action against that group. Each group member should be told specifically where the results will be shared and asked to sign a written document to give permission for this to be done. This is not meant to prohibit the delivery of grouped data to individuals having overall responsibility for those groups; instead it is to protect individual confidentiality as described above.

• Profiled information must not be used to discriminate against demographic groups or individuals and must therefore be reported and interpreted very carefully.

(25)

________________________________________________________________________

CCL Administrators are your first point of contact for setting up and administering your 360 survey. They ensure the survey is setup according to your contracted specifications. CCL Administrators can help you with questions when you are using the administrative site to monitor the status of participants. They are available by sending an email to

[email protected] or by telephone 336 286 4267. Technical support for participants and raters accessing the 360 BY DESIGN system is available by sending an email to [email protected] or by telephone tollfree within the United States at 877 477 1416. For calls placed from outside the United States use 952 746 5747.

When you are ready to administer the survey, the CCL Administrator will need the following information. An order form is provided to facilitate the process:

• Selected CCL competencies

• Selected CCL standard open-ended questions (optional)

• Custom questions, numerically scored and/or open-ended (optional – additional fee)

• Custom rater categories (optional – additional fee)

• Participant names, e-mail addresses (phone numbers are also preferred and helpful)

• Selection of either local norms or CCL general norms (a minimum of 30 participants must complete the identical survey to use local norms)

• Number and description of desired group profiles and identification of participants who belong to each group (a minimum of five participants are required for a group report to be produced)

• Name, phone number, and e-mail address of a Client Administrator (and alternate contact) who can make decisions about your assessment project

• Name(s) of certified facilitator(s) to whom the feedback reports will be shipped (each individual must have completed a CCL Certification Workshop)

• Name, phone number, and e-mail address of an IT contact person who can assist participants and raters with internal connectivity questions

• A completed timeline, which includes the specific date(s) your feedback reports are needed. Note: Date(s) must allow for CCL to have five days’ scoring time once respondent surveys are received, PLUS shipping time for getting feedback reports to the designated facilitator.

• How you wish to pay: CCL invoice, credit card, purchase order

• For overnight shipping of the feedback reports, the name and account number of your shipping company (FedEx, DHL, UPS, etc.) must be provided

• Standard shipping method is Federal Express 2nd day air. Shipping charges are billed separately, along with any exception fees incurred during the assessment process.

The implementation of 360 BY DESIGN involves five distinct steps that will take you from survey construction through the delivery of the final feedback reports.

Step 1: Customizing the Survey (see Building Your Organization’s Survey, Section 2.4)

(26)

________________________________________________________________________

360 BY DESIGN uses e-mail messages to communicate with participants and their raters. The e-mail messages include:

• Welcome and instruction messages for both participants and their raters

• Reminder messages for both participants and their raters

• Connectivity instructions; who to contact with questions

• Customized content that could include any organization-specific instructions that would be helpful to this specific group of participants and raters

• Purpose of the survey

• Date by which the survey should be returned

• Usernames and passcodes

• Assurance of confidentiality (and exceptions e.g., Boss, Superiors, Others categories)

(27)

________________________________________________________________________

Once the competencies are chosen and submitted, CCL will begin the process of

constructing the 360By Design survey to fit your needs. The following time line provides guidelines related to project planning for the 360 BY DESIGN assessment process:

360 BY DESIGN Order Form

Establishing a Timeline Instructions: Complete all fields highlighted in yellow.

Activity Enter a Date

(mm/dd/yy) Guidelines

Invitation emails sent to

participants 01/20/04

Recommend this date occur 6-8 weeks prior to the feedback delivery date.

Due date for participants to

enter raters 01/30/04 Recommend this date occur 1-2 weeks after sending invitation email.

Due date for participants and raters to submit

completed surveys 02/20/04

Recommend this date occur 2-3 weeks after participant due date to enter raters.

Client contact requests

scoring 02/27/04

Printed feedback reports are shipped to certified

facilitator(s). Enter ship to

location below. 03/05/04

Please allow a minimum of 5 business days after scoring is requested.

Feedback event date 03/15/04 Allow time for shipping.

Participants receive access to electronic feedback reports

and development guide 03/22/04

Require this date occur after participants meet with certified facilitator(s).

To achieve rich and complete feedback, CCL strongly advises that participants and raters have at least 10 business days to complete their survey. When the same individuals are completing multiple surveys, this time becomes critical. Similarly, when the survey is lengthy,

(28)

________________________________________________________________________

Step 4: Project Monitoring

360 BY DESIGN contains three levels of monitoring:

• The designated administrator for your organization can access a read-only program log containing the return status for every participant.

• Each participant in the process can access a read-only participant log to monitor the return status for each of his or her rater groups, but does not see rater name to ensure confidentiality.

• CCL administrators can access participant and rater information.

Step 5: Report Delivery

After the designated cut-off date for completing the instrument, CCL scores and prints feedback reports. Reports are printed in duplicate, one copy in black /white and one in color. The participant’s name and scoring date are displayed through a window in the report cover. Reports are shipped to the certified feedback facilitator for distribution and review with the participant.

How this feedback is delivered to the individuals and groups in the organization is largely up to the organization, with some restrictions.

Because 360 BY DESIGN is a sophisticated assessment tool and because effective facilitation is critical to its successful use, we require that only certified professionals administer it. The certification for 360 BY DESIGN is incorporated into CCL’s two-day Certification Workshop. Individuals can register for an open-enrollment certification workshop at CCL or organizations can arrange for a custom program to be conducted on your site

The feedback report provides data on the skills and perspectives identified as important to your organization. From the report, a participant can learn how to maximize strengths and to take action in areas of needed development. Once an individual has received feedback on his or her survey results, then he or she may choose to share that feedback with others in the organization.

To view sample feedback reports for both individuals and groups, see Section 8 of this Guide.

2.7 Completing a Survey – as a Participant or Rater

The 360 BY DESIGN interface is easy to use and navigate. The navigation choices on each screen are designed to provide clear paths. Both participants and raters see the name of the person on screen for whom they are completing a survey.

The participant receives an invitation email containing a link to the survey on the Web and

(29)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 28 passcodes for entry. He or she in turn selects and submits a rater list that includes rater names, email addresses, and reporting relationships. The system sends an invitation email to all raters. Participant and raters complete the survey and submit it to CCL for scoring.

Instructions are provided at each step along the way.

The system recognizes each rater uniquely using a combination of last name and email address. Occasionally participants use different email addresses for the same rater. If both email addresses for the rater are valid, the system recognizes the rater as two unique

individuals. As a result the rater receives different passcodes in invitation emails requesting feedback from participants. The passcode is always associated with the corresponding feedback request from an individual participant.

The system generates all passcodes and automatically sends out customized communication emails. The passcodes are unique to each individual and should never be shared or swapped.

The raw data are transmitted and scored by the system. The data are secure and raters’

identities are anonymous.

It is important to note that if not enough observers respond for a specified group, the submitted data are compiled into an All Observers category to protect each rater’s anonymity. For example, at least three peers must reply for the participant to receive Peer group feedback. Otherwise, feedback from the one or two peers who do respond will be included with other feedback and will appear as part of the All Observers category. Note:

The Boss, Superior, and Other category feedback is not anonymous if only one rater

responds in the category. This element is made clear to both the participant and the rater in all instructions.

2.8 Contact Information for Questions

For questions about 360 BY DESIGN pricing or administration, to place an order or for inquiries about additional services from CCL, please contact:

Assessment and Development Resources Group Center for Creative Leadership

P. O. Box 26300

Greensboro, North Carolina 27438 336-545-2810 (phone)

336-545-6035 (fax)

[email protected]

(30)

(31)

________________________________________________________________________

Design of Sessions for Feedback Delivery and Development Planning Materials Development for Your Organization’s Instrument

Feedback Workshop for 360 BY DESIGN

Preparing for and Facilitating a One-to-One Feedback Session How to Read and Interpret the Feedback Report

3.1 Design of Sessions for Feedback Delivery and Development Planning Goals for Feedback

At the Center for Creative Leadership, our studies and experience indicate that leadership development is most effective when three elements exist in the right mix for an individual and for a work group. These elements are: assessment, challenge, and support.

As participants make sense of their feedback report data, they are engaged in the activity of assessment for development. When participants work to accomplish goals to become more effective in their work, and to gain new perspectives, they need to seek a balance of support and challenge in order to make the best developmental progress. The Assessment-Challenge- Support framework is described in Three Keys to Development: Defining and Meeting Your

Leadership Challenges (Center for Creative Leadership, 1999) from the Ideas Into Action Guidebook Series.

Determining the Settings for Feedback: Workshops and One-to-One Sessions A small group or workshop setting is highly effective for giving participants an initial

introduction to their feedback reports and to the development planning process. Because the 360 BY DESIGN Feedback Report is rich in data, it is helpful to allow participants a

minimum of a few hours, and preferably overnight after receiving their reports, to study the data and complete the Development Planning Guide exercises prior to a one-to-one session with a facilitator.

We recommend one-to-one sessions especially for coaching for development. When

participants have had time to study and reflect on the feedback data, they are able to come to a session ready to make sense of next steps. The charts below outline components and time estimates of activities for presenting and interpreting the feedback reports. The “Design Components for Feedback Workshop” chart indicates which activities can be done in small group or workshop settings and which belong in one-to-one conversations. As you plan and deliver the feedback interpretation and workshops, be sure to separate these sessions in time from the organizational processes of selection, promotion, and performance appraisal systems.

(32)

________________________________________________________________________

(33)

________________________________________________________________________

©2001, 2004 Center for Creative Leadership. All Rights Reserved. 31 3.2 Materials Development for Your Organization’s Assessment You will need to develop interpretive materials and handouts to accompany your organization’s instrument. The document entitled 360 BY DESIGN Library of

Competencies (see Section 5.5) has a brief description of each scale. In feedback workshops or in one-to-one feedback, the following activity and handout can be used in setting the context for receiving feedback.

A Self-assessment or Reflection Exercise

Lead a 5-10 minute activity in which participants get acquainted with the dimensions on which they will receive feedback in a few minutes. The handout is a list of your

competencies, with the brief description of the content of the scale (see Section 5.5). The instructions may be designed to fit your outcomes.

Example of instructions to participants: (see following page)

(34)

________________________________________________________________________

(35)

________________________________________________________________________

Feedback Workshop Presentation General Outline of Workshop

Step 1: Give an Orientation

Step 2: Discuss 360 BY DESIGN Research Step 3: Set a Context for Understanding Feedback Step 4: Explain How to Read the Feedback Report

Step 5: Provide Time to Read the Individual Feedback Report

Step 6: Ask for General Questions about Structure/Format of Feedback Report Step 7: Guide Participants in Analyzing Feedback Report

Step 8: Ask for Specific Questions about Individual Feedback Reports

Step 9: Refer Participants to Corresponding Exercises in the Development Planning Guide

Step 10:Let Participants Know Whom to Contact or Where to Go If They Have Further Questions/Concerns (when the workshop is followed by a scheduled one-to-one feedback session, this step is included).

Workshop Trainer Notes Step 1: Give an Orientation

Introduction and opening remarks

• Purpose of this workshop

• Purpose of the organization’s 360 feedback initiative

In this workshop, you are our individual clients. This is an “assessment for development” initiative. This is not a program for “assessment for selection” or

“assessment for promotion.” The feedback data can provide you with meaningful information and help you shape your leadership development goals for the coming year.

Our assumptions:

• You own the data. You make the decisions about what the data mean, and you decide what to do with the data.

• You are already effective and high-performing managers.

• You are already successful.

• You are in charge of your own career.

• You as an individual are my client.

Example: Talking points for “Assessment-Challenge-Support” as a framework for

(36)

________________________________________________________________________

• Assessment is the information, presented formally or informally, that tells you where you are now; your current strengths, development needs important to your current situation, and your current level of effectiveness.

• Challenge is the element of an experience that are new and that may call for skills and perspectives not currently available to you; or the elements that create imbalance for you and provide an opportunity to question established ways of thinking and acting.

• Support is the element of an experience that enhance self-confidence and provide reassurance about your strengths, current skills, and established ways of thinking and acting.

Step 2: Discuss 360 BY DESIGN Research

Example: Talking points on the background of 360 BY DESIGN survey construction.

1. Tell the story of how the organization identified key dimensions, or competencies, which were built into our survey using the competencies from the Center for Creative Leadership.

2. To the extent relevant to the audience, give a snapshot of the CCL research studies behind the competencies that were chosen.

Step 3: Set a Context for Understanding Feedback

Facilitate a 5-10 minute group conversation, drawing on the knowledge and experience of participants in the room, about how to receive this feedback data. The purpose of the conversation is to review guidelines that will help the participants who are receiving 360 feedback for the first time and will serve as a reminder for those who have received it before. This can also be a partner exercise or individual reflective activity in response to a series of questions. For example:

• For whom will this be the first time receiving multi-rater feedback? How many of you have received this kind of feedback once before? Twice? More than twice?

• What do you know about the mind-set with which you need to read the data? What advice do you have to offer to those who are experiencing 360 feedback for the first time?

Step 4: Explain How to Read the Feedback Report

In introducing the feedback report, it is helpful to offer participants a “view from 20,000 feet.”

A suggested process is to outline the next several steps with participants and

360 BY DESIGN Facilitator s Guide

360 BY DESIGN

Facilitator’s Guide

Table of Contents

Section 1: Purposes and Uses of 360 BY DESIGN

Section 2: Process Design and Management