Evaluating Collective Impact: The Strive Partnership s Evaluation and Response

Evaluating Collective Impact:

The Strive Partnership’s

Evaluation and Response

Strive Cradle to Career Network Convening

CRADLE TO CAREER CIVIC INFRASTRUCTURE

Five Overarching Goals:

1. Prepared for school

2. Supported inside and outside of school 3. Succeeds academically

4. Enrolls in postsecondary education 5. Graduates and enters a career

Coordinating Action to Support Student Achievement:

• Early Childhood Leaders • District Superintendents

• College and University Presidents

• Community, Corporate, and Private Funders • Business and Parent Leaders

“Put concerned people in one

room, agree upon statistically

definable goals, and then

coordinate action and spend

the dollars to hit the targets.”

Pe

rc

en

ta

ge

Report Card Year

85 80 75 70 65 60 2009 2010 2011 68% 74% 81%

Percentage of Outcomes Trending Positively

But are we adding value?

Should we be doing more?

•

Strive hired OMG to conduct evaluation (2011)

•

Goal—Identify role(s) Strive plays in catalyzing

progress, and identify ways in which it could play

an even stronger role in future success.

•

Process—Conducted 26 stakeholder interviews,

3 focus groups with collaboratives, and surveyed

461 people with 36% response rate (164).

7

Purpose: the commonly held (and unique) reason for coming together, including the

vision and goals, and how the group will work together to reach them.

Value Added: perceptions of whether the group has added value to

community-wide efforts and/or to individual organizations.

Membership: understanding of what membership entails, what is expected of

members and what they can expect, benefits and drawbacks of participation. Also the extent to which an appropriate variety of members is participating.

Process and Structure: clarity around the operating structure and processes

used for decision-making, accountability, and providing feedback/input.

Staff and Resources: skills and talents of the leadership and leadership

group(s), as well as other human and financial resources.

Communications: clarity and sufficiency of internal and external, formal

and informal communications efforts.

OMG Evaluation Framing

Purpose: the commonly held, unique reason for coming together—

vision and goals, and how group will work together to reach them.

•

Nearly 80 percent believe the vision and

goals are clear

•

75 percent believe they are the right ones

for the community

Value Added: perceptions of whether the group has added value to

community-wide efforts and/or to individual organizations.

• The Partnership has added value to community level efforts

in four major roles

• Convener – creating cradle to career framework;

facilitating ongoing discussions on education issues

• Capacity Builder – providing committees &

collaborative with critical non-monetary resources such as training and facilitation to improve performance

• Network weaver – refreshing/maintaining the

network; securing stakeholder buy-in

• Promoter of data based decision making – leading

selection of shared outcomes; establishing baseline; developing annual report; changing media approach

Membership: understanding of what membership entails—roles and

responsibilities of members, benefits and drawbacks for members.

•

Stakeholders have varied perceptions of the

benefits they should and do derive from being

in the partnership

•

Those closest to the leadership of the

Process and Structure: clarity around operating structure, processes

used for decision-making, accountability, and providing feedback/input.

• Confusion exists about roles and responsibilities of the partnership’s Executive Committee

• Interpretations of what Strive the anchor or

backbone organization is accountable for, and to whom, vary widely

• There is lack of clarity about partnership’s

structure--participants without direct connections to staff want to know how to access/provide input to partnership

Staff and Resources: skills and talents of the leadership and

leadership group(s), as well as other human and financial resources.

•

Strive backbone staff are considered by

participants to be knowledgeable,

hard-working, and helpful

•

However, at times, the role of staff is

unclear to participants

Communications: clarity and sufficiency of internal and

external, formal and informal communications efforts

• Opinions vary about how well-informed partners feel • Communication structures and/or processes do not

exist between committees/collaboratives, and are minimal between committees/collaboratives and Executive Committee

• Most participants praise Strive Partnership Report Card as an excellent tool to report region’s progress • Many feel there is little communication about Strive

Executive Committee Feedback

•

Finalize key strategies around each of eight key

outcomes (“success drivers”)

•

Realign existing work

•

Start /stop collaborative efforts based on key

strategy needs

•

Then focus on “opportunities” that emerged

from the evaluation

Opportunities

Lack of clarity about what “membership” or “affiliation” with Strive means, how it is defined, and who else is

involved. Similarly, questions exist about the specific resources and supports that Strive can offer members.

Proposed Response:

• Define partner vs. member vs. stakeholder

• Clarify roles/responsibilities and benefits of membership

• Develop process for becoming an official partner of the partnership • Clarify staff roles and responsibilities

• Identify resources available to partners

• Expand partnership to those committed to improving shared outcomes

16

Opportunities

Lack of clarity around the partnership’s structure, roles and responsibilities of the Executive Committee and

Strive staff, especially for “farther out” participants without direct connections to staff.

• Refine the Executive Committee structure to establish four working groups: Collaborative Action, Continuous Improvement,

Community Will, and Advocacy and Funding Alignment

• Add two staff positions for collaborative action and community will • Develop and distribute organizational charts to all stakeholders

• Host semi-annual partnership events to update and get input from all partners and stakeholders.

17

Proposed Response: Issue:

Opportunities

Varied opinions about how well-informed partners feel about what goes on in the Partnership. Internal

communications mechanisms (in particular, emails) are perceived as lacking a sense of continuity and a full

picture of Strive’s work.

• Develop a comprehensive communications strategy

• Develop a better partnership with Public Allies to better align the work of the 40 AmeriCorps members in Greater Cincinnati to the Partnership’s cradle to career vision and our shared outcomes

18

Proposed Response: Issue:

Opportunities

There are some roles that participants are not sure are appropriate for Strive, or do not believe that Strive has effectively played to date, including:

-- Leveraging and aligning resources -- Pursing state policy work

-- Driving change when leadership doesn’t exist

19

Proposed Response: Issue:

• Explore the appropriateness of pursuing these activities with various members of the Partnership

• Determine the “value-added” impact of doing this work as well as the “expense”

Discussion Questions

20

•

What should the partnership’s role be in

fostering collaborative action?

•

A leader & driver of collaborative action?

•

A catalyst for collaborative action?

•

A supporter of collaborative action?

•

What else are we missing?

Discussion Questions

There are some roles that participants are not sure are appropriate for Strive, or do not believe that Strive has effectively played to date. Is it appropriate for a Strive Partnership civic infrastructure to:

•Leverage and align resources? •Pursue state policy work?

•Drive change when a lack of community leadership exists around an issue?

Discussion Questions

22

What roles and responsibilities are most critical for the Strive Executive Committee to play:

• Overall agenda setting?

• Decision-making about Strive’s overall direction, and/or specific goals and strategies in each

priority area?

• Defining operational expectations for committees/collaboratives?

• Setting a precedent by making organizational level changes?

Discussion Questions

23

•

Does the role of the Executive Committee

change as the partnership matures?

•

What are the individual roles/responsibilities

for Executive Committee members beyond