Out of Many, One: Four Perspec4ves on Managing Limited Submission Programs

(1)

Out of Many, One: Four

Perspec4ves on Managing

Limited Submission Programs

Pollyanne Frantz, Appalachian State University

Kimberly Gray, Georgia Regents University

Carol AshcraF, North Carolina State University

Lesley Brown, University of North Carolina CharloJe

NORDP 5

th

_{Annual Research Development Conference}

May 15, 2013

(2)

Today’s session is about…

Four ins4tu4ons:

• Appalachian State

University

• Georgia Regents

University

• North Carolina State

University

• University of North

Carolina CharloJe

Four perspec4ves:

• Limited submission

management process

• The good & bad of

these processes

• Lessons learned along

the way

• Q&A with par4cipants

@ end of session

(3)

Appalachian State University

Peer Assisted Review Enterprise (PARE)

• What PARE is:

–

 

15 faculty members appointed for two year terms

to serve as peer reviewers

–

 

Reviewers are assigned to three categories: arts &

humani4es, social sciences & educa4on, & STEM

–

 

3 of 5 from a given category are called to review

–

 

Reviews include quan4ta4ve ra4ngs & open-‐

ended feedback

–

 

Reviews are completed electronically through

AsULearn site

(4)

Submission

NORDP 5

th

_{Annual Research Development Conference}

May 2013

NC State Limited Submission Process Overview:

Timeline, RSS feed, subscribe to listserv, send an email

(5)

NORDP 5

th

_{Annual Research Development Conference}

May 2013

NC State currently ac4ve limited programs, with links to

internal requirements and sponsor’s RFP

(6)

NORDP 5

th

_{Annual Research Development Conference}

May 2013

(7)

To contact us…

Carol AshcraF

919.513.2466

hJp://research.ncsu.edu/

rdo/

Lesley Brown

704.687.1880

hJp://research.uncc.edu/

proposal-‐development

Pollyanne Frantz

828.262.7789

hJp://orsp.appstate.edu

Kimberly Gray

706.729.2326

hJp://gru.edu/research/

(8)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

* Georgia Regents University does not yet have a Carnegie classification that reflects the recent consolidation of two institutions (Augusta State University & Georgia Health Sciences University). Prior to the merger, Augusta State University was classified as a Master's L: Master's Colleges and Universities (larger programs) and had 6,500 undergraduates, 300 graduate students, and $3 million in external funding, whereas Georgia Health Sciences University was classified as Spec/Med: Special Focus Institutions--Medical schools and medical centers and had 400 undergraduates, 2,000 graduate students, 10 doctoral programs, and $75 million in external funding.

Appalachian State University Georgia Regents University North Carolina State University University of North Carolina Charlotte

Institution profile Boone, NC

UNC system school Carnegie classification: Master's L: Master's Colleges and Universities (larger programs) 15,712 undergraduates 1,877 graduate students 1 doctoral program 871 FT faculty $12 million in funded proposals Augusta, GA USG system school Carnegie classification: no current classification* 6,245 undergraduates 2,767 graduate students (523 residents) 10 doctoral programs 651 FT faculty $75 million in funded proposals Raleigh, NC UNC system school

Carnegie classification: RU/H: Research Universities (high research activity)

34,000 undergraduates 9,500 graduate students 61 doctoral programs 2,000 FT faculty

$286 million in funded proposals

Charlotte, NC UNC system school

Carnegie classification: DRU: Doctoral/Research Universities 21,179 undergraduates 5,053 graduate students 20 doctoral programs 1,061 FT faculty $28 million in funded proposals

How do you identify limited submission programs?

Pivot, Grants Resource Center, and sponsor announcements

? Mostly Pivot funding alerts; secondarily, Pivot tracked RFPs; also NSF alerts, historical data, tips from faculty, staff

Pivot and agency websites

Who identifies them? Pollyanne Frantz, Grants Resources & Services Director, & Amy Love, Assistant Director

faculty and/or grant officers Carol Ashcraft, Research Funding

Development Manager Susan Robinson, Information Coordinator for the Office of Proposal Development

How do you announce

them? Email messages, newsletters, blog postings and web site page for limited submissions

There are no

announcements Almost exclusively via the funding listserv (research deans, dept. heads, directors of centers, institutes. & labs, &

self-subscribed faculty). Very narrowly defined programs are announced only to the department or faculty group that is eligible.

Susan announces by targeted emails and posting to website

(9)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

2

Do you have a "notification of interest" deadline?

Yes No Yes: usually a week after the listserv

announcement Yes

If faculty miss the "notification of interest" deadline, can they participate in the review?

No N/A Almost never. No

Do faculty identify

themselves? Yes Yes Yes Yes

Do you have a standing committee for all reviews, or do you use an ad hoc panel for each review?

We have a standing stable of faculty reviewers (18) through the Peer Assisted Review Enterprise (PARE). Reviewers are assigned to broad academic categories (arts & humanities, social sciences & education, and STEM)

Neither really, just the AVP for Research Administration and the SVP of Research

Ad hoc Both

If you have a standing committee, how does that work? If you have ad hoc panels, who invites

them? How does that person decide who should be on the panels?

PARE reviewer selection is primarily based on reviewer availability as well as whether reviewer must be recused due to status as applicant in internal competition.

N/A Ad hoc panels are seated based on recommendations from dept. heads, others, & on expertise, on previous review experience if applicable, & sometimes REACH NC, other. My limited submissions spread-sheet identifies reviewers who served on the panels, plus suggested reviewers and who suggested them. I identify & invite. I also keep a "reviewer selection guide" spreadsheet, which will eventually be a searchable database.

Standing committee is elected by the faculty with representation from each college plus the library. Ad hoc panels are invited by the VC for Research and

Economic Development. Panels often consist of deans plus the Provost and VC for Research, sometimes selected on the basis of expertise.

(10)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

3

Do you have institutional exceptions to this process? If so, what determines when these exceptions are and what they are?

No N/A Yes, occasional exceptions are made for

equipment, instrumentation or facility improvement RFPs (but not MRI). For these programs, when I identify them, I notify the Vice Chancellor, who decides how to conduct the selection. I invite reviewers.

No

Do you have an

evaluation form? Yes. Evaluation form mirrors sponsor review criteria and includes sections for reviewer comments on concept paper strengths and weaknesses

No Yes, it is a very simple form asking for strengths, weaknesses & suggested improvements. It also includes a link to the RFP.

We solicit white papers, C.V.s, and current grant support info. White papers are based on the

information that will be required in the proposal to the agency.

Do you have review meetings? What role do they play?

No. Reviewers complete evaluations independently via a PARE site in the institution’s learning management system (Moodle).

No Yes, review meetings are where the actual selections are made. Very, very few reviews can be conducted w/o a meeting. Some reviews also include presentations by each faculty member directly to the panel.

Yes. Discussion of proposals and decision making.

Do your panels' selections move forward, or do they have to be approved by someone else before the panels' selections can be confirmed?

Reviewers' ratings are final and do not have to be approved by someone else before moving forward

N/A The panels' selections have to be approved by the Vice Chancellor before the faculty can be notified, but the selections usually are approved.

Panel selections are final.

Who notifies PIs of the review

results? How are they notified?

Grants Resources & Services staff. Applicants are notified via e-mail.

Grant Officer I notify each one personally by email. Director of Proposal Development notifies by email.

(11)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

4

Is feedback

provided? Feedback is provided to both selected and non-selected applicants and consists of brief comments about the strengths and weaknesses of the application.

informal feedback from SVP

of Research Yes. We provide feedback to help selected faculty prepare stronger full proposals and help unselected faculty compete more effectively in the future. I write review summaries based on the reviewers' written and oral comments about each preproposal. Comments are unattributed to conceal reviewers' identities.

Yes and no. The elected committee provides feedback in the form of written comments from the committee chair. Ad hoc committees do not.

Are reviewers' identities kept confidential?

Yes N/A Yes No

Do you have any practices regarding PIs who are

re-competing?

No, although applicants are allowed to submit external reviewers' comments.

No If a re-competing PI was selected before, then that PI must submit the sponsor reviews, if any, with the new internal preproposal.

If their proposal scored highly, they are required to provide the review sheets and describe how they will respond to the reviewers' concerns. If the committee finds their response

compelling, they are allowed to resubmit.

Do you have any practices to prevent "reviewer burn-out" from repeated and frequent service on different review panels?

No, but we are adjusting the review schedule and PARE assistance calendar because of this threat. The request for review service sometimes segues into a plea for review service.

N/A Yes, I try to avoid consecutive invitations to review for the same person. I keep a "reviewer selection guide" that will eventually become a searchable database.

Elected committee serves a 2-year term and only reviews 2 programs. Most other proposals are reviewed by deans, who aren't allowed to burnout. :-)

(12)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

5

Does your institution demonstrate concern for a fair and open review process based on merit? How do you ensure that your process is fair, objective and open?

We try. We consider ourselves to be like Switzerland in this process: neutral.

No Yes, we usually let faculty know who else is competing & encourage collaboration when possible. Reviewers' identities are kept confidential so they may freely provide instructive comments w/o jeopardizing professional relationships. We have a c-o-i policy for reviewer selection to identify possible c-o-i or appearance of c-o-i. We seat some reviewers from outside the depts., colleges of the competing faculty.

Yes. The faculty committee is part of faculty governance. When selections are made by the deans and provost, the projects are evaluated based on their alignment with strategic initiatives at the university.

Do you post your

process on a website? Yes No Yes, we post a description of the overall process, how the reviewers are selected, & every step of the review. Faculty can sign up for RSS feeds about

announcements, subscribe themselves to the funding listserv, & email me directly from that page.

Yes

Do you track your limited submission program activity? If so, how? What do you measure?

Yes. We measure the number of internal competitions conducted, the number of internal applications received, and whether the selected applicants are successful in the external competition.

No Yes: # of programs, # of programs to which proposals were submitted, # of reviews, # of competitors, # of proposals selected in reviews, maximum possible award amount per program, total possible award amount for all programs each year, reviewer info. We also post a table of current programs & deadlines.

No. We keep a record, but we don't assess the level of activity.

(13)

Out of Many, One: Four Perspectives on Managing Limited Submission Programs

NORDP 5

th

_{Annual Research Development Conference}

6

What percentage of your time do you estimate limited submission program management requires?

20% (PARE reviewers also assist with reviews of selected internal funding opportunity programs)

Very limited More than 50%. I don't know how to

estimate this. Susan

Robinson monitors daily, and I monitor weekly, but that doesn’t capture the time it takes to run internal competitions.

Out of Many, One: Four Perspec4ves on Managing Limited Submission Programs