MD Program Evaluation Plan. (August, 2009)

(1)

MD Program Evaluation Plan

(August, 2009)

The Commonwealth Medical College (TCMC) requires its medical students to develop

competency in the areas of patient care, medical knowledge, practice‐based learning and improvement,

interpersonal and communication skills, professionalism, and systems‐based practice. The ACGME’s

Core Competencies were used when the curriculum was originally planned in 2007 and have remained

in the forefront as we revisit the initial curricular vision to make it a reality for our first cohort of

students who arrive in August, 2009. The expected level of competency attained must be sufficient to

allow these new physicians to be successful in graduate medical education programs as well as to reflect

the vision and mission of The Commonwealth Medical College. The MD Curriculum defines the

milestones of competency development. Components of self‐reflection and assessment are important

for both faculty and students. Within each program course, assessment and feedback becomes part of

the culture of professional behavior and develops proclivity toward lifelong learning.

As a new institution, evaluation and assessment are integral parts of curricular development,

linking learning outcomes to authentic forms of assessment that will provide indicators of a student’s

path of accomplishment as they matriculate through their program of study. The course directors and

faculty have committed to designing a program of instruction that incorporates active learning

strategies along with formative and summative assessment opportunities. TCMC has obtained PDE

approval and preliminary accreditation from the LCME to begin offering the medical curriculum to

students beginning in August 2009.

Key components of the MD curriculum include:

 The medical school will function with a distributive model of medical education developing

three Regional Campuses (Scranton, Wilkes‐Barre and Williamsport) to provide students

experiences in diverse clinical settings.

 Clinical education starts on day 1 with students assigned to a “Learning Community” that

corresponds to their Regional Campus, a family from that Regional Campus to follow for a

continuity experience and a clinical mentor. Additional clinical experiences will provide students

with training in an interdisciplinary setting by working with health professional training

programs at regional colleges and universities.

(2)

service. Students will be exposed to community health improvement projects throughout their

medical school experience.

 The curriculum is an integrated competency‐based system. Competency‐based implies a

developmental curriculum with multiple measures to document the path to mastery. It is not a

one‐sized fits all course model. Each course has identified key measures that will document the

competencies contained therein. Narrative evaluations by course directors and clinical faculty

are imperative for many of the competencies not measured by traditional examinations. Course

based assessment provides for both formative and summative measures.

 Students begin year one with basic science, move through an organ based system in year two,

an integrated clerkship in year three, and subinternships in year four. Years one and two will

occur on the Scranton campus with three one week experiences at the student’s regional

campus (assigned upon entry). Years three and four will occur at the student’s regional campus.

 With the distribution of students across three regional campuses, consistency is provided

through the use of the TCMC portal – where technology provides a common opportunity for

academic activities and evaluation procedures. This is imperative in years three and four when

students are at their regional campus.

 The LCME Database and PDE Application provide roadmaps for curricular development and

assessment of competencies. Overall Review Process

The MD Program Evaluation Plan covers a four year academic year program delivery cycle with

data collection points throughout the four academic years. Programmatic review occurs in May through

July of each year with an extensive, systemic review occurring the first four year cycle of the program.

This allows for continued development as the first cohort of students matriculates through the program.

After the first cohort completes the four year program cycle, the normal cycle of periodic review of the

four year cycle begins. In addition, student performance is reviewed on an annual basis (at minimum) or

as necessary if the Associate Dean for Academic Affairs or the Associate Dean for Student Affairs

requires review of a student’s record. All students are reviewed annually by the Committee on

Academic and professional Standards (CAPS).

The first four year cycle of the MD program will have full review of each year of the curriculum

each year. The first class arrives in August, 2009 resulting in a complete review of year one in May thru

July, 2010. In August, 2010 the second cohort arrives and the first cohort moves into the second year of

the curriculum. In May, 2011 a full review of the curriculum will be undertaken for both years one and

two. The same process holds for the 2011/2012 academic year when the first three years will be

reviewed. The final full review occurs in 2012/2013 with the graduation of cohort one and the full

review of years one through four. The AAMC Graduation Survey, which is the gold standard for medical

school overall program evaluation, is administered to students upon graduation from the college, will

give us an overall view of the four year program in the summer of 2013. Each subsequent graduating

(3)

perspective in addition to the annual data collected internally. Longitudinal data analysis begins in year

five. Five‐ and Ten‐year TCMC Post Graduation Surveys will be sent to graduates beginning in 2018(Five‐

year) and 2023(Ten‐year).

The annual program review process enables the faculty and administration of TCMC to make

corrections to the curriculum long before the first cohort completes the program. The Curriculum

Action Committee (CAC) has overall institutional oversight for the curricula of the college. There is an

MD Committee of the CAC that oversees the MD Curriculum for compliance with LCME and ACGME

competencies. There are four subcommittees of the MD Curriculum Committee: Year One, Year Two,

Clinical Years (3 & 4), and Assessment. Members of the Assessment Subcommittee of the MD

Curriculum Committee are charged with conducting the program review on an annual basis. This is a full

curricular review for the first four years of the program (2009‐2013). After that a periodic review

schedule will be maintained with 25% of the courses in the program undergoing full review each year,

the remaining 75% partial review. The entire program will cycle through complete review every four

years. Full review requires each course director to do a self‐study of their course reviewing learning

outcomes and competency assessment through the review of student data, evaluation data and course

fit within the overall curriculum map. Curriculum maps are reviewed and updated as necessary. Partial

review uses student data and evaluation data such as course evaluations to ensure curriculum

compatibility and student satisfaction.

The Committee on Academic and Professional Standards (CAPS) of TCMC is responsible for the

annual review of all student records. The Associate Dean for Academic Affairs handles the initial review

of student records and preparation of reports for the Committee on Academic and Professional

Standards. At any time, the Associate Dean for Academic Affairs or the Associate Dean for Student

Affairs may request that the group convene if either officer determines that a student is in significant

academic or behavioral difficulty. The CAPS convenes to review student records and suggest a course of

action for remediation or dismissal.

MD Annual Program Evaluation Plan

The Curriculum Action Committee provides oversight for all curricula at TCMC. It currently has

two major curricular committees: the MD and the MBS. (See Figure Two: Curriculum Action

Committee Structure at the end of this document).The MD Curriculum Assessment Sub‐Committee will

meet annually to review the MD Curriculum for the MD Curriculum Committee. A report with

recommendations will be sent to the MD Curriculum Committee. That group will review the report and

suggest program revisions as needed. The MD Subcommittee is comprised of a rotating group of

program course directors. Each year of the program has a sub‐committee within this structure. All

course directors are members of their respective year sub‐committees. The MD Curriculum Committee

will send the report completed by the MD Curriculum Assessment Sub‐Committee and the subsequent

actions taken to the TCMC Curriculum Action Committee (CAC), who has curricular oversight for the

(4)

Curriculum Committee in early July to ensure overall impact with all TCMC academic programs (for the

present – MD and MBS). The CAC sends their findings to the Associate Dean for Academic Affairs. In

July, the MD Course Directors and Faculty will make changes to the program recommended by the MD

Curriculum Committee and approved by the CAC. In August, the MD Program Curriculum and Progress

Reports are reviewed by the Dean’s Cabinet (presented by the Associate Dean for Academic Affairs). A

report to the TCMC Board of Trustees follows in September. This process occurs annually in order to

provide students with the most relevant and up‐to‐date program.

Figure One: MD/MBS Program Review Cycle provides an overview of the program evaluation

process. Later tables detail the program and student evaluation process components on an annual

basis.

Figure One: MD Program Review Cycle

The MD Annual Program Evaluation Plan allows for review of program curriculum and student

data. In addition, budget analysis and program unit analysis is completed for all program units who

serve the MD Curriculum such as Student Affairs, Center for Learning Excellence and the Library among

others. This is accomplished as part of the TCMC Institutional Evaluation and Assessment Plan.

MD Program Review (April/May) by Course Directors and MD Assessment Sub‐committee of MD Curriculum Committee

Curriculum Advisory Committee Review of MD Program Evaluations and Recommendations (July)

Leadership Review and Implementation of Curricular

Recommendations (July/August)

MD Curriculum Evaluation and Assessment Cycle Begins

Again (August) MD Curriculum Evaluation

and Assessment Reports to TCMC Board of Trustees

(September) Data Collection throughout

(5)

Table One: MD Annual Program Evaluation Plan

Activity Dates Assessment

Activity

Collection of Data Product Due Responsible

August thru June Admissions Data On‐going Admissions

Projection –

December

Final Admissions

Report ‐ June

Director of

Admissions

and MD

Admissions

Committee

December and

May

Course Grades Formative (December)

and Final Grades

(May)

Grade Analysis

Report

December and

May

Registrar

August thru May Review of

narrative course

data for each

student

Student Narrative

Reports for each

course in MD Program

Review of

narrative course

data by Course

Directors at the

conclusion of each

course using

Competency

Based Rubrics.

Students

experiencing

difficulty will have

their narrative

reports reviewed

by additional

course faculty, the

Associate Dean for

Student Affairs or

the Associate

Dean for

Academic Affairs

as needed. A final

report on each

student will be

submitted to the

Associate Dean for

Academic Affairs

in May by all

course directors.

Course

Directors

(6)

Activity Dates Assessment Activity

December, May

and June

Review of Student

Records

Review of grades and

student narrative

reports by the

Committee on

Professional and

Academic Standards

Students

experiencing

difficulty will meet

with the Associate

Dean for Academic

Affairs and a

remediation plan

will be put into

place

Associate Dean

for Academic

Affairs,

Director,

Center for

Learning

Excellence, and Associate Dean

for Student

Affairs

March/April Request for

Program Review

MD Curriculum

Committee

Assessment Sub‐

Committee requests a

full or partial review of

Year Based Programs.

First four years of

cycle – all will be full

review, thereafter –

25% of all courses in

each year will be full

review, 75% will be

partial review

MD Program

Report

MD Course

Directors and

MD Curriculum

Committee Assessment Sub‐ Committee Course Conclusion

Course Evaluations

by Students

Student Course

Evaluations

Statistical Analysis

and Qualitative

Comments

Reported to

Course Faculty,

Course Directors,

and Associate

Dean of Academic

Affairs

Office of

Evaluation and

Assessment

Course

Conclusion

Course Evaluations

by Faculty

Faculty Course

Evaluations

and Qualitative

comments

reported to

Program Director

and Associate

Dean of Academic

Affairs

Office of

Evaluation and

Assessment

(7)

April Advising

Evaluations by

Students

Student Advising

Evaluations

and Qualitative

comments

reported Center

for Learning

Excellence,

Advisors, Program

Director and

Associate Dean of

Academic Affairs

and Associate

Dean of Student

Affairs

Office of

Evaluation and

Assessment

April Advisor Survey Faculty Analysis of

Advisees

and Qualitative

comments

reported Center

for Learning

Excellence,

Advisors, Program

Director and

Associate Dean of

Academic Affairs

Office of

Evaluation and

Assessment

May (prior to May

15)

Program Retreat Associate Dean for

Academic Affairs, MD

Course Directors and

Course Faculty, others

as invited

MD Program

Review Report

and

Recommendations

Compiled and

Reported to MD

Curriculum

Assessment Sub‐

Committee

Associate Dean

for Academic

Affairs

May (after May

15)

Curriculum and

Progress Review

MD Curriculum

Assessment Sub‐

Committee – review of

all related data and

reports

MD Curriculum

Report

MD Curriculum

Assessment

Sub‐Committee

(8)

June MD Curriculum

Committee of the

Curriculum Action

Committee (CAC)

Review of MD

Program Report

from MD

Assessment Sub‐

Committee and

others as needed

Program is

reviewed at the

MD Curriculum

Committee Level

and

recommendations

forwarded to the

CAC for

institutional

approval

MD Curriculum

Committee of

the CAC

July Curriculum Action

Committee (CAC)

Review of MD

Program Report

from the MD

Curriculum

Committee of the

CAC

Associate Dean

for Academic

Affairs and MD

Curriculum

Committee of

the CAC

July/ August Changes made to

Program as needed

after Curriculum

Action Committee

(CAC) review

Course

Directors and

Faculty

August MD Program

Report Presented

to Cabinet

Review of MD

Program Report

Associate Dean

of Academic

Affairs

September MD Program

Report presented

to TCMC Board of

Trustees

Review of MD

Program Report

Associate Dean

of Academic

Affairs

Table Two: First Five Year Evaluation Process details the MD Curriculum evaluation process over the

next 6 years providing details for the first four year curriculum cycle and subsequent annual reviews.

(9)

Table Two: First Five Year Evaluation Process August, 2009

First Cohort (Class of

2013) arrives

MD Program and

Student Evaluation

Process Begins

September, 2009‐ April 2010

Data collected

May, 2010

Full review of Year 1

Courses by MD Course

Directors and MD

Assessment Sub‐

Committee

June, 2010

MD Curriculum

Committee Review

July, 2010

TCMC Curriculum

Action Committee

Review

Program Changes made

as needed

August, 2010

Cohort 2 arrives

Cohort 1 enters Year 2

MD Curriculum Report

and Student Data

Analysis to TCMC

Leadership

Data collected

September – MD

Curriculum Report and

Student Data Analysis

to TCMC Board of

Trustees

May, 2011

Full review of Year 1

and 2 Courses by MD

MD Assessment Sub‐

Committee

June, 2011

MD Curriculum

Committee Review

July, 2011

TCMC Curriculum

Action Committee

Review

as needed

August, 2011

Cohort 3 arrives

and Student Data

Analysis to TCMC

Leadership

Data collected

to TCMC Board of

Trustees May, 2012

Full review of Year 1, 2,

and 3 Courses by MD

Committee

June, 2012

MD Curriculum

Committee Review

July, 2012

TCMC Curriculum

Action Committee

Review

as needed

August, 2012

Cohort 4 Arrives

and Student Data

Analysis to TCMC

Leadership September, 2012‐ April

2013

Data collected

to TCMC Board of

Trustees

May, 2013

Cohort 1 Graduation

AAMC Graduation

Survey

Full review of Year 1, 2,

3 & 4 Courses by MD

June, 2013

MD Curriculum

Committee Review

July, 2013

TCMC Curriculum

Action Committee

Review

as needed

(10)

Committee

August, 2013

Cohort 5 Arrives

and Student Data

Analysis to TCMC

Leadership

Data collected

to TCMC Board of

Trustees

May, 2014

Cohort 2 Graduation

AAMC Graduation

Survey

Regular Annual Review

Cycle Begins by MD

Assessment Committee

of the MD Curriculum

Committee.

25% of courses in the 4

years of the program

are given full review,

remaining 75% receive

partial review. Full

review involves all

facets of curriculum,

compliance with LCME

and ACGME

competencies, student

progress, course

evaluations, sample

student work review,

narrative review, etc.

Partial review is course

evaluations and student

data review.

June, 2014

MD Curriculum

Committee Review

July, 2014

TCMC Curriculum

Action Committee

Review

as needed

August, 2015

Cohort 6 arrives

Cycle repeats as in

2013/2014

MD Student Data Collection

The MD Program will collect data for each MD student during the four years they are enrolled in

the program. This data is compiled from a variety of sources: 360° Student Evaluation, examination

(11)

course directors and faculty, checklist evaluations by clinical faculty, standardized patient and clinical

simulations, objective structured clinical examinations (OSCE), case analysis, oral examinations,

laboratory practical examinations, presentations, preceptor ratings, NBME Subject Exams, essays,

papers, and community research project. 360° Student Evaluation includes advisor and patient surveys,

self and peer assessment along with grades and narrative information from courses. Data collection

varies by course and year in program. All are detailed in the LCME Database. Data collection begins

with pre‐matriculation surveys completed in the summer before the first year classes and concludes

with the AAMC Graduation Survey and 5‐Year and 10‐Year Post Graduation Surveys. (See LCME

Database for details of competencies assessed: medical knowledge, practice based learning and

improvement, interpersonal skills and communication, professionalism, patient care, and systems based

practice.

Questions about the MD Student and Program Evaluation Plan should be directed to Dr. Barry Linger,

Associate Dean for Academic Affairs or Dr. Susan Perlis, Assistant Dean for Evaluation and Assessment.

In addition to the MD Program Evaluation Plan which addresses curricular improvement and student

matriculation progress based upon student outcomes, components of the MD Program are evaluated

through program unit offices that service the program. Student Affairs, Admissions, the Center for

Learning Excellence, among others, all participate in the TCMC Institutional Evaluation and Assessment

Plan at the unit level assuring that all components of the organization that touch the MD Program are

carefully reviewed for mission integration and service to constituents.

Update: August 6, 2009

(12)

Figure Two: Curriculum Action Committee Structure

Curriculum

Action

Committee

MD Curriculum

Year 1 Year 2

Clinical

Curriculum Assessment

MBS Curriculum

Assessment