Video Script: Program Review Process and Steps at BGSU

Introduction:

Welcome! In this presentation brought to you by the Office of Institutional Effectiveness, we will address program review at Bowling Green State University – specifically, we will guide you through the implementation steps of the overall program review process. So, without further ado, let us begin with a contextual overview.

Overview:

·       As seen in Articles 11 and 12 of its Academic Charter, Bowling Green State University is committed to the continuous review and improvement of its academic programs as essential for effective student learning and data-driven strategic planning. Program review is intended to be helpful, meaningful, collaborative, and useful to program faculty.

·       At BGSU, the Comprehensive Program Review (CPR) process is under the purview of the Provost and Senior Vice President for Academic Affairs, in collaboration with the deans of the undergraduate colleges, the dean of the graduate school, and the academic unit heads and faculty in individual departments.

·       The Office of Institutional Effectiveness (OIE), reporting to the Provost and Senior Vice President for Academic Affairs, has charge of the coordination and monitoring of program reviews for the University at large, including: making a program review schedule widely available and updating it on an ongoing basis, providing assistance with logistics and details of the program review process, and developing program review resources as necessary.

Purpose: [At this point, you may be wondering: But what is the purpose of program review anyway?]

Program review fulfils many purposes, including:

·       Fostering a culture of critical reflection on teaching and learning

·       Ensuring continuous improvement in light of best practices from peer institutions

·       Monitoring program performance with respect to mission and student learning.

·       Informing effective planning and resource allocation

·       Demonstrating contribution to the University’s Mission and Strategic Plan’s Initiatives

Timeline: [Let us turn now to the timeline for program reviews]

·       The schedule for academic program review is set by the Office of Institutional Effectiveness (OIE) with input from College Deans/Associate Deans, the Program Review Academic Committee (PRAC), and the Provost and Senior Vice President for Academic Affairs in compliance with BGSU’s Academic Charter.

·       Program reviews must occur at least once every 5 years – or earlier as may become necessary or as dictated by annual reporting such as the Program Vitality Analysis. In some instances, a program that has already participated in a review may be asked to be reviewed again outside of its typically scheduled review to assure that progress towards improvement is being made

·       The program review process itself should span no more than 3 academic semesters

Roles and Responsibilities: [With respect to…}

The BGSU Program Review Handbook provides a detailed description of roles and responsibilities for faculty involved in the program review process, Program Review Coordinators, College Deans/Associate Deans, Provost Designee in the Office of Institutional Effectiveness, the Program Review Advisory Committee, and the Provost and Vice President for Academic Affairs. College Dean(s)/Associate Deans are responsible for identifying the Program Review Coordinators in their respective College/School.

Major Process Steps: [With this slide, we now dive more deeply into each step of the program review process]

Major steps in the program review process include:

(1) Request of institutional data from the Office of Institutional Research (OIR) and the Office of Institutional Effectiveness (OIE) and Completion of a SWOT analysis

(2) Completion of a Memorandum of Understanding (MoU) highlighting four to five questions/issues that came up during the SWOT analysis and need to be prioritized in a more in-depth manner in the self-study report

(3) Completion of a data-based and data-driven self-study report, inclusive of an Action Plan, with input from faculty, staff, and students

(4) Selection of External Reviewers and Submission of a No-Conflict-of-Interest form and Independent Contractor Agreement (ICA) by those external reviewers

(5) Completion of a site visit (either virtual or in-person) completed by a team of external reviewers. The visit should include class observations, tour of facilities, interviews of students, faculty, and staff, etc.

(6) Provision of an external review report and executive summary by the team of external reviewers

(7) Review of the external findings by the program and response made to the College Dean/Associate Dean and Revision of program’s Action Plan against the external findings.

(8) Once the revised Action Plan is approved by the Dean/Associate Dean, implementation of the final Action Plan ensues and follow-up. Now, let us review each of these steps one-by-one.

Step 1: Initial Meeting:

·       An initial meeting is scheduled with the College Dean and/or Associate Dean designee, Program Review Coordinator, program chair(s), and the OIE Provost Office Designee to establish a timeline for program review and to discuss the following:

o   Steps in the program review process;

o   Forms, templates, reporting platforms and protocol to be used;

o   Data needs;

o   Guidelines for conducting the SWOT analysis, MoU questions/critical issues, Self-Study format, Action Plan format, selection of external reviewers process, visit logistics, funding, etc.;

·       If a graduate program or cluster of programs with a graduate program is being reviewed, the Dean of the Graduate College (or designee) must also attend the initial meeting.

·       The OIE Provost Designee facilitates the initial meeting.

Step 2: Requesting Institutional Data and Conducting a SWOT Analysis:

·       The OIE Provost Designee and Program Review Coordinator meet again to discuss data needs to complete a Strengths/Weaknesses/Opportunities/Threats (SWOT) analysis of the program or cluster of programs under review.

·       Since results from the SWOT analysis inform the critical questions and/or issues to be articulated in the formal Memorandum of Understanding (MoU) for the Self-Study, data categories ought to reflect those to be included in the Self-Study.

·       A non-exhaustive list of data categories is provided in the Program Review Handbook. They include, for example: enrollment and demographics data, faculty scholarly and instructional productivity data, data relative to student learning, data relative to resources and facilities, and evidence relative to curricular quality, etc.

Step 3: Developing a Memorandum of Understanding (MoU):

·       The Memorandum of Understanding (MoU) serves as a mechanism for the program faculty, Program Review Coordinator, College Dean(s)/Associate Dean(s), OIE Provost Designee, and Provost to outline the primary focus or foci for the program review.

·       The MoU is never to be developed as a stand-alone, contextually self-serving document, but should always be aligned to the strategic plans of the respective College, Department/School, and University at large. In this sense, the MoU serves to support that the program under review contributes to BGSU’s Mission, Vision, and Strategic Plan.

·       Examples of potential issue(s)/question(s) in alignment with BGSU’s strategic plan and initiatives are provided on this slide:

o   Redefining student success: Provide undergraduate and graduate students a demonstrably superior and innovative learning experience that intentionally prepares them to lead meaningful and productive lives (BGSU Strategic Objective I).

o   Increasing and connecting research and creative activities for public good: Support and focus BGSU’s research and creative activities to serve the public interest and support commitment to the public good (BGSU Strategic Objective II).

·       A MoU Template is provided in the Program Review Handbook.

Step 4: Program Self-Study and Action Plan Submission and Internal Review and Approval

·       The program faculty, under the direction of the Program Review Coordinator, is responsible for preparing the Self-Study and Action Plan. The self-Study process is an opportunity indeed to engage faculty, staff, administrators, and students in the self-assessment and improvement of the program or cluster of programs.

·       The Self-Study (not including attachments and Action Plan) should not exceed 50 pages. Supporting evidence for claims made in the Self-Study may be attached and/or uploaded separately provided they are clearly referenced in the narrative and a consistent nomenclature is followed. A sample outline for the self-study is provided in the Program Review Handbook.

·       Data should be used to inform and support claims made in the Self-Study. Data from the Office of Institutional Research (OIR) are the primary source of data. Additionally, OIE may also supplement data from Academic Performance Solutions (APS). The Office of Academic Assessment (OAA) may also provide data relative to Bowling Green Perspective outcomes, student achievement outcomes, graduate surveys, employers’ surveys, co-curricular outcomes, etc.

·       The Self-Study process should result in the development of a measurable, feasible, actionable, Action Plan for continuous improvement of the program or cluster of programs within a specified timeframe. This Action Plan must be approved by the Dean/Associate Dean designee.

·       The Program Review Coordinator submits the Program Self-Study and Action Plan to the College Dean(s)/Associate Dean(s) as well as to the OIE Provost Designee for feedback and review. The College Dean(s)/Associate Dean(s) and the OIE Provost Designee review the Self-Study and Action Plan and provide feedback specifically on goals, priorities, and the feasibility of the Action Plan within the context of the College and University.

·       Prior to review of the Self-Study and its evidence by external reviewers, a first layer of internal review occurs. The internal review is completed by the OIE Provost Designee who works collaboratively with PRAC members who serve as “second readers.” The OIE Provost Designee issues a Self-Study Approval Status Letter (SSASL) with recommendations for improvement of the Self-Study. The OIE Provost Designee, in consultation with PRAC members, may necessitate multiple reviews of the Self-Study to confirm finetuning. In this reiterative process, multiple SSASLs may be issued. OIE approves the final version of the Self-Study (and accompanying evidence) to be shared with external reviewers and explains next steps.

Step 5: Selection of External Reviewers and Visit Logistics:

·       Three months in advance of the external review visit, the Program Review Coordinator will work with program faculty to identify and submit the names and credentials of 4 to 6 recognized peers from similar programs at other universities.

·       Working from this slate of potential reviewers, the OIE Provost Designee will communicate agreement on a list of at least three (3) potential reviewers – two required and an additional one should in extremis replacements become necessary. Budget allocations from the Provost’s Office will cover only 2 external reviewers.

·       Once two external reviewers have been confirmed by the Provost’s Office, a No-Conflict of Interest Form, a current C.V/resume, and an Independent Contractor Agreement (ICA) must be executed and archived with the Office of Institutional Effectiveness (OIE).

·       Review visits can be conducted on site or virtually. Programs requiring the use of laboratories must arrange for a site-only visit.

·       The Program Review Coordinator is responsible for scheduling and coordinating all aspects of the external reviewers’ site/virtual visit, including meetings with the College Dean(s)/Associate Dean, Dean of the Graduate College (as applicable), and the OIE Provost Designee who must be included on the visit exit session. A copy of the finalized visit schedule is due to the Provost Designee in the Office of Institutional Effectiveness (OIE) six (6) weeks prior to the visit.

·       At least one month prior to the scheduled visit, the Program Review Coordinator should provide the following materials to each of the external reviewers:

o   1. The program’s Self-Study & Action Plan and all accompanying evidence;

o   2. Copy of Bowling Green State University’s Mission, Vision, and Strategic Plan;

o   3. Copy of the finalized visit schedule for the visit (if virtual, all Zoom or virtual links should be included with special attention to time zones);

o   4. Copy of Sample Questions for Use by External Reviewers During the Visit (a copy of which is featured on the OIE’s website);

o   5. Copy of the Executive Summary Template;

o   6. Copy of the program’s fully executed Memorandum of Understanding (MoU).

On this next slide, a sample of visit schedule is featured.

Step 6: Reception of External Reviewers’ Findings Report and Executive Summary:

·       The external reviewers’ final consolidated report of findings along with their Executive Summary are due three (3) weeks after their visit and should be sent directly to the OIE Provost designee who will forward these documents to the Program Review Coordinator and College Dean(s)/Associate Dean(s). These documents are kept on file in the Provost’s Office and in the Office of Institutional Effectiveness.

·       There are no page limits to the external reviewers’ final report of findings; however, the Executive Summary is not to exceed 5 pages. A Template of the Executive Summary is provided in the Program Review Handbook.

Step 7: Response to External Reviewers’ Findings and Revision to Action Plan:

·       Following receipt of the external reviewers’ final report of findings and Executive Summary, the Program Review Coordinator shall meet with faculty and the College Dean/Associate Dean to discuss the review outcome. Within two weeks of the conclusion of the meeting, the Program Review Coordinator shall complete the Response to External Findings template which is to be found in the Program Review Handbook as well.

·       The response from the College Dean/Associate Dean commits the College to a course of action. The College Dean/Associate Dean’s response could endorse the program review and/or program/cluster report as written; it could commit to only specified parts of the reports; it could adopt revisions suggested in the program response; or it could add recommendations overlooked in both documents.

·       The faculty response to the external reviewers’ report that is agreed upon by the Program Review Coordinator and the College Dean/Associate Dean will be formally integrated into the finalized Program Action Plan to be implemented the following semester.

·       When revised, a final copy of the program’s Action Plan shall be submitted to the College Dean/Associate Dean and the OIE Provost Designee.

Step 8: Implementation of Action Plan and Annual Update and Follow-Up:

·       The final Action Plan must be implemented by the program the following semester. So, if a visit occurs in Spring, Fall of the same year would be the implementation start-date.

·       Ongoing program faculty conversations should occur on the implementation of the Action Plan.

·       The Program Review Coordinator shall provide annual follow-up progress reports (endorsed by the College Dean/Associate Dean) in a platform and/or reporting venue set up by the Office of Institutional Effectiveness (OIE).

This concludes our presentation. If needs for clarification or questions exist, do not hesitate to contact us at institutionaleff@bgsu.edu