Office of the Provost

Academic Program Review for Graduate Programs


Introduction

This document explains the purpose of, and procedures for, academic reviews at the University of Delaware. Academic Program Review and Planning (APR) is a function of the Office of the Provost in conjunction with the University Faculty Senate and is coordinated by the Vice Provost for Academic and International Programs. It is designed to evaluate the quality, productivity, and role of each academic unit and program in the fulfillment of the University's mission and strategic goals. APR serves to encourage self-study and planning within units, to ensure comparability among review reports, and to strengthen the linkages connecting the planning agendas and practices of individual units with those of their colleges and of the University as a whole. Reviews inform budgetary planning decisions at every level of administration. Workloaded policies must also be reviewed as part of the unit's APR and/or periodic review for accreditation or re-accreditation. Although this document has been designed for the review of academic units within colleges, it can also serve as a model for the review of other types of units that exist within the University of Delaware. 


The Process
  1. The Provost and Deans select units to be reviewed and submit lists to the VP-AIP for coordination. 
  2. Review Panel appointed; VP-AIP appoints Panel Chair and sets schedule. 
  3. Unit conducts thorough "self study"; and submits to VP-AIP and Dean for distribution to panel. 
  4. Panel reviews self-study; may request additional information from unit. 
  5. Panel conducts 2-day review of the unit and prepares report to VP-AIP. 
  6. Department prepares formal response to panel report and submits to VP-AIP. 
  7. VP-AIP, Dean and Unit Head hold "follow-up" meeting to discuss final APR report (self study, review panel report, unit response), particularly recommendations and steps needed to meet goals. 
  8. Dean submits report on timeline for implementation of recommendations and how unit fits into college strategic plan.
  9. Panel report, departmental response and dean's report reviewed by Provost and Faculty Senate Academic Priorities Review Committee.

Organization and Responsibilities

Academic units will normally be reviewed at ten year intervals, but this schedule may be accelerated in individual cases at the discretion of the Provost. Where possible and desirable, University reviews will coincide with accreditation and other externally imposed reviews, or with reviews for new degree programs. The Faculty Senate Academic Priorities Review Committee will have the opportunity to suggest programs or programmatic areas for review. The VP-AIP coordinates all reviews, working with the unit under review and with the review panel to ensure that the process will be fair, efficient, and effective. 

Each review is conducted by a REVIEW PANEL. The panel will normally be composed of five members (typically four external members and one internal UD faculty members), all from disciplines related to that of the unit under review. The Review Panel members will be selected as follows: 

  1. one University of Delaware member or external member chosen by the Dean of the college whose unit is under review, or, in the case of the review of a college, by the Provost. 
  2. one University of Delaware member chosen by the Faculty Senate Committee on Committees and Nominations.
  3. one University of Delaware member from the faculty of another unit or external member chosen by the unit under review. 
  4. two external members representing the discipline of the unit under review chosen by the VP-AIP. 
The VP-AIP will appoint one of the panelists to be chairperson of the panel. After consulting with the chair of the panel and the head of the unit to be reviewed, the VP-AIP will establish the time of the review in advance and will set a schedule for the accomplishment of the review. The office of the VP-AIP is also responsible for handling transportation and hotel accommodations for external reviewers. 

The review process can be summarized into three parts: self-study, review, and follow-up. In preparation for the review, the unit will undertake a self-study based on the guidelines that appear later in this document. The self-study phase sets the agenda for all that will follow. It establishes the unit's responsibility for its own planning and goal setting within the context of college and University priorities. 

Panelists should read the self-study carefully and are encouraged to request additional materials as needed. The VP-AIP will work to ensure that representatives from diverse constituencies within the unit are included in the review process. During the site visit, University of Delaware panelist(s) are expected to participate in the full schedule of events. All panel members will participate in the review and in the formation of the panel's written report. It is the responsibility of the review panel chairperson to ensure that the internal and external members of the panel work together throughout the review and that the final report reflects the interaction of all members of the panel. 

The effectiveness of the review will depend on the implementation of the recommendations contained in the report. Since implementation will be tied directly to the report's incorporation of the goals that have been defined for the unit in college and University planning agendas, it is essential that the panel approach its charge from the perspective of the unit's role within the University. The review panel's report should be completed within one month of the review and forwarded by the review Panel Chairperson to the VP-AIP. A model schedule for reviews and format for panel reports are included in this document. 

Upon receipt of the review panel's report by the VP-AIP, it is distributed to the Dean, head of the unit, and department faculty for review. The department then provides a formal response to the report and submits it to the VP-AIP and Dean. 

The panel report and department response are discussed at a follow-up meeting with the VP-AIP, the Unit Head, and the Dean of the college in which the unit is located. The purpose of this meeting will be to discuss the recommendations of the report and to agree on what steps will be taken by the unit and by other administrators to reach University, college, and the unit's goals as contained in the report. The results of this meeting are summarized in a written document to be prepared by the Dean. This document will become the blueprint for planning and budgeting decisions regarding the unit and it will be distributed to the unit's faculty. The panel report, department response, and dean's report will be reviewed by the Faculty Senate Academic Priorities Review committee during its regularly scheduled meeting with the Provost and VP-AIP. 


Self-Study Guidelines

The self-study document describes the unit as it has evolved and is presently constituted and indicates its aspirations for both short-term and long-term development. It should place the unit within the context of college and University planning priorities and of developments within the unit's discipline; and it should address the major issues confronting the unit and suggest how they might be resolved. The preparation of this document must necessarily be based upon open discussions that include all faculty (and in some cases professional staff) in the unit. 

The self-study document shall include the following elements (where appropriate): 

  1. A statement of the current status and future aspirations of the unit.
    • Identify one or two departments in other institutions that may be similar to the unit being reviewed at present.
    • Identify one or two departments in other institutions that can serve as a model for future growth of the unit.
    • What factors were used in comparing the unit to academic departments in other institutions?
    • In comparison to similar departments at other institutions, what are the unit's major strengths and weaknesses?
  2. A statement of the purposes, development and planning for undergraduate education in the unit.
    • Description of undergraduate curricula and programs for majors and non-majors.
    • Statistical data on course enrollments, class size, and number of majors and number of minors.
    • Description of the unit's role in meeting University curricular needs, such as group requirements, multicultural courses, honors courses, and prerequisites for other programs.
    • Information about student quality and achievements.
    • Description of the advisement plan for undergraduate students and analysis of who provides advisement.
  3. A statement of the purposes, development and planning for graduate education.
    • Description of the graduate curriculum.
    • Statistical data on the size of graduate program(s), including number of students in masters and doctoral programs, and number of funded students and source of funding.
    • Quality of graduate students as defined by numerical indicators and career placement of graduates.
    • Career placement of graduates relative to market conditions.
    • Description of the advisement plan for graduate students and analysis of who provides advisement.
  4. Staffing of the unit.
    • A statement of the unit's present demographics.
    • Breakdown of FTE faculty, professionals and salaried staff by rank, function, workload, race, age, and gender.
    • Brief biographies of all unit faculty and professionals.
    • Information concerning the use of non-faculty instructors (if any) in the unit, either teaching assistants or supplemental contract holders.
    • The Office of Institutional Research and Planning will within reason provide demographic data to set the unit into a University-wide context.
    • Faculty and staff development and training activities provided by the unit or by external sources.
  5. The research and scholarly productivity of the unit.
    • Analysis of the relation of research productivity to faculty workload, teaching, and support of graduate students. 
    • Faculty Promotion and Tenure policy.
    • Evidence of faculty/professional staff productivity.
    • Data on the sources and amount of external research funding.
    • Description of the unit's relationship to University research centers and to other interdisciplinary research groups.
    • Regional and national professional activities of the faculty, staff and students.
  6. The public service function and productivity of the unit.
    • Statement of the unit's role in public service.
    • Evidence of the public service activity as expressed in contract research for government or non-profit agencies.
    • Evidence of other forms of public service, such as speaking in schools, assisting in social service agencies, government agencies, hospitals, cultural organizations, and similar activities.
  7. The administrative structure and function of the unit.
    • Description of duties, responsibilities and reporting relationships of key administrative positions.
    • Charges, operating procedures, and reporting relationships of standing committees.
    • Brief analysis of the leadership initiatives of the incumbent administrators and committee chairs.
    • Procedures for filling administrative positions and committee seats.
    • Leadership training activities (provided by the unit or by external sources) for unit leaders.
    • Description of research centers and other ancillary units administered by the academic unit under review, along with analysis of the reporting relationship and extent of oversight of the ancillary unit.
    • Analysis of working relationship with the college dean's office and with other units within the college and within the University, including analysis of services that are shared.
  8. Information concerning the University service of the unit's faculty and staff.
  9. A statement of the unit's accomplishments in support of the University's goal to create a diverse faculty, staff, and student population.
  10. A description of the unit's facilities, including space and equipment.
  11. A statement of University support for the unit, including factors such as library resources, research, equipment, graduate, and staff support. 
  12. The goals and costs associated with the unit's planning to meet current and future needs, and the relation of the unit's plan to college and University planning goals. This statement should incorporate proposed changes to either reduce or to increase the unit's resources and to justify these proposed changes.
  13. Review of workload policies.

Typical Review Schedule

The purpose of the suggested review schedule is to ensure that reviews are thorough and fair. All members of the unit under review should have the opportunity to interact with the review panel. Reviews normally begin with a dinner hosted by the VP-AIP on the evening that the external panelists arrive and attended by the members of the panel and the Dean of the college. The dinner serves as an orientation to the review and gives panelists the opportunity to ask questions about the unit. 

The panelists will meet with selected faculty leaders from the unit:  possibly members of the unit's steering committee, chair of its major committees, or its full professors. The panelists are escorted by designated unit faculty on a tour of the unit's facilities and other campus facilities pertinent to the unit's operations. 

The panelists meet with the faculty responsible for the unit's academic programs. Appropriate representatives from the dean's office, VP-AIP, and other university-wide offices also participate as needed. 

During the review, time should be set aside for individual faculty and professional staff to meet with panelists. The panel also meet with undergraduate and graduate students from the unit both in individual sessions and as a group. 

It is important that there be an attempt to include all faculty of the unit in some portion of the review. To expedite the investigative rather than social nature of all aspects of the review, it is suggested that faculty be included as members of diverse constituencies within the unit, such as unit committees or sub-disciplines. 

The review team may also meet with the Provost or the person designated by the Provost. Members of the committee may also conduct any other interviews that they, the VP-AIP, the Dean, or the Unit Head will request. 

The review will include time for the panelists to meet alone to outline their report and to distribute the writing responsibilities. The report is due one month following the completion of the review. At the conclusion of the review, the team will hold an exit interview with the Provost (or designee), Dean, and VP-AIP. 


The Review Report

The report should place the unit under review in the larger context of University priorities and of developments in the unit's discipline. It should take account of the unit's role within the University. It should address the major issues facing the unit, comment on the compatibility of the unit's purpose, achievements, plans and goals with those of the University and college mission and planning documents, and suggest strategies for achieving unit and University goals. To accomplish these purposes the report should consider the following points as appropriate to the mission of the unit. 

Competitiveness

  • Identify one or two departments in other institutions that may be similar to the unit being reviewed.
  • Identify one or two departments in other institutions that can serve as a model for future growth of the unit being reviewed in the next five to ten years.
  • In comparison to similar departments in other institutions, what are the unit's strengths and weaknesses?
  • What will the unit have to do to achieve or maintain national or regional competitiveness in the next decade?
Undergraduate Program
  • How well is the unit performing its undergraduate teaching function?
  • Do undergraduate students receive appropriate mentoring and advisement?
  • Is the curriculum sound and sufficiently rigorous?
  • Is the unit properly staffed to fulfill its undergraduate responsibilities?
  • Are classes the appropriate size to accomplish its teaching goals?
  • Is the unit fulfilling its responsibilities to non-majors with regard to the general education initiative, distribution requirements, multicultural courses, the Honors Program, and prerequisites?
Graduate Program
  • How effective is the unit in performing its graduate teaching responsibilities?
  • Is the research and scholarly productivity of the unit's faculty appropriate to its graduate responsibilities?
  • Are the graduate program's admissions criteria appropriate?
  • Do graduate students receive appropriate mentoring and advisement?
  • How successful are the unit's graduate programs nationally and regionally in attracting qualifies graduate students and placing graduate degree holders in professional employment?
  • How competitive are the unit's graduate programs nationally and regionally in attracting qualified graduate students and placing graduate degree holders in professional employment?
  • Is the curriculum sound?
  • Are students receiving faculty mentoring and assistance in finding professional employment?
Faculty Research and Scholarship
  • Are the research, creative activity, and scholarship of the faculty appropriate to the unit's mission and overall responsibilities with regard to quality and quantity?
  • Are research facilities and library resources sufficiently supportive of faculty research?
  • Are faculty generating external funding to the degree that they might?
  • What role are faculty playing in the University's research centers and interdisciplinary research groups?
  • Are the faculty engaged in regional and national professional organizations?
Faculty and Staff
  • How well are faculty and staff resources being used?
  • Are promotion and tenure policies appropriate to the unit's missions and aspirations?
  • Is the unit successfully hiring and promoting minority and women faculty?
  • Are faculty and staff workloads equitable and appropriate to the unit's missions?
  • How does the unit rank among those in similar institutions regarding research productivity and quality, external funding and teaching loads?
  • Are staff positions and expectations clearly defined?
  • Does the unit provide development and training programs to faculty and staff?
Leadership
  • Does the unit's leadership take appropriate and timely action to ensure the unit's smooth functioning?
  • Does the unit's leadership interact appropriately with other University units, including the college dean's office?
  • Does the unit have an effective leadership development program in place?
  • Does the unit have a mission statement and long range plan that are endorsed by the faculty and that are used as the basis for annual planning?
Public Service
  • Is the unit meeting its public service obligations?
  • Is it performing a satisfactory amount of public service research and assistance?
  • Is it, where appropriate, making the effort to introduce students to professional public service opportunities?
Diversity
  • Is the unit taking appropriate steps to meet the University's goal to achieve a diverse faculty and student body, to offer multicultural courses, and to promote respect for all people?
University Citizenship
  • Is the unit providing good University citizenship?
  • Do its members encourage and contribute to interdisciplinary activities?
  • Should it concentrate its efforts and resources in a different way in order to create the greatest possible synergy throughout the University?
University Support
  • Is the unit receiving adequate support from its college and from the University at large in the context of budgetary constraints affecting higher education in general.
  • Are library and other resources appropriate to support the unit's programs?
Plans, Goals, and Resource Allocation
  • To what degree is the unit central to the mission of the University and of the unit's college as reflected in University and college mission priorities?
  • How do the unit's plans and goals serve to fulfill its mission?
  • Is the unit trying to do too much?
  • What, if any, of the unit's requests for additional resources does the panel support, and why?
  • How might the unit's resources be redistributed to realize its goals and those of the University?
v
Check List of Responsibilities
Academic year prior to site visit
Units selected for review.
Prior to site visit
VP-AIP, Dean, and Unit Head meet to discuss timetable and composition of panel. 
Prior to site visit
Unit Head communicates review plans to unit faculty.
Prior to site visit
Dean of college whose unit is under review (or Provost, in the case of a review of a college) appoints one University member or external member to serve on review panel.
Prior to site visit
Faculty Senate Committee on Committees and Nominations appoints one member of the University of Delaware to serve on the review panel.
Prior to site visit
VP-AIP appoints two external members to review panel.
Prior to site visit
Academic unit under review appoints one faculty member from another UD unit or external member to serve on the review panel.
Prior to site visit
VP-AIP appoints one of the review panelists to serve as chair.
Prior to site visit
VP-AIP, Review Panel Chair and Unit Head establish time and set schedule for review process.
Prior to site visit
Unit under review conducts self study and prepares self study report; submits report to Dean and VP-AIP at least 6-8 weeks prior to site visit.
Prior to site visit
VP-AIP confirms travel/accommodations for external review panel members; confirms schedule for site visit.
1 month prior to site visit
VP-AIP distributes self study document and other reports to review panel members. 
2-4 weeks prior to site visit
Review Panel reviews self study and Dean's assessment, requesting additional materials as necessary.
Ongoing
VP-AIP ensures that representatives from diverse constituencies within the unit are included in the review process.
Ongoing
Review Panel Chair ensures that internal and external review panel members work together throughout the review process.
After site visit
Review Panel formulates written review panel report.
After site visit
Review Panel Chair ensures that review panel report reflects interaction of all panel members; submits report to VP-AIP.
After site visit
VP-AIP distributes review panel report to the Dean, Unit Head, and unit faculty.
After site visit
VP-AIP schedules follow-up meeting with Dean and Unit Head.
After site visit
VP-AIP, Dean, and Unit Head participate in follow-up meeting to discuss report recommendations and determine steps for implementation.
After site visit
Dean prepares written summary of follow-up meeting and distributes to Provost, VP-AIP, and Unit Head.
After site visit
Provost and VP-AIP meets with Faculty Senate Academic Priorities Committee to discuss outcome of unit's APR.

Academic Program Review for Graduate Programs


Based on the self-study report and the program site-visit, the review team is requested to evaluate and comment about the specific areas enumerated below, if applicable, in making its evaluations of the unit's strengths, weaknesses and overall quality. NOTE: Each item should be compared and, where appropriate, contrasted to national standards or norms as reflected in similar peer programs at other and/or in accrediting agency guidelines.

  1. Unit Graduate Goals and Curriculum

    1. The clarity of unit graduate goals, objectives, and rationale as expressed in the self-study report and as perceived in the interviews with the faculty, students, and administration. Also comment on whether the unit's goals are consistent with University and College academic priorities and goals.
    2. The quality of the graduate curriculum in fulfilling the goals and objectives of the unit. Also provide some comment on the quality of the instruction.
    3. Is there evidence that the unit has clear goals on graduate student learning outcomes, assessment process(es) are in place and that the results are being utilized.
    4. Any unique characteristics that you identified in the unit such as unique research or training opportunities, interdisciplinary components etc.
    5. The appropriateness of the graduate training as measured by the placement of the unit's graduates and your perception of regional/state/national needs for graduate of the program.

  2. Faculty

    1. The quality of the graduate faculty as to their ability to provide quality instruction, supervision, and outstanding research opportunities to the students.
    2. The overall research strength of the graduate faculty as compared to faculty in similar programs at peer institutions.
    3. The morale and collegiality of the graduate faculty, and any impact that these traits are having on the students and the program.

  3. Students

    1. The admission standards and procedures for admitting students. Also comment on the quality of the graduate students enrolled in the program.
    2. The overall administration of the unit as it relates to the students. Also comment on the adequacy, appropriateness and clarity of the program guidelines as to mentoring, selection of research (or studio) projects, overall supervision, and requirements for students completing the program, etc.
    3. The morale and perceptions of the students as related to the unit, the college and the University.

  4. Program Productivity

    1. Program productivity as indicated by the graduation numbers, retention, time to degree, enrollments trends etc.
    2. Student productivity as indicated by publications, presentation at local/state/national/international meetings, shows, exhibits, awards, etc.
    3. The appropriateness of the placement of the graduates of the program.

  5. Adequacy of Financial Support for the Program

    1. The number of graduate assistantships provided by the program from both intramural and extramural funding, as well as the level of remuneration provided to the graduate assistants.
    2. The appropriateness of the extracurricular workload (i.e., teaching and non-degree related work) required of those students receiving graduate assistantships.
    3. The adequacy of extramural funds (i.e. grants, training grants, etc.) to support the program. Also comment about whether the department is using extramural funds in the most effective manner to support the students.
    4. What, if any, of the unit's requests for additional resources does the panel support, and why? How might the unit's resources be redistributed to realize its goals and those of the University?
    5. Evaluation of resource requests and recommendations must be framed under three budget scenarios, i.e., a 20% reduced budget environment, constant budget environment, and a 20% increased budget environment.

  6. Quality and Adequacy of Facilities

    1. Laboratory and studio facilities (where appropriate).
    2. Equipment (including instruments).
    3. Library resources
    4. Computer resources
    5. Office and classroom space.
    6. Overall environment in regard to contributing to the intellectual development of students and faculty.

  7. Competitiveness

    1. Identify one or two graduate programs in other institutions that may be similar to the graduate program being reviewed.
    2. Identify one or two graduate programs in other institutions that can serve as a model for future growth of the graduate program being reviewed in the next five to ten years.
    3. In comparison to similar graduate programs in other institutions, what are the unit's strengths and weaknesses?
    4. What benchmarks should be used to measure the unit's effectiveness and efficiency in the use of its resources?
    5. What will the unit have to do to achieve or maintain national or regional competitiveness in the next decades?

  8. Leadership

    1. Does the unit's leadership take appropriate and timely action to ensure the graduate program's smooth functioning?
    2. Does the graduate program's leadership interact appropriately with other University units, including the Office of Graduate Studies and Research Office?
    3. Does the graduate program (if applicable) have a mission statement and long range plan that are endorsed by the faculty and that are used as the basis for annual planning?

  9. Public Service/Outreach

    1. Is the graduate program meeting its public service obligations?
    2. Is it performing a satisfactory amount of public service research and assistance?
    3. Is it, where appropriate, making the effort to introduce students to professional public service opportunities?

  10. Overall Conclusions About the Graduate Program

    1. What two things did you find most commendable about the program? (You may list more if warranted)
    2. What two things were of the greatest concern to you about the program? (You may list more if warranted)
    3. What major changes would you recommend in the program?
    4. What, in your opinion, is the maximum student capacity of the program relative to the current intramural and extramural funding, the available facilities, and the capacity of the graduate faculty to provide competent instruction and supervision to the students in both the classroom and in research (or studio) activities?
    5. Should the program be continued based on the variables that you evaluated in this report?