Our interpretation of the major issues and concerns that resulted in this recommendation:

Institutional Research Issues:
1. The Office of Institutional Research is in disarray and unable to provide reliable and useful information needed for institutional assessment and planning. What are the reports for which OIR is responsible? When are they due? What is done with the data? What data do the colleges, administration, and other units on campus desire and/or need? We need to draw a clear distinction here between the three primary types of information that OIR provides, that is, accountability-driven reporting information, institutional- level planning information, and assessment-driven college- and program-level information.
2. The Alumni surveys as currently designed, do not meet the needs of the educational programs. What information is solicited in the current Alumni surveys? What additional information is desired/required by the various educational programs? How can duplication of effort be eliminated by having OIR conduct these surveys? This comment speaks to the lack of clear distinction between the three types of information OIR provides. The Alumni and GS surveys were not originally designed to provide assessment-driven data but rather planning and perhaps reporting data. They are crude and imprecise tools for program-level assessment, and should be replaced by more specific and useful measures.
3. The OIR website is inadequate and has not been updated to include program assessments and various statistical reports. What information should be available on the OIR website and available to the university community? Could this information on the website reflect the tripartite structure suggested above?
4. Analysis of student data, originally the responsibility of Institution Research, has been transferred to the Registrar’s Office. Enrollment Planning has increasing responsibility for student recruitment and retention, should this responsibility and the analyst position be transferred back to OIR?

Program Assessment and Review Issues:

5. The annual program assessment reports vary considerably in depth, detail, and usefulness across departments, with little consistency in data elements, analysis, and use. What is the institution’s program review process? What is the process by which departments/programs develop their assessment plans and produce their annual assessment reports? What is the most effective motivator to ensure participation? What role does the dean play in review of these plans and reports? Once the reports are received by OIR, what is/should be done with them? How are/should annual assessment used to improve program quality? All of these issues are being directly addressed under Recommendation 5.
6. The Handbook for Academic Program Assessment is outdated and inadequate. What information should be included in a revised Handbook? Already addressed under Recommendation 5, by replacing old handbook with two new handbooks, Program-based review and assessment and Course-based review and assessment.
7. There are no institutional incentives to encourage assessment of programs (both undergraduate and graduate) to identify those with lower enrollment for possible elimination, as a way to make the best use of limited resources. Is this view (i.e., elimination of low enrollment programs) compatible with the institution’s educational philosophy and objectives?
8. Assessment of student learning outcomes and program effectiveness at both the undergraduate and graduate levels is in various stages of development. Which programs are most fully developed in terms of assessment of learning outcomes? How can the database being developed in the College of Education be expanded to other colleges’ programs? How can assessment-driven data tools be developed by/through OIR and used to further develop assessment and program review processes? How can emerging assessment technologies (such as ePortfolios) be piloted and expanded to programs for which it would be suitable?
9. The relationship between program level assessment policies, procedures, and documents, and the university’s institutional assessment plan remains in transition. This will be addressed directly under recommendations 1 and 5.
10. There is no clearly articulated institution-wide assessment plan. How many non-academic units engage in assessment? Is assessment tied to units’ identified objectives? to its strategic planning? This will be addressed under recommendations 1 and 5; one of the key considerations concerns the need to engage co-curricular programs as well as the range of academic programs.

Information Systems Issues:

11. The lack of a relational database management system has exacerbated the problems in Institutional Research in accessing reliable data, needed for assessment and planning. The assessment-driven data needs analysis should help clarify the various types and levels of functionality we should be addressing here.
12. The university’s automated degree audit program to facilitate the advising process is non-functional. How would a relational database management system solve the degree audit problem? How would it make possible access to data needed by OIR to carry out its responsibilities? How would it provide the data needed for effective, integrated institutional planning? for financial planning?

Institutional Planning Issues:

13. Allocation of resources (e.g., graduate assistantships, library resources) to departments/programs appears to be based on historical patterns rather than a planning process based upon a vision for a college and the institution. Does each college have a strategic plan? Is it compatible with the university’s strategic plan? How are resources allocated to the colleges and departments/programs? Are resources allocated based on identified need or primarily by tradition? Do the decisions made regarding these assignments reflect the information gathered from institutional, college, and program level program reviews?
14. Faculty in some departments are concerned about the over reliance on non- tenure track lecturer and adjunct faculty to teach lower division courses. What is the ratio of tenure track to lecturer and adjunct faculty in Arts and Sciences departments? What is an “acceptable” ratio? How are new tenure-track faculty positions allocated to departments? What role do lecturer and adjunct faculty play in their departments? How can OIR gather this data?
15. The increase in doctoral program enrollments reflects stated institutional objectives, but the university did not plan for a decline in master’s program enrollments. Institutional directives to expand graduate programs have not been clearly linked to the institution’s vision and mission. What data are needed to identify changing patterns in graduate enrollments? How can the changing patterns in master’s and doctoral enrollments be explained? How can this all be linked directly to institutional, graduate school, and programmatic planning?
16. With budget and planning institutionalized at the college and department levels, some confusion persists over the institutional budgetary process. How is financial planning done? What data is used? Which administrators are involved in the process? How do college and department budgetary plans inform institutional planning? Some of this will be clarified under Recommendation 1.

How this recommendation links to other recommendations?

Recommendation 2 is tightly linked to several recommendations – recommendation 1, institutional planning, recommendation 3, integrated database management system, and recommendation 5, institutional assessment and planning.

What has already been done to address this Recommendation:

Reorganization and expansion of Institutional Research, including elevation of Director to AAVP level; hiring of an Associate Director, Data Integrity Officer, Assessment Coordinator, and full-time secretary (in addition to the technical records specialist and work study students). Increase in budget for OIR.

Revised Program Evaluation and Course Evaluation Handbooks have been developed.

What activities are currently underway?

1. Identification and completion of the routine reports OIR is responsible to produce.
2. Identification of barriers that impede access to databases.
3. Identification of the data dictionaries for the various databases on campus.
4. Identification of assessment-driven data needs at the college and program levels.
5. Identification of emerging assessment technologies/resources and commitments to pilot their use in selected colleges and programs.
6. Identification of assessment resources and personnel across the institution who can serve as OIR and/or institution-wide resources.
7. Organization and structure of assessment-driven aspects of new OIR website.
8. Banta workshop on institutional planning and linking that planning to institutional and programmatic assessment (May 10).
9. OIR personnel will attend the Association of Institutional Research Convention in San Diego, May-June 2005 to gather resources, information and expertise on IR functions.

What needs to be done to meet this recommendation?