This program of evaluation and dissemination will provide the raw material and the setting for research projects designed to explore how students approach research tasks and how we can create better conditions for learning in a complex print and electronic information environment. This effort will have three kinds of outcomes: we will learn how to improve teaching on our campus, we will develop effective teaching practices that can be adapted for use in other academic settings, and we will use the project as a laboratory to develop new theory about how students learn. We feel this process of evaluation and dissemination is key to improving and sustaining the impact of the program both locally and nationally.
We intend to build on a study done by Barbara Fister at Gustavus in 1990 exploring the special characteristics of the undergraduate research process. The previous study challenged the standard process approach taught in library instruction programs. It will be exciting to return to the questions raised in that study and focus on how students cope in a hybrid print and electronic environment to expand our understanding of how we can best teach research skills in the current information environment. The library annually awards a competitive scholarship to a student to undertake a research project of benefit to the college; we may be able to design a series of interviews to be carried out and analyzed by a student.
One major component of the faculty summer programs will be considering how to measure student outcomes. The information we gather will not only be useful on the micro-level (such as how to improve a single classroom exercise) but also can serve as the basis for some larger-scale research projects undertaken by faculty and librarians involved in the project. It will also provide an opportunity for librarians to develop new means of measuring student outcomes. This change in the paradigm of evaluation of academic libraries from counting volumes and transactions to measuring how a library program affects student learning, is a challenge that all academic libraries struggle with as accreditation standards evolve and as the focus for measuring excellence shifts from size of collection to the library's impact on the curriculum. Our project will consciously address those issues and will seek to develop methods that can be applied at other colleges.
Gustavus faculty are productive scholars, interested in improving their teaching as evidenced, in part, by the current focus on institution-wide assessment of student outcomes. We are engaged in a campus-wide discussion of assessment that includes the development of department-by-department student outcomes and measures for discovering how well we meet them, in preparation for our next North Central Association accreditation visit. This is a critical moment for faculty in the pilot program to learn more about how to assess student learning of the research process and to incorporate undergraduate research outcomes in their departmental assessment plans. Faculty will have the opportunity to learn methods of classroom assessment that will provide them with quick, practical feedback, and will be encouraged to integrate their findings into department-wide assessment and goal-setting. We will also assist faculty research into pedagogical issues in their discipline, encouraging them to share what they learn with colleagues in their disciplines through conference presentations and publications.
In the library we will compare survey responses from several years' worth of student and faculty evaluations of traditional library instruction sessions with surveys gathered from classes involved in the program. This should give us an overall notion of how students in general respond to the new model and the librarians' role as part of a collaboration. We will also be able to explore how electronic resources and a hands-on computer classroom have affected student attitudes.
Completed assignments will also provide raw material for assessing the program and for developing new methods of assessment that can be adopted in other settings. The project leaders will work with faculty to collect a sample of completed research assignments from students in the redesigned courses and work up a protocol for assessing aspects of the students' research abilities, from choosing appropriate evidence to how well they use that evidence in writing. A heuristic for evaluation will be a useful assessment tool for teachers as they grade papers. It certainly would help us discover whether, across the board, we have particular strengths or weaknesses that need to be addressed in future faculty development activities. We can compare these with a sample of completed research assignments from courses that are not intentionally teaching research skills in this manner and see if our program leads to significant improvement. We could also use this data to examine differences encountered among disciplines and at different course levels. Beyond applying this data to local situations, we will use this experience to work toward developing a heuristic for measuring student research skills in context that will be applicable in other institutions to measure student outcomes. Academic librarians engaged in instruction have lacked a means of assessing their work in context; evaluation of our efforts tends to focus on testing and on evaluating bibliographies rather than on assessing how well information has been put to use in practice. A heuristic for this kind of assessment could be a useful tool for other library instruction programs interested in assessing their effectiveness.
We will collect "before and after" syllabi and research assignments from the faculty in the pilot program and will map out what changes were made in terms of teaching the research process and addressing the special characteristics of research in a print and electronic environment. We will follow up with structured conversations with the faculty in our reunions to get their sense of how well they feel we are meeting the outcomes above or any others that they have individually set. Out of an examination of syllabi and assignments we will create an evolving "best practices" clearinghouse for our faculty to share assignments, ideas, and assessment tools. The information gained from syllabi, assignments, and structured conversations will help us arrive at effective models for integrating research processes into the curriculum that could be presented to both librarians and teachers in the disciplines through conference presentations and in such publications as Research Strategies, College Teaching, The American Biology Teacher, and College Composition and Communication.