Student Achievement Assessment Committee

2002-2003 academic year

Learning Outcomes:

At the completion of baccalaureate degree studies in Psychology, students will:

1 Exhibit broad knowledge about human behavior from a variety of psychological perspectives (e.g., biological, cognitive, developmental, social).
2 Have the necessary skills in research and other forms of inquiry in order to develop new knowledge about behavior.
3 Be able to communicate their knowledge of psychology to others.
4 Have the necessary skills and content knowledge to be an informed and critical consumer of existing knowledge.
5 Be prepared for post-baccalaureate studies in psychology or related disciplines, or for entering the workforce in areas related to their training.

Annual Report

1. Learning (or Service) Outcomes assessed this year:

Two separate assessment activities were initiated during 2002-03:
A. An across-the-curriculum compilation of student perceptions of how well we have met our student learning outcomes. Where possible, we compared majors with non-majors. The outcome of these analyses is reported below.
B. Establishing specific learning outcomes for our introductory statistics course (PSYC 270) in order to ensure that students have the necessary skills to engage effectively in research and inquiry. This activity is ongoing.

2. Assessment Methods and Procedures:

A. Student Learning Outcomes Across the Curriculum.
Our Assessment of Student Learning (ASL) instrument is given to students during the last week of class. The 43-item survey asks students to indicate the extent to which they feel, for example, that they: learned to think critically about ideas in the course; developed a better understanding of themselves and others; learned how to express themselves orally or in writing; connected what was learned in the course with what was learned in other courses in the social sciences; or acquired skills that will be used in graduate school or their careers. Factor analysis of responses from approximately 4000 students over four semesters (Spring 2001 through Fall 2002) revealed, as did previous analyses using smaller datasets, that six factors account for nearly two-thirds of the variance in the responses. We refer to those factors as: Inquiry, Growth, Tools of Inquiry, Integration, and Career Relevance. (The second sentence of this paragraph contains sample items for each factor; see previous Assessment Reports for details about the instrument and the results of the factor analysis).

One of the original goals is developing the ASL was to develop a profile for each course to see how well our learning outcomes are met across the entire curriculum. We suspected that some courses necessarily focus on some learning outcomes more strongly than others, but we hoped that when aggregated across the required and elective courses that psychology majors take, all five learning outcomes would be achieved. With four semesters worth of data in hand, we developed a profile for all courses for which at least 25 completed surveys were available. Of the 28 different (non-Honors) courses offered during this two-year span, 18 met this criterion. Of the 18 courses for which profiles could be constructed, eight were courses in which a substantial number of non-majors were enrolled. For those eight courses, separate profiles were constructed for majors and non-majors. A factor analysis of the data from non-majors only showed that four factors accounted for nearly two-thirds of the variance in the responses. Not surprisingly, the Career Relevance factor did not emerge from the responses of the non-majors. (Although not surprising, this result lends support to the validity of the instrument.) The four factors that emerged for the non-majors were: Inquiry, Growth, Tools of Inquiry, and Integration.

Each individual item could have a score ranging from 1 to 5, indicating that students strongly disagreed (1) or strongly agreed (5), for example, that they learned to think critically about ideas in the course. A score of 3 indicates no endorsement either way. The average score for the items that were part of each of the six factors was computed (with equal weights given to each item on the factor regardless of its factor loading). The profile for a course is the average scores along each of the factors for that course. The 18 profiles are attached to the end of this report. For the eight courses in which a substantial number of non-majors were enrolled, the profiles for majors and non-majors are presented (majors are shown in the solid gray bars, non-majors in diagonally striped bars).

The following table is a summary of the profile analysis. Each column represents one of the five factors. Within each column is a listing of courses with the position on the list being determined by the average score for that particular factor, in descending order. That is, PSYC 350 had the highest score on the Inquiry factor, PSYC 309 the lowest. PSYC 406 had the highest score on Growth whereas PSYC 290 scored highest on Tools of Inquiry. Note that all 28 courses are included in the table, which means that the scores for 10 courses may not be particularly stable due to small sample size. Also note that we’ve attempted to give the reader a sense of the magnitude of the scores by presenting the course numbers in different fonts; see the Key to the right of the table to determine the range of scores that corresponds to each of the fonts used.

NOTE: Many of the courses listed here were taught by different instructors over the two-year period.

B. Establishing Specific Learning Outcomes for PSYC 270 Quantitative Methods I.
Our introductory statistics course is taught by at least three different (and sometimes up to five different) instructors each semester. This initiative grew out of the recognition that we have no standardized learning outcomes for the course. Because statistics is a prerequisite for the research methods course (PSYC 290) as well as for the junior-level labs in psychology (e.g., PSYC 328, PSYC 333, PSYC 320, PSYC 313), it is crucial that students coming out of the statistics course acquire the same core skills. Anecdotal reports from instructors of the courses for which PSYC 270 is a prerequisite indicate that students are not learning what they should in that course. When new instructors (particularly graduate students) are asked to teach the course, it would be useful to provide some guidelines as to what the students are expected to learn.

In late spring we asked current and past instructors of PSYC 270, as well as current and past instructors of the research methods course and the junior-level lab courses, to indicate what skills and concepts a student should expect to learn in PSYC 270. The data have been slow in coming in, so we anticipate making the request yet again, perhaps not using an open-ended format as we did initially. Three current PSYC 270 instructors, including one who teaches at Firelands, have expressed interest in working with the Assessment Committee to develop a set of learning outcomes for the course. We recognize that establishing standards for a particular course is not something that should be taken lightly. Accordingly, there is no plan to move towards a common exam for PSYC 270.

3. Inferences from Assessments:

A. Student Learning Outcomes Across the Curriculum.
Across all five factors and the eight courses for which the comparison between majors and non-majors can be made, it appears that our majors feel that they are learning slightly more than the non-majors. The difference, though, is so slight as to not be particularly meaningful.

More interesting are the results shown in the above table. First, we are doing well in the Inquiry learning outcomes: none of the courses received a score below 3.00 (which would indicate that students felt that they did not learn Inquiry skills), and 27 of the 28 courses had scores above 3.50, with nine scoring above 4.00. The average score on this factor across all courses was 3.82. We are also doing reasonably well in the Career Relevance outcomes, with 27 of the 28 courses achieving scores of 3.00 or above and an average score across all courses of 3.63. It is heartening to see that research methods (PSYC 290) ranked fourth on this factor, for it indicates that students see the importance of learning how to do research, regardless of what their future plans may be.

Similarly, it is reassuring that research methods ranked highest in the Tools of Inquiry factor, with our junior-level lab courses and advanced statistics course taking eight of the next nine positions. Learning outcomes in the Tools of Inquiry factor include learning how to write, how to do statistical analyses, how to use the computer to do research. Understandably, the courses scoring the lowest on this factor tend to be large lecture courses, the opportunity for students to become actively engaged in research activities in a large lecture course being reduced.

Not surprisingly, scores on the Growth factor were generally lower than those for the other factors discussed. Inter- and intra-personal growth is expected during college, but perhaps not as much from classes as from interacting with others. Of the top six courses in this category, four are courses in clinical psychology and one is introductory psychology, with courses in industrial-organizational, human sexuality, and developmental psychology not far behind. If we assume that students are rating the items on this factor on the basis of having learned “what makes people tick,” the rankings make sense. Consistent with that notion, the bottom two courses in the Growth category are introductory and advanced statistics, with the junior-level lab courses close to the bottom.

Most surprising were the scores on the Integration factor: eight courses were rated as not having made connections with other content domains and only two were given ratings above 3.50, the lowest such number across all the factors. (The ratings of those two courses are based on fewer than 25 surveys, so a more stable estimate of the score on this factor may be lower in magnitude.) Note that students feel that we are doing a decent job of ensuring that they connect what they learn in our courses with what they’ve learned from other courses in that a majority of the courses were rated 3.00 or above on this factor. The average score on this factor across all courses was 3.10.

Part of the reason for the comparatively low ratings on the Integration factor may be an artifact of how the items themselves are scored. In five separate items, students are asked to indicate the extent to which they connected what they learned in their psychology course with what they learned in courses in the natural sciences, social sciences, etc. The instructions for filling out the survey state that a score of (1) should be entered when responding to items that do not apply to the course being rated. Students who haven’t taken a wide variety of courses would enter a (1) for most of those items, with the result being that a low score on this factor could be as much a reflection of the courses that students have taken as it is a lack of opportunity to integrate knowledge across content domains.

Another possible reason for the low scores on Integration is that students may feel that the content and skills acquired in some courses are unique to the field. For example, PSYC 415, the psychology of religion and spirituality, is a course that is typically not found in undergraduate psychology curricula. Moreover, although there are several courses offered in the philosophy of religion at BGSU, the study of religion and spirituality from a psychological standpoint makes it unique among the courses at BGSU. In addition, courses in research methods and statistics have low ratings on this factor, and although what is learned in those courses has broad application, students may erroneously believe that what they’ve learned is useful only in psychology.

B. Establishing Specific Learning Outcomes for PSYC 270 Quantitative Methods I.
In progress.

4. Actions Taken/Program Improvements:
A. Student Learning Outcomes Across the Curriculum.
Other than presenting the results of this analysis to the faculty, no particular actions have been, or likely will be, taken as a result of this assessment. For one, it appears that the full complement of learning outcomes is covered in our curriculum. For another, it is not clear what an ideal pattern of course profiles should be so we don’t know what specific actions we should take. We will use these results as benchmark data for future across-the-curriculum assessments, although additional data will be needed from many of our courses to establish stable benchmarks.

B. Establishing Specific Learning Outcomes for PSYC 270 Quantitative Methods I.
None yet taken.