Student Achievement Assessment Committee

The Department of Physics and Astronomy developed the following learning goals during the past year, that at the completion of baccalaureate degree studies in Physics, students will be able to:

·         demonstrate a thorough conceptual understanding of the basic fields of physics;

·         use mathematics to describe and manipulate fundamental physical constructs and to solve problems;

·         use computational methods to solve physics problems;

·         use basic experimental apparatus common to the study of physical phenomena;

·         communicate scientific ideas effectively, both orally and in writing.

Much of the Department's effort this past year went into developing these learning goals and starting to develop additional strategies for assessing them. Our development and implementation are at various stages of completion.

The most mature assessment area regards the goal of a thorough conceptual understanding of the basic fields of physics. Our foremost specific goals include Newtonian understanding, Einsteinian understanding, and understanding of electricity and magnetism. Of lower priority are additional subject areas, such as waves, optics, and kinetic theory. Student assessment is performed routinely in the courses devoted to these topics, as described more carefully below. Additional assessment of understanding may also be performed outside of individual classes. Many content areas are included on the physics GRE, although they are not well separated, making their use as assessment tools problematic. Very recently we became aware of the Major Fields Assessment Test (MFAT) in physics from the Educational Testing Service. We are in the process of evaluating the usefulness of these standardized tests for assessment. As part of a dual Physics-Education master's thesis, one of our graduate students is planning to develop an assessment tool for Einsteinian thinking.

The goal of conceptual Newtonian understanding can be assessed using the nationally recognized Force Concept Inventory (FCI) test. This test is administered in Physics 201, our largest service course, plus Physics 211, which includes our majors. The test is given twice in each class, as a pre-test near the beginning of the semester and then again as a post-test near the end, so that progress can be directly measured. It should be noted that the number of participants typically decreases from the pre-test to the post-test because of drop-outs, etc. We have now accumulated data for over several years. The data are listed below:

Semester                    Number           Pre-test           Post-Test             Improvement              Gain

Fall 1997                      271/251           35% +/- 1%     43% +/- 1%                 23%                 .12

Spring 1998                 171/154           34% +/- 1%     41% +/- M                   21%                 .11

Summer 1998               49/42               35% +/- 2%     46% +/- 2%                 31%                 .17

Fall 1998                      147/131           34% +/- 1%     41% +/- 1%                 21%                 .11

Spring 1999                 140/134           35% +/- 1%     40% +/- 1%                 14%                 .08

Summer 1999               35/29               32% +/- 2%     43% +/- 3%                 34%                 .16

Fall 1999                      144/133           36% +/- 1%     41% +/- 1%                 14%                 .08

Fall 2000                      245/179           36% +/- 1%     43% +/- 1%                 19%                 .10

Honors (1998-00)        42/42               47% +/- 2%     59% +/- 2%                 7%                   .24

The Gain is equal to the ratio of the percentage improvement divided by the total possible

percentage improvement

Gain = Post percent - Pre percent 

             100 percent - Pre percent

which takes into account the understanding of students as they enter as well as when they exit the class.

One of the advantages of the FCI is that national data are available. Reports in the literature for large, though heterogeneous, student samples from across the country indicate typical Gains of 0.16 t 0.03 to 0.23 f 0.04 for courses taught in traditional fashion as we do here. Larger Gains are found for courses taught using more interactive techniques: 0.41 ± 0.02 to 0.48 t 0.14. Our Gains are below the national average, and we would clearly like to improve in this area. The average post-test score is near 43%. This is far below the value usually claimed to represent Newtonian thinking (85%), or even the value achieved by instructors who have been applying modern pedagogical techniques (65%).

In response to these data, we have proposed changes in the way we teach the Physics 201-202 and 211-212 sequences, including shifting more contact hours from large lectures to smaller classes, and reducing the class size in those smaller classes. The data above support the conclusions in the literature that smaller classes (the Summer and Honors sections) and a more interactive style (Honors sections) do significantly improve student learning for our students. Unfortunately, we have not received the necessary resources to proceed with these changes. Although large classes provide difficult obstacles, we are currently making efforts to adapt some active learning exercises to our current large-class setting. A pilot section of Physics 101 was taught in Spring 2001 and a new graduate course for our Master of Arts in Teaching is being taught this summer with more active and hands-on oriented teaching styles, which the literature suggests are more effective.

All of our learning goals are assessed as a fundamental part of the major courses. The small upper level classes facilitate individualized assessment, so with moderate effort an instructor can maintain a relatively clear picture of each student's progress. A variety of tools are designed for assessment, including homework exercises, examinations, and classroom discussion. Conceptual understanding and especially mathematical descriptions and problem solving are strongly emphasized in nearly all modes of assessment. The description of the MFAT claims that it includes assessment of problem solving, and so we may find another tool there. The use of computational methods is specifically targeted in Physics 401 and 402, where students are given numerous opportunities to demonstrate their skills in this area. Other 400-level courses also incorporate students' use of computational methods, so their retention and ability to apply these methods are assessed over time. Students' use of experimental apparatus is developed and assessed in a series of required laboratory courses. Above the introductory level, students are given the opportunity to design and perform their own lab experiments, demonstrating their individual progress in this area. Students' communication of scientific ideas is an integral part of most courses in the major; the small classes make communication very common for each student. A primary means of assessing communication is the research experience, described next.

A required research project serves as a kind of capstone experience, involving most if not all of the learning goals. The details of the project are decided by the student and their advisor, so may or may not include computational or experimental methods, but the other goals are always involved. The final step in completing the project is for the student to present the results, usually in written form. For the first time this past Spring the results were presented orally as well as part of the Department's regular seminar series.