Student Achievement Assessment Committee

The Chemistry Department learning objectives place a strong emphasis on problem solving . Although this often takes the form of numerical solutions to specific problems, our primary goal is to develop analytical skills. We want our students  to be able connect data and observation to concepts and then to use those concepts to solve problems that differ significantly from previous experienced examples. This is difficult in a conventional introductory course where the student is expected to “learn” concepts that have been presented in lecture and to work though problems remarkably similar to previously worked examples.

In fall 2005 Dr. Endres initiated an alternative instructional approach for General Chemistry. This is a two semester sequence for science majors: Chem 125 (3 lectures, 1 recitation and a three hour lab) and Chem 127-128 (3 lectures, 1 recitation and a separately graded three hour lab.)

Two programs were studied prior to this trial. We visited summer classes at  Indiana University/Purdue University at Indianapolis to observe the operation of General Chemistry classes utilizing Peer Led Team Learning (PLTL.)  (Typically two hour recitations called workshops conducted by undergraduates. Students work in  small teams from worksheets containing inquiry questions and traditional problems.)    Dr. Endres attended a week long chemistry based POGIL workshop (Process Oriented Guided Inquiry Leaning)  in summer 2005. Both programs involve active learning and small group activity and are designed to help develop analytical skills.

POGIL is funded and promoted by the National Science Foundation and disseminated through the Pogil Foundation. (www.Pogil.org)  A POGIL based course has several key features
             Active student participation during class time
             Emphasis on working within a small peer group
             Classroom time is devoted to specific workshop activities
Students are presented with data and leading questions that ask them to discover relationships and figure out how to solve problems. Such activities are in clear contrast to a traditional lecture format, with an instructor presenting material to be learned; in POGIL the instructor functions as a facilitator. The textbook readings and selected problems are an important part of the course, and are assigned for completion after the workshop activity.

In most studies the POGIL method (and PLTL approach) result in a significant decrease of D-F and Withdraw grades but less change at the A-B levels. Students generally demonstrate improved analytical skills as well as mastery of content.

The goal in 2005 was to incorporate many of the key features of  POGIL. In practice there were a several important limitations that prevented a full implementation:
the class was larger than ideal and it was scheduled in a tiered lecture hall
textbooks and support materials had been selected prior to this decision
Many of the needed special materials were developed on a just in time basis.  Consequently only about 35% of the class time could be programmed for POGIL activities.
           
In an effort to maximize the effectiveness of the peer groups  we assigned students so that the members of the group shared a common recitation time and a common laboratory session. Students in a group were expected to sit together in lecture and in recitation and we arranged for them to shared adjacent laboratory lockers.  A group functioned together for about 475 minutes of class time per week.

A typical class session began with the distribution of printed materials and approximately 20 minutes was devoted to work on this project by each of the groups. The instructor made few if any opening remarks and then proceeded to observe activities of individual groups. The session might be interrupted to ask for progress reports. When two groups reported different results they might be asked to try resolve the conflict rather than relying on the instructor to pass judgment. As a rule, a reporter for each group submitted a brief written summary (answers, comments, questions) which provided an attendance record.) Recitations operated in a similar manner except the second part of the session was devoted to a quiz on previously covered material.

The remaining class  time often included a follow-up discussion of the previous day’s exercise. When class time was devoted to formal lecture and working through computational problems, this was deliberately not on the same topic as the day’s exercise.

Grading was based mainly on performance on traditional written exams (three one-hour exams and a final exam.) Quiz scores and attendance were included but were a minor factor in determining grades. A nationally standardized exam (over the material in the full year of General Chemistry) was offered as a means of improving grades. The instructor made a conscious effort to grade at a level consistent with the grading in this course in 2003 and 2004.

Grading records for 2003-2005 contain each student’s score on each individual exam question; this  permits detailed analysis since many comparable questions occurred from year to year.

Results and Observations

1. The 2005 test offering was actually able to cover a  wider range of topics and in slightly better detail. In 2004 and 2003 the several chapters were nearly eliminated because of the pressure of time (same instructor, comparable textbook.)  This is significant since the usual concern is that the worksheet approach covers less material than a traditional lecture course.
2. The classroom was clearly an active environment with student conversation and discussion at a much higher level than in the past.
3. Student performance, especially on the first exam, showed measurable improvement over past years.
4. 1. Exam questions related to workshop topics had significantly higher grades than comparable questions in 2004 and 2003. Grade distributions showed fewer students with very low exam scores.
5. Students generally were uneasy with the approach.
6. 1. Many expressed the sentiment “this is not the way I learn.”
   2. Many felt that they needed to see the solution of a comparable problem before they would be able to “solve” such a problem on their own.
   3. By mid-semester this attitude had intensified, despite reassurances that they were collectively performing better than past classes.
   4. The end of semester student evaluations were extremely critical of the instruction style used in the course. Many student expressed the sentiment “the instructor should TEACH more.” This is not the experience reported in other studies where the POGIL approach has high student acceptance.
7. The groups generally functioned effectively in class. About half of the groups worked together outside of class. Several groups collapsed, typically due to poor attendance and/or participation by part of the group.
8. The class size was a problem. With 75-80 students we needed 15 groups of 5-7 students. (The POGIL model suggests groups of 3-4 students.) There were too many groups for the instructor to observe in a period and the groups were too large to draw out all participants. The tiered lecture hall and fixed seating also impedes effective functioning of these larger groups and the travels of the instructor. 

Student Grades and Attrition
            2005    84  students,     6 dropped or received WF,  2 F, 11 D
The retention and success rate in 2005 appears to be a modest improvement over 2003 and 2004. It is worth noting that the drops, D, F and WF grades were approximately equally distributed among students who had B’s and those who had C’s in Chem 125. (A grade of C is a prerequisite.)
                                   
            2004    78 students,  8  dropped, or received WF, 1 F, 8 D  
            2003    60 students,  11 dropped or received WF, 3F, 6D, 3-Inc
       
Recommendations

If the department were to adopt POGIL or a similar approach in this course we need a better implementation—

• the textbook and supporting materials should be reviewed and better tailored to this approach
• Commercial workshop materials should be adopted or material should be prepared locally, with adequate time to review and revise. A minimum of 45 project modules should be available per semester.
• Class size should be carefully reviewed. The 70-80 student size of the fall 127 class is too large for effective implementation. The fall Chem 125 class has approximately 250-275 students  and would require multiple lecture sections (5-8) and additional instructors. The logistics of forming groups with matching recitation and lab sessions can be resolved but would need some attention.
• Student acceptance would probably be better in Chem 125. Students who have been successful in Chem 125 probably want chem. 127 to follow the same pattern because they “know it worked for me..; this is how I learn and study.” More attention needs to be paid to improving student acceptance of the methods.
• The Teaching Assistants in recitations (second and third year doctoral students) and laboratory need additional training to be effective in this style of instruction.
• The laboratory work should be revised to make it also reflect a Guided Inquiry approach.