Student Achievement Assessment Committee

a) Linguistic Proficiency:

• Grammar: Ability to communicate effectively through appropriate utilization of grammatical functions.
  
• Reading: Ability to understand the main thesis and main supporting ideas as well as some nuances of

  written materials.
  
• Writing: Ability to develop and sustain a topic coherently in writing.

b) Knowledge of Hispanic Literatures and Cultures:

• Ability to present original research on Hispanic literatures and cultures in written form.

a) Linguistic Proficiency:

• Grammar and Reading: Students were asked to take the "University of Wisconsin College-Level Placement Test for Spanish (Grammar and Reading Comprehension)." Most of the students graduating in December 2001 and May 2002 had taken the same test when they entered the program. Our objective is to compare results and draw conclusions.
  
• Writing: Students were asked to turn in an outcomes portfolio consisting of one paper from one of their earliest classes as majors in our program and one paper that represented their best work up until that point. At the same time, they were asked to write a brief statement commenting on the development of their writing skills in Spanish using the two documents in their portfolio as evidence for their evaluations.
  Also, students were asked to participate in an assessment interview to discuss their outcomes portfolio and the Spanish program. The interviews were conducted as conversations; however, during the course of the interview, faculty asked each student six questions: 1) Why did s/he select the two pieces presented in the portfolio? (i.e., how are the papers representative of the students' beginning efforts and their best work in Spanish?); 2) What s/he see as her/his particular strengths as a Spanish major (e.g., linguistic ability, cultural skills, critical thinking, etc.)?; 3) What areas did s/he believe needed improvement as a Spanish major?; 4) What areas would s/he would have liked to have learned more about?; 5) What experiences in the Spanish program had particularly formative (e.g., classes AYA program, Sigma Delta Pi, etc.)?; and 6) What else would s/he like to tell us about her/his experience as a Spanish major at BGSU? Faculty members responded to the students' remarks on Questions #2 and #3 by offering their own evaluation of the students' strengths on the basis of the outcomes portfolio.

b) Knowledge of Hispanic Literatures and Cultures:

• We use the same instruments employed to assess the linguistic proficiency in writing. Students were asked to turn in an outcomes portfolio and to participate in an assessment interview. Questions #2 and #3 from the previous section were modified in the following way: 2a) What did s/he see as her/his particular strengths in terms of knowledge of Hispanic literatures and cultures?; and 3a) What areas of this knowledge did s/he believe needed improvement?. Faculty members responded to the students' remarks on Questions #2a and #3a by offering their own evaluation of the students' strengths on the basis of the outcomes portfolio.
  
  

• Grammar/Reading Diagnostic Test: Nine of the ten graduating students took both Entrance and Exit tests. All of those who took the tests show improvement in their scores (see Appendix). The lowest improvement was 1.5% and the highest was 38%. Over 50% of the student showed an improvement of 25% or more.
  Four of the nine students received scores of 90-100% (69-77 correct answers of 77 total questions). Two received scores 80-89% (62-68 correct answers of 77 total questions). Two students received scores of 70-79% (54-61 correct answers of 77 total questions). One student received a score of 66% (51 correct answers of 77 total questions).
  The most common grammatical errors (Questions 21 & 30) were about higher level functions (past subjunctive in subordinate clauses after an indefinite or negative antecedent and long possessive forms respectively). At their level of linguistic proficiency, students are not expected to exhibit mastery of those functions. Other common errors were questions testing the uses of the subjunctive (Questions 7, 23, & 27), special syntactic constructions (Question 35), and problematic semantic distinctions (Questions 5 & 37).
  
• Portfolios: Faculty saw an outstanding level of improvement among the ten students who presented their portfolios and participated in the assessment interviews. An analysis of the papers revealed a noticeable improvement between the first and second papers. In their second papers, the students utilized more complex grammatical structures and more coherent discourse, with extensive and sophisticated vocabulary. They also demonstrated their knowledge and understanding of Hispanic literatures and cultures.
  
• Evaluation of the Hispanic Program: Nine of the ten graduating students returned the Spanish Program Evaluation form. The average rating of their experience in the Spanish program is 3.3 (4 being 'excellent' and 1 'poor').
  The majority stated that the strengths of the program are: 1) The Study Abroad experience (AYA Spanish), 2) Variety of classes offered, 3) the professors, 4) Spanish Club. There was no consensus regarding the weaknesses of the program. The students offered a variety of suggestions to improve the program: more interaction with native speakers, provide more focus on topics beyond literature and history in courses, eliminate combined undergraduate and graduate courses, offer more courses in conversation and grammar, and implement a placement test.

4. Actions

• Curricular Revision: This academic year, the Spanish faculty made various efforts to diversify and improve the curriculum. A committee began discussions about the restructuring of the major gateway courses (SPAN 351 and SPAN 352). To address the needs of Spanish Education students, who must take the ETS Praxis Exam for their certification, a course Advanced Oral Expression (SPAN 489) was taught during the Spring 2002 semester. The offering of this course was also motivated by comments from the students that "their weakest skill was oral communication" (2000 Summary of Assessment, 2). Another improvement to the Spanish curriculum was the proposal of two Independent Study offerings in "Readings in Hispanic Studies" (SPAN 279 and SPAN 379 at 1-3 credit hours). Previously, students wishing to do an independent study could only do so at the advanced level (SPAN 470). These new courses allow students at the beginning and intermediate levels to study a particular topic related to Hispanic literatures and language. Also, a proposal was submitted for Selected Topics courses (SPAN 280 and SPAN 380) offer course on special topics in Peninsular, Latin American, and US Latino literatures, film, culture, language, and linguistics. Two other courses were also proposed (SPAN 295 and SPAN 395) for all-day workshops about specific topics. These course proposals are currently pending approval from the College of Arts and Sciences.
  
• Computerized Grammar/Reading Test for Placement and Assessment: The university has not allocated the funds to purchase this test, which both internal and external reviewers have recommended in the past.
  
• Changing the Assessment Process: Last year, the Spanish faculty recommended the appointment of an Undergraduate Coordinator to the Chair of the Department of Romance Languages. The Undergraduate Coordinator would "promote intellectual and professional development of students through such activities as organizing presentations on professional and intellectual issues …" and to "coordinate the student assessment process for majors" (2001 Summary of Assessment, 3-4). This position did not receive the necessary course release from the College of Arts and Science administration. Due to this year's financial constraints, the Department of Romance Languages could not provide its own funding. Therefore, this plan has not been undertaken.
  
• Future actions: For this coming year, we intend to continue efforts to restructure the gateway courses, with the aim of standardizing textbooks and curriculum as recommended by students in previous years. The committee, which has worked on this issue during the past year, will present their findings to the rest of the Spanish faculty in order to implement the needed changes.
  
  Appendix

Comparison of Grammar and Reading Scores: Entrance vs. Exit

G = Grammar / R = Reading
(Note: A December 2001 graduate, turned in a portfolio and evaluation form, but did not take an Exit test)

French

Preparation/Learning Outcomes

Assessment took place in two stages this year since we had graduating seniors in both December 2001 and May 2002. In each case, all 6 students* who had applied for graduation (all of whom were in EDHD) were sent a letter about the assessment procedure by the eighth week of the semester. They were asked to submit the following materials**:

• an outcome portfolio consisting of one paper from one of their earliest classes as a French major and one paper from a more recent class that represented their best work, in their opinion;
• a short analysis in English of their views on their progress in the French major, based on the comparison of the two aforementioned papers. They were asked to address in particular the improvement of their overall critical thinking skills, their knowledge of French and Francophone literatures and cultures and the improvement of their oral and/or written skills in French;
• a completed program evaluation form providing their feedback on the French major.

Students were also asked at that time to schedule an appointment to retake the Grammar and Reading Comprehension sections of the University of Wisconsin College-Level Placement Test, which they had initially taken at the beginning of French 351, the first course taken during their major.

Finally, they were asked to schedule a portfolio review meeting / exit interview with the undergraduate advisor and one other faculty member.

* Ultimately we did not assess one of these students since she had only completed teaching certification coursework at BGSU. She already held a B.A. in French from the University of Toledo.
** A second student did not cooperate with us on any aspect of the assessment: portfolio, program evaluation, scheduled exit interview, or exit exam.

Format of Assessment Interviews

At the end of each semester, two members of the French section held assessment interviews with the graduating seniors (1 in the Fall and 3 in the Spring) to discuss their outcomes portfolio and the French program. The interviews were conversational in nature; however, during the course of the interview, faculty asked the students seven questions:

• Could you tell us in general terms about your experience in the French program at BGSU?
  
• Do you think that the fact that you have studied French has prepared you for the job market? Do you feel competent in your abilities in French?
  
• Are you planning to emphasize your knowledge of French when you do a job search? If so, what has given you confidence to do so? If not, why are you hesitant to do so?
  
• Do you feel that your French studies have contributed to your intellectual development? If so, in what way?
  
• In your opinion, what role did the AYA program play in your learning of the French language and/or your understanding of French culture?
  
• In your opinion, what are the strengths and areas needing improvement in this program?
  
• What would be your advice to an incoming Freshman who wishes to study French and to study abroad?

Summary of Student Learning Outcomes

Faculty saw a clear improvement among all but one of the students, particularly in terms of their linguistic skills and their knowledge of French and Francophone literatures and cultures. Students were, without exception, able to communicate more effectively in the more recent papers in their portfolios. They were able to express more complex ideas, to utilize more sophisticated vocabulary, and to demonstrate improved grammatical skills. Most also showed stronger critical thinking skills. There were extremely few comprehension problems, with the exception of one student who, to our dismay, appears to have made little linguistic progress during her studies.

Based on available statistics, there was a tremendous improvement in two students' performance on the Wisconsin test from the beginning of their studies to the end. In one case, the entry score was 23/64 and the exit score 41/64, a 78% improvement. In the other, the entry score was 28/64 and the exit score 57/64, a 104% improvement. Another graduate demonstrated some improvement, less significant only because she was a strong student initially: her entry score was 53/64 and her exit score 59/64, an 11% improvement. One student did not take the test during 351 but scored 54/64 at his "exit"; another scored 33/64 for the entry exam, but did not take the exit exam. For these last two students, then, lacking entry or exit exam scores prevent us from being able to quantify progress.

Summary of Program Evaluation

Without exception, the students praised the study abroad programs (quality, organization, location, etc.) in France and Burkina Faso, which they cited as central to their experience. Many students praised the classes on campus, both in their quality and their diversity (i.e. both French and Francophone content), as well as their small size. (Two, however, wished that we offered a wider variety of classes, especially at the 400-level.) They called the professors accommodating to students and committed/dedicated to French studies.

While the students' remarks varied, we heard three common areas of dissatisfaction. The first related to the traditional lack of emphasis on oral skills in classes taken on campus. We have heard this complaint in the past and have already taken steps to remedy it by offering one course focusing solely on oral skills at the 300-level since Fall 1999. Since this year's group of graduates entered the program before Fall 1999, they did not take "French 356: Skills for Oral Proficiency". However, the students assessed during 2001-2002 were aware of this change to the program and believed that it would constitute a great improvement in students' ability to strengthen oral skills early on in the major, before studying abroad. More recently, we have also included an oral skills course at the 400-level, which we will explain in greater detail below.

As was the case last year, several students expressed dissatisfaction with an educational methods course for students from all languages and would prefer that such a course be language-specific.

Another area of general concern was the French House: students regretted that activities were not better attended or offered at more convenient times. One student suggested that we offer activities there (such as the weekly conversation table) after 9pm; this would be problematic given that it is also a residence.

In the written program evaluations, one student rated the program as "excellent", one as "good/close to excellent", one as "good" and one as "average". As mentioned previously, one student did not turn in an evaluation.

Actions

As stated in last year's report, The French section has repatriated French 451: Advanced Composition and Conversation" from the program in Tours and offered it on campus for the first time in Spring 2002. Effective Fall 2002, this course will be a requirement for French Education majors upon return from France. Two graduates enrolled in French 451 during Spring 2002 and both were highly satisfied with it. One student of 451 in particular passed, on her first try, the (historically problematic) Oral Production section of the PRAXIS exam in April 2002. She has already been hired to teach French at Southview High School in Sylvania, Ohio. We have also raised the number of credits required for the A&S major from 27 to 30 and thereby included 451 as an on-campus requirement for A&S students as well. We are confident that the inclusion of this course in the on-campus major curriculum will continue to aid all students in their post-study abroad language acquisition and also facilitate the Education majors' preparation for the PRAXIS exam.

We are very pleased that Dr. Bonnie Fonseca-Greber has been hired to teach, among others, the foreign language Methods course. We hope that her French background and pedagogical knowledge will inspire confidence in our French Education majors. We also applaud the addition of this linguist to our French section which, diverse and complementary in other ways, had been lacking in that area.

Finally, the French section has continued to discuss how we might improve the French House experience. We have contemplated making it co-ed, although no concrete steps have been taken to date. We have also initiated steps to offer a room and board scholarship to live in the French House, not to AYA returnees as discussed last year, but rather to incoming Freshman with 4 years of high school French who would enroll in 300-level French courses during their first year of study. Scholarship recipients would, however, have the option of living elsewhere on campus (a necessity given that males are not able at present to live in the French House). We hope to offer these scholarships as of Fall 2002, and we also hope that such an opportunity would encourage our students to live in an environment conducive to the continuation of their oral French progress, perhaps leading to majoring/minoring in French and/or studying abroad.