Student Achievement Assessment Committee

Learning Outcomes

Upon completion of the baccalaureate degree, students in the Integrated Language Arts area of the Adolescent/Young Adult teacher education program are expected to:

1. Demonstrate competence of subject matter in the content area of specialization
2. Demonstrate competence in the theories and practices of sound pedagogy
3. Apply theories of human development and learning to plan, implement and assess daily lessons and units of instruction
4. Exhibit professional and ethical behavior when working with students, their parents, other educators and community members.

Annual Report

Learning Outcomes assessed this year: 

Last year we focused on assessing outcomes #2 and #3. This year, we have focused on assessing outcome #1:

1.Demonstrate competence of subject matter in the content area of specialization

Last year we set aside outcome #1 because we were still working out lines of communication across the School of Teaching & Learning and the English Department, which when in place, would begin to allow us to better assess our students as well as better provide services to meet their needs. During 2005-2006, we improved those lines of communication, and while we still have a great deal more work to accomplish in order to put our program strongly in place before NCATE review, we accomplished a great deal in 2005-2006 that is worthy of analysis and discussion.

2. Assessment Methods and Procedures:

Outcome #1: Demonstrating competence of subject matter in the content area of specialization

The principle tool for assessing competence in the theories and practices of sound pedagogy is the Praxis II: English Content Knowledge: Language and Literature Exam, grades 7-12, administered by Educational Testing Service (ETS). During the past several academic years, we have been working systematically to get accurate scores from ETS that demonstrate the true pass rates of our program completers on the PRAXIS II English Content Knowledge: Language & Literature Exam, and while we still feel that we do not have completely accurate data being provided by ETS to BGSU, we initiated a plan to improve students’ competence based on the information we have at this time.

As of the completion of this summary statement, the most recent data we have on Praxis II English Content Knowledge Exam is from the 2004-2005 academic year. Of all of the BGSU ILA students who took this exam during that time period, a total of 41 out of 47 passed the exam, giving BGSU a pass rate of 87.20%. This data tells us that, overall, we are successfully preparing our students to understand English Content Knowledge as it is assessed by the ETS exam.

The table below shows the passing rates of BGSU ILA students across those five years in comparison to the state averages.

However, while these are solid numbers, they are slightly below the state average of 92.40%. In addition, when we track our students’ scores across a five-year period, the data reveal a significant dip in scores during the 2003-2004 academic year—down to 68.60%—and then a return to a more normal score in 2004-2005. While we are concerned about this dip, we are attempting to sift through the various factors that may have caused this one-year decline.

One ongoing potential factor is that we have never received accurate score reporting from ETS, based on their system of score reporting to institutions of higher education. While they send individual data to each student who takes the exam, they do not send this individualized, disaggregated data to universities. The data we receive is merely aggregated data, and the system by which ETS identifies test takers is inconsistent. For example, there have been individuals who take the English Content Knowledge test who claim BGSU as their home institution, but who are not actually enrolled in our AYA or ILA program. Instances such as these skew the data considerably, and without a system of individualized reporting, we cannot sift through these non-program individuals and exclude them from our data.

Another factor not accounted for in the data above are students who take the test more than once, failing the first time but passing the second time. These students are counted twice in the data, but they really should only be counted once, and at the moment we have no way of distinguishing who took the test, and how many times. As a result, we have never had a clear picture of how our students are doing at any particular time on the Praxis II English Content Knowledge: Language & Literature Exam.

Another potential factor in the low score for the 2003-2004 school year is that the ETS test is not aligned with the ILA major as it was laid out by the state of Ohio and developed in particular at BGSU. This problem lies not with our ILA program, but with the ETS test. Advocates from both NCTE (the National Council of Teachers of English) and NCATE (the National Council for Accreditation of Teacher Education) argue that the ETS test is designed around a very traditional core of American and British literature selections that do not accurately represent what high school students should know, or what Language Arts teachers should teach. As a result, our students are having difficulty because the test is assessing them on only a small portion of all that they are being taught in their ILA program. Our ILA program, as it should be, is a comprehensive program designed to prepare students not just to be teachers of English literature, but teachers of the six language arts: reading, writing, listening, speaking, viewing, and visually representing.

Inferences from Assessments:

Outcome #1: Demonstrating competence of subject matter in the content area of specialization

We can make a general conclusion that our most recent group of ILA students taking the Praxis II English Content Knowledge Exam are competent at an 87.20% passing rate. We can also draw the conclusion that our pass rate (with the exception of the 2003-2004 school year) represents a generally successful trend across four year. While the decrease in score for the 2003-2004 school year represents a statistically significant decline, we don’t want to get too worried about it if the general trend shows solid or improving scores. Of the 37 students who took the test in 2003-2004, seven did not pass the test. If we reacted to this one year of data, revamping our program dramatically to scaffold these students through the test, we would be doing so for seven students, instead of focusing on the needs of the hundreds that we have moving through our program in any given year. Nevertheless, we recognize that the downward dip in the 2003-2004 year suggests that there are some unresolved issues. It is clear from these data, as well as from informal surveys with ILA majors preparing to take the Praxis II exam, that a significant number of students feel unprepared or under-prepared to take the exam. In addition, our scores are lower than the state average, and we would like to match if not exceed state scores.

While we need to get clearer information from ETS so that our assessments accurately reflect our program, it is clear that we need to assess the needs of our students and initiate a plan of action to ensure the success of our ILA program.

Actions Taken/Program Improvements: 

Outcome #1: Demonstrating competence of subject matter in the content area of specialization

We have taken a number of actions this past academic year to address our ILA students’ competence of subject matter in their content area of specialization. Some measures have been taken to improve our data collection methods. Other measures have been taken to improve our delivery of content and services to students. These actions include: 1) requiring students to submit their individual score reports, 2) holding Praxis II training sessions for students, hosted jointly by the Department of English and the School of Teaching & Learning, 3) Establishing ongoing meetings across departments integral to the ILA major, 4) Developing and Refining particular courses, and 5) networking with other ILA programs across the state to share successes and solve problems.

1. Individual Score Reporting. We now require students’ individual score reports. At the moment students are providing them, but we are also in conversation with ETS to provide them as well. While progress has been made, there are still inconsistencies in how scores are reported to us at BGSU which make it difficult to translate into the kind of data we need to assess each program accurately. As we have already discussed, only aggregated data are currently available from ETS.
2. Praxis Preparation Sessions. The School of Teaching & Learning and the English Department met in October of 2005 to explore the problem of why some of our students seem to have difficulty passing the Praxis English Content Knowledge Exam. We initiated a study of the exam—its structure, and the design of particular questions and question-categories—with the plan of familiarizing ourselves with the structure of the exam in order to better train students how to pass the exam. We also initiated a study of the other ILA programs across the state to see how they were structured, and to identify elements that we might want to incorporate into our program. In addition, one faculty member—Tim Murnen from the School of Teaching & Learning—took the Praxis II English Content Knowledge Exam in January of 2006 in order to get an insider’s view on the exam structure, and the experience of taking the exam. From this study, we developed a two-part workshop sequence that we implemented in the Fall of 2005, and again in the Spring of 2006—which provided students general test-taking strategies, as well as guidance with particular test question structures. We also established a plan to have students form self-paced study groups around their most troubling test questions and issues, and several of these groups formed and began to meet weekly. Special Thanks goes to Kris Blair, Interim Chair of the English Department, for supporting and directing much of this joint effort.
3. Ongoing ILA Meetings with STL, English, Speech, Drama, Journalism. Representatives from the School of Teaching and Learning and the other departments have begun to meet at regular intervals to address questions of course content, to ensure that our students pass the Praxis II exam, but more importantly that our program provides the kind of content we feel represents the goals of an Integrated Language Arts major.
4. Development/Refinement of Content Courses. One example of the kind of curriculum refinement in which we have engaged is the reinstatement of Journalism 414 to replace Journalism 450. JOUR 450 was a Journalism Law course intended for Journalism majors; however, because the State of Ohio had called for ILA majors to have exposure to journalism law, it had been deemed a requirement several years ago. However, throughout our process of assessment of curriculum, we realized that our ILA majors did not really belong in that course. Instead, they would be better served with a course that prepared them to teach journalism or run the school newspaper. JOUR 414: Supervision of High School Publications had been on the books in the journalism department, but they had not offered the course in years. We worked to re-design that course around the needs of our ILA majors, including a significant component of journalism law. Beyond the journalism course, we will be refining our grammar & linguistics courses, as well as other potential courses in English literature.
5. Networking with other ILA Programs Across the State. Tim Murnen set up a workshop at the spring conference of OCTELA (the Ohio Council of Teachers of English Language Arts) in March 2006 to open lines of communication across ILA programs to scaffold each other through the NCATE review process. Using ODE (Ohio department of Education) grant money, he brought in speaker Charles Duke from NCATE to guide participants through the process of preparing programs for NCATE review.

The 2005-2006 school year was a busy one for us in the Integrated Language Arts program in the AYA program in the School of Teaching & Learning. The 2006-2007 school year looks to be just as busy if not moreso, as we prepare for NCATE review.