Student Achievement Assessment Committee

The assessments conducted for 2005-2006 were the Freshman Development Program (FDP) Assessment and the Necessary Components Retention Program Assessment (NCRPA). The first assessment highlighted in this report is the FDP Assessment. The goal of the Freshman Development Program is to facilitate the academic and personal adjustment of first-year African, Latina/o, Asian and Pacific Islander, and Native American students. There were 383 participants in the program for 2005-2006. This program consists of group meetings (Partners in Excellence meetings) and individual appointments four times per semester with an advisor in the CMAI department.
This is the second year that the FDP was assessed. Based on the analysis of the 2004-2005 Freshman Development Program Assessment, the following list was generated as actions steps to take for the 2005-2006 Program:

Actions Taken:

• Revise the learning outcomes and rubric to better assess the multiple layers covered during the Freshman Development Program
• Provide more and earlier opportunities for students to interact with their peers at the Partners in Excellence (PIE) meetings to assist with building more collegial connections among peers
• Engage students in more virtual discussions, peer support, and exchanges on Blackboard by creating a more cohesive support group per individual advising caseload and to facilitate building a community among participants outside of the PIE meetings
• Investigate further the influence of family on the participants’ academic success and how the Freshman Development Program can involve the family with the community service component of their participation
• Investigate further the influence of the faculty and how the Freshman Development Program can enhance this connection
• Increase the influence of the Freshman Development Program’s expectations of their success by revising the FDP Units to communicate this expectation more clearly
• Continue to provide the personal support and mentoring components because the data revealed that pressures and experiences as African, Latina/o, Asian and Pacific Islander, and Native American students may influence academic success

All of the above listed actions were implemented for 2005-2006 by:

1. Revising the learning outcomes, criteria, and rubric
2. Making the Partners in Excellence meetings for FDP participants more interactive
3. Upgrading the FDP Blackboard communities to provide more opportunities for virtual discussions and support
4. Revising the FDP Units and CMAI Web site to provide opportunities for students to interact with faculty and provide information for family participation and involvement
5. Collaborating with the Breakfast of Champions Program to include parent participation with the FDP students’ annual community service project for fall 2005
6. Continuation of the individualized support and advising provided to FDP participants

The 2005 – 2006 FDP Assessment

The assessment of the learning outcomes for Freshman Development was conducted using a Web-based testing system on Blackboard with a pretest and posttest. The pretest or the Initial Assessment Interview was available on Blackboard from August 22, 2005 through September 30, 2005. The posttest or the Final Assessment Interview was available on Blackboard from March 20, 2006 through April 30, 2006. The students took the pretests and posttests individually at any time within the established dates. See Appendix A for a copy of the assessment questions for both the pretest and posttest. See Appendix B for a summary of the FDP Units as listed on the Freshman Development Program Matriculation Management Checklist.
There were 285 students that took the pretest and 193 students that completed the posttest. Each test consisted of 19 multiple choice questions that ranked their responses to questions targeting a specific learning outcome. Question 20 was an essay question that asked for their comments, suggestions, and general information about their experiences with the program and Center for Multicultural and Academic Initiatives staff.  
The revised learning outcomes of the Freshman Development Program are listed in Table 1. This table includes the learning outcomes and the description of the criteria used to measure each learning outcome. The overall percentage of students taking the assessment that scored at the highest level (Criteria 4) for the measured learning outcomes are listed in the last column of Table 1. Raw data is available from the Center for Multicultural and Academic Initiatives upon request. The raw data will list the percentage of students at every level of the measured criteria for each learning outcome by individual advisor and overall for all participants in the FDP. Table 2 is a frequency distribution of the themes that were identified based on an analysis of the data from the essay question (# 20).

Table 1: Freshman Development Learning Outcomes

Table 2: Themes from Question 20 Frequency Distribution

Analysis of the FDP Assessment Data Summary:

• The data demonstrates that students are losing motivation to achieve academic and personal success as the academic year progresses. Initially, 82% of the students taking the Freshman Development Program Assessment scored at the highest level of the measured learning outcome. However, near the end of the year, only 72% of the students taking the assessment tested at the highest level for the learning outcome ‘Motivation and Achievement of Academic and Personal Success.’
• Fall 2005 pretest indicated that 55% of the students were at the highest level for understanding and appreciating diversity and by Spring 2006 only 48% of the students remained the highest level. This may indicate that some students may be struggling with understanding and appreciating diversity as they become more involved in the campus community and have more interactions with diverse individuals.
• Students show an increase in their academic self-esteem and their ability to stop perceived negative perceptions that faculty or peers have a negative impact on their academic performance with 21% of the students testing at the highest level in the fall and near the end of the program, 36% of the students testing at the highest level.
• Students in the Freshman Development Program experienced a decrease in their understanding of their academic strengths and weaknesses because on the pretest 28% of the students tested at the highest level and on the posttest 26% tested at the highest level.
• Time management had the highest frequency rating of 40 for a skill that students hoped to gain from the Freshman Development Program. However, students did not demonstrate significant skills gained in this area with 23% testing at the highest level in the fall and only 27% testing at this level in the spring.
• The FDP highest level of increase for any learning outcome is the understanding of BGSU’s academic policies with 39% of the students testing at the highest level on the pretest and 57% testing at the highest level on the posttest.

The 2005-2006 NCRPA

The Necessary Components Retention Program Assessment (NCRPA) measures students of color attitudes, perceptions, and behaviors concerning their learning experiences at colleges and universities in relations to their experiences with university activities and services (Taylor & Shuford, 2004). The NCRPA has six subscales that measure Ethnic and Peer Connection, Process of Social Integration, Personal Worth & Competence, Reliable Alliances, Faculty Guidance, and Leadership Opportunities.

The NCRPA takes approximately 20 minutes to complete and responses are recorded on the assessment packets and scanner sheets using a #2 pencil. The assessment consists of 9 demographic questions. Students circled or wrote their responses to the demographic questions on the assessment packet. There were an additional 101 statements or questions that used a Likert scale to rate their perceived most appropriate response to each question on the scanning sheets.

Raw data from the scanner sheets were compiled by the BGSU Data Processing Center. This raw data and the demographic data were forwarded to the BGSU Statistical Consulting Center for computation. All tables shown in this report were generated by the BGSU Statistical Consulting Center.

This NCRPA assessment was given at the April 7, 2006 Partners in Excellence meeting to 91 Freshman Development Program participants. These students had participated in the program since beginning their enrollment as new freshmen in August 2005-2006.

Table 3 is the demographic description of the students taking the assessment. Please note that one student failed to complete the demographic section of the assessment.

Table 3: NCRPA Demographic Description for 2005-2006
The FREQ Procedure

Age

Sex

Racial/Ethnic Classification

Enrollment Status

Academic Classification/Rank

Transfer Status

Present College

College Residency

Grade Point Average (Accumulative)

Analysis of the NCRPA Data Summary:

Overall, the students taking the assessment were traditional college students with 64% of the population being female and the remaining 36% male. Most students were African American at 62% with the next highest population being represented by 23% Latino students. The majority of students taking the surveys were Ohio residents at 92% of the population. Eighty-nine percent of the population had grade point averages ranging from 2.5 to 4.0 and 54% had grade point averages within the 3.0 to 4.0 range.

Below is a summary of the highlights of the analysis of the data. The following rating scale was used: 1 = Strongly Disagree, 2 = Disagree, 3 = Not sure, 4 = Agree, 5 = Strongly Agree.

Note: For this part of the analysis N = 91 unless otherwise noted. See Appendix C for the tables used to summarize the data found in this brief analysis. Appendix D contains the questionnaire used to conduct the NCRPA. Contact the Center for Multicultural and Academic Initiatives for complete information.

Ethnic and Peer Connection
Ethnic and Peer Connection represents the sense of security and comfort that a student may feel as a result of close relationships with others.  It addresses issues related to ethnicity, racial identity, and cultural effects on peer relationships.  Peer attachment has a positive relationship with coping strategies such as self-disclosure, self-direction, confidence, and social support. 

• Students participating in the Freshman Development Program have found it less challenging to connect socially with other students at BGSU (Mean = 1.95)
• Students have felt a strong emotional bond with at least one other BGSU student (Mean = 4.05)
• Students felt comfortable with the ideal of socializing with students outside of their racial and ethnic groups (Mean = 4.63).
• Students are not sure about or have a tendency to disagree with the statement that being the only person of their race/ethnicity interacting in the BGSU classroom environment makes them feel isolated  (Mean =  2.62)

Personal Worth & Competence Scale
Students’ feelings of personal worth and competence are associated with their perceptions related to being an academically and socially competent member of the university community.  These perceptions result from both external validation and internal reflection.  The external dimension is enhanced by external acknowledgment from the faculty and peers on academic and leadership success and both pre-college and college educational involvement.  The internal dimension is related to intellectual and interpersonal competence (e.g., self efficacy, academic achievement, motivation, learning styles and career maturity).

• Completing their degree is important (Mean = 4.29)
• Most students feel that it is important to be perceived as academically competent as a BGSU student (Mean = 4.04)
• Students completing the assessment tended to agree that knowing that students of color are sometimes viewed as less competent motivates them to work harder (Mean = 3.95)
• Most participants disagreed that things come easy for them (Mean = 2.34)
• Most students disagreed with or felt uncertain about the statement, ‘I have not experienced many personal challenges at BGSU (Mean = 2.57)

Reliable Alliance
A sense of Reliable Alliance comes from knowing that one's peers, faculty, and administrators provide assistance and advocacy in crisis situations.  Peer support and informal contact with faculty are related to college survival (King & Taylor, 1989).  Accordingly, a healthy alliance with individuals who possess varying levels of expertise and experience increases students’ ability to work through personal crises. 

• The majority of the participants in the Freshman Development Program felt that there were people they could go to for assistance if they were experiencing a crisis at BGSU (Mean = 4.16) and that there are a lot of people that they can depend on for help at this university for help (Mean = 3.73)
• A lower number of students felt that they could depend on certain faculty members more than any other group of people at BGSU for help (Mean = 2.65)

Leadership Opportunities
Leadership Opportunities focus on the psychosocial benefits associated with serving in leadership or paraprofessional roles that require students to provide assistance to other individuals. 

• Most students tended to disagree with the statement that they spend 5 to 10 hours per week participating in academic organizations and clubs (Mean = 2.45) or that they were involved in 3 or more academic organizations (Mean = 2.51)
• Spending 10 or more hours per week participating in collegiate athletics was disagreed with by most students with a mean average rating of 1.83
• On average, students indicated that they spent 15 or more hours per week participating in academic activities (classes, labs, studying, research, etc.) (Mean = 3.68)

Process of Social Integration Scale
Affiliating with other individuals on campus that share common interests and have similar attitudes influences the process of social integration for college students. The Process of Social Integration subscale in this instrument represents three levels of affiliation that describes the process in which students of color may use as a way to establish a supportive community:  Campus membership, Social Enclaves-level I (i.e., minority programs, services and organizations) and Social Enclaves-level II (small friendship circles).

• The majority of students participating in the Freshman Development Program found this campus to be welcoming (Mean = 3.83)
• Students tended to disagree with the statement that they have seriously contemplated transferring to another institution (Mean = 2.68)
• Most students did not feel that their sexual orientation limited their sense of connections with other students (Mean = 1.76)
• Most students tended to agree that they feel comfortable participating in activities with individuals who are from a different race, ethnic group, sexual orientation, or socioeconomic status (Mean = 3.86)

 Faculty Guidance
Guidance in the form of advising, mentoring, challenging, and supporting from faculty members and administrators is fundamental to student learning and academic success.

• Most participants disagreed with the statement that they do not feel comfortable discussing their career interests and concerns with faculty and campus administrators at BGSU (Mean = 2.06)
• Students tended to disagree that the Student Financial Aid Office is their only source of information concerning scholarship, paid internships, and employment opportunities (Mean = 2.75)
• Most students tended to disagree with the statement that there are no faculty members on campus that they can seek guidance from when stressed out about class (Mean = 2.30)

Overall, where alpha = .05 there were significant correlations between the subscales of Social Integration and Leadership Opportunities and then between Social Integration and Personal Worth and Competence (p = <.0001 for both). There was no correlation between the subscales of Leadership Opportunities and Ethnic and Peer Connections.

ACTIONS STEPS FOR 2006 – 2007 (based on the analysis of all the data from FDP Assessment and the NCRPA) are as follows:

• Incorporate opportunities for students to build stronger social connections to develop their leadership and personal worth and competence in the Freshman Development Program
• Nearly all the students participating in the Freshman Development Program have merit-based scholarships, yet they tend not to participate in academic organizations and clubs. An action step for 2006-2007 is to provide more opportunities for students to explore the academic clubs and organizations through their participation in the Freshman Development Program.
• Provide more support for students in the Freshman Development Program for combating some of the Personal Worth and Competence challenges that they experience as a collective group. This subscale area was the lowest rated overall on the NCPRA.
• Students tend to rely on other departments outside of the Student Financial Aid Office for information concerning scholarships, paid internships, and employment opportunities. These areas are going to be enhanced for the 2006-2007 Freshman Development Program Participants.
• Provide more support to develop better time management and goal setting skills for participants
• Provide students with more support in the spring to understand their academic strengths and weaknesses and making informed major and career decisions
• Make enhancements to assist FDP participants with the understanding and appreciation of diversity