Student Achievement Assessment Committee

Learning/Service Outcomes for On-Campus Residents and Resident Advisors (RAs):

Aligned with the mission of the University and the Division of Student Affairs, the Office of Residence Life (ORL) exists to
• Provide comfortable, affordable, well-maintained, safe and secure housing for residents.
• Create a residential living/learning environment that stresses community-building, promotes inclusiveness, and supports residents’ academic learning.
• Encourage positive interactions between residents and hall/house staff to facilitate residents’ personal growth.
• Involve residents in social, cultural, and leadership programs offered by the office of Educational Initiatives and Greek Affairs.
• Assist with first-year student transition and retention through enhancing residents’ satisfaction with living accommodations and roommates, and through facilitating their social adjustment on campus.
• Create an environment that challenges and facilitates Resident Advisors’ personal growth.
• Promote an environment that supports a diverse resident population.

Annual Report

1. Learning/Service Outcomes Assessed during the 2003-2004 Academic Year

(1). Learning and service outcomes assessed by The Spring 2003 RA Staff Survey

This assessment was to measure the following learning outcomes for residents:
• RAs provide academic support for residents (i. e., encouraging them in their academic goals, planning and implementing programs involving faculty in the residence halls, soliciting residents’ need, supporting residents in managing conditions impacting their academic success).
• RAs interact daily with residents in positive ways to facilitate residents’ personal growth.
• ORL disciplinary system results in student learning and personal development
• ORL promotes an environment that supports a diverse resident population

This assessment was also to measure the following learning outcomes for RAs:
• RAs participate in community service projects as a result of their RA position.
• RA position challenges RAs to grow personally.
• RAs improve various skills as a result of the RA position, e. g., confrontation skills, conflict resolution skills, counseling skills, mediation skills, etc.

(2). Learning/service outcomes assessed by the ORL 2003-2004 Resident Satisfaction Survey

The goal of this assessment was to measure whether ORL has achieved the following outcomes:

• Providing clean and well-maintained housing.
• Promoting safety and security in residence halls/houses.
• Cultivating a sense of community through offering community-building activities on each floor and encouraging residents accepting of differences of others.
• Enhancing residents’ satisfaction with their roommate.
• Supporting residents’ academic success through creating a quiet learning environment, through providing information on academic resources, and through residential technological support.
• Involving resident in social, cultural, and leadership programs/activities offered by the Office of Residence Life.
• Promoting resident governance through residents’ participation in Hall Council and Resident Student Association (RSA) activities.
• Residents have positive interactions with Hall/House staff.
• Serving a diverse resident population.

(3). Learning outcomes assessed by the 2004 National Residence Environment Survey (Also called the National Study of Living-Learning Programs, or NSLLP)

The goal of participating in this Survey was to measure whether students at BGSU residential learning communities (RLCs) accomplished the following learning outcomes:
In relation to their peers living in the conventional residence halls, RLC students
• Reporting better academic and social transition to BGSU.
• Reporting more interactions with peers and faculty and having more involvement in co-curricular activities, including more frequently using residence hall co-curricular resources.
• Reporting higher intellectual abilities and higher intellectual growth.
• Reporting higher self-confidence and a higher appreciation of diversity.
• Reporting a stronger sense of civic engagement and less alcohol-related problems.

(4). Learning outcomes assessed by The ORL 2003-2004 SMART Program Evaluation

The goal of this evaluation was to measure whether the SMART Program has achieved the following outcomes:

• Assisting first-year students of color with their transition to BGSU.
• Helping create conditions conducive to first-year student retention: better social adjustment, increased university involvement, and higher satisfaction with residential living.

(5). Learning outcomes assessed through other activities, e. g., the Fall 2004 Resident Input for the RAs, and assessment conducted by Greek Affairs
The goal was to measure whether RAs and Greek Affairs achieved the following outcomes:
• Supporting residents’ academic learning.
• Promoting residents’ civic engagement.
• Involving students in leadership programs.

2. Assessment Methods and Procedures

(1). The Spring 2003 RA Staff Survey

The paper-and-pencil RA Staff Survey was distributed to the entire population of RAs at BGSU in May 2003. This Survey was a self-report questionnaire, consisting of Likert-scale items and open-ended questions. 152 RAs responded out of 176, resulting in a response rate of 86.4%. All the data analyses were done in Summer 2003. After entering the data manually, we analyzed the numerical data by using SPSS v. 11.0 for Windows. First, reliability co-efficients were reported. Then, both descriptive analysis (i. e., frequencies) and ANOVAs (to determine the significant difference among RAs in different buildings) were conducted. Qualitative data generated through the open-ended questions were also transcribed, categorized, and thematized. Finally, based on the quantitative and qualitative results, an executive summary and a full report were generated and distributed to stakeholders.

(2). The ORL 2003-2004 Resident Satisfaction Survey

This paper-and-pencil survey is a self-report questionnaire, consisting of 180 variables (most of which are Likert-scale items) and one general open-ended question. It was distributed to the entire on-campus resident population at BGSU in January 2004. 3563 out of 6415 students responded, resulting in a 56% response rate. The usable surveys were scanned and the prescreened data were analyzed by using SPSS v. 11.0 for Windows. First, reliability co-efficients were reported. Next, frequencies and means were obtained on different learning/service outcomes measured. Then, inferential analyses (i. e., T-tests, Ch-Square tests, and ANOVAs) were conducted to determine the significant mean differences regarding residents’ perceptions of their residence environment by gender, race/ethnicity, class, hall/house, sexual orientation, learning community, and transfer status. Qualitative data generated through the open-ended question were also categorized and thematized. We compared overall results with those obtained from the 2002-2003 Office of Residence Life Satisfaction Survey. We also calculated the performance gap by computing the differences between the percentages of those who marked “Somewhat or very important” and the combined percentages of those who marked “Satisfied” and “Very satisfied”, using pair-wise deletion method and eliminating those marking “No opinion”. The notion of a performance gap is used to identify cases where residents’ view of the importance of an item is substantially different from their satisfaction ratings on that item. The purpose is to capture the degree of congruence between ORL goals and residents’ needs. Finally, based on the quantitative and qualitative results, an executive summary and a full report were generated and distributed to stakeholders.

(3). The 2004 National Residence Environment Survey

In fall 2003, ORL contracted to participate in the National Study of Living-Learning Programs (NSLLP). This study is the first multi-institutional study of residential learning communities in the nation and BGSU was one of the 34 universities participating in the study. The study was developed by a collaborative team of researchers led by Dr. Karen Kurotsuchi Inkelas from the University of Maryland. In fall 2003, in collaboration with the Office of Institutional Research, we provided the residential learning community students sample (413 students) and the carefully matched control sample (1584 students) to the national data collection staff. We also provided the text for the survey invitations and reminders as well as incentives for participating in the survey. In March 2004, the 2004 National Residence Environment Survey was fielded at BGSU through the Internet. The overall response rate of BGSU students was 41.1%, ranked 6th highest among the 34 participating universities. In April and May 2004, the data generated from the Web survey were analyzed by the NSLLP national collaborators. In June, we received our copy of the customized report based on the results of the BGSU participants.

(4). The ORL 2003-2004 SMART Program Evaluation

To assess whether the SMART Program assisted the retention of first-year students of
color: At the beginning of fall semester 2003, we requested assistance from the Office of Institutional Research in calculating the Fall 2002-Fall 2003 retention rate of SMART mentees.
To assess whether the SMART Program assisted with creating conditions conducive to first-year student retention: We distributed the Spring 2004 SMART Mentee Survey--a self-report questionnaire--in February 2004. 38 out of 110 mentees responded to this survey, resulting in a 35% response rate. After scanning, we used SPSS 11. 0 for Windows to analyze the quantitative data. The qualitative data generated through the open-ended questions were transcribed and analyzed as well. A full report and a summary were generated and disseminated among the SMART Staff.

(5). Other assessment activities, e. g., the Fall 2004 Resident Input for the RAs, and assessments conducted by Greek Affairs
The hall/house staff used self-report surveys to gather information.

3. Inferences from Assessments

(1). Key findings from The Spring 2003 RA Staff Survey

• Over 95% of RAs reported “Agree” or “Strongly Agree” that they encouraged students’ academic goals, supported creating conditions impacting students’ academic success or personal development, and had positive interaction with residents.
• Approximately 95% of RAs reported that the RA position has challenged them and helped their personal growth. The experience has improved RAs’ wide range of skills, particularly confrontation skills, conflict resolution skills, counseling skills, programming skills, time management skills, advising skills, mediation skills, team management skills, administrative skills, crisis management skills, oral presentation skills, etc.
• In terms of enhancing diversity, nearly 90% RAs reported “Strongly Agree” or “Agree” that ORL promotes an environment supporting a diverse student population and a diverse staff respectively.
• 45.4% RAs reported having planned and implemented programs involving faculty in residence halls.
• 48% RAs reported “Agree” or “Strongly Agree” that ORL discipline system results in student learning and changing behavior. RAs did not reach consensus regarding how ORL can improve the way to respond to student misconduct, as indicated from RAs’ contrasting comments voiced in their answers to the open-ended questions.
• Some RAs suggested ORL conduct student need assessment and offer more events and programs based on student interests.
• Some RAs insisted that overnight guest policy in certain halls needs revision.

(2). Key findings from the ORL 2003-2004 Satisfaction Survey

The results from the quantitative part of the Survey indicated that ORL has largely fulfilled its mission in providing a well-maintained, safe and secure, and positive living/learning environment supporting residents’ academic study and personal development and promoting residents’ interactions with hall/house staff. Specifically,
• In comparison with the results obtained from the 2002-2003 ORL Satisfaction Survey, there is a notable increase of reported satisfaction level on almost all the comparable variables.
• Approximately 80% residents reported being satisfied with Custodial and Maintenance staff in keeping the residence halls clean and well-maintained.
• Approximately 90% residents reported being satisfied with safety and security inside and outside their hall/house, with lighting surrounding their hall/house, and with quality of fire prevention measures.
• Regarding building community to enhance student personal growth: Approximately 80% residents reported being satisfied with the sense of community on their floor, with fellow residents’ accepting of the differences of others, and with the relationship with their roommate.
• Regarding supporting residents’ academic learning: Slightly over 80% residents agreed that their RA/House Staff member serves as an important resource (campus activities, academic assistance, etc) and that the residence hall/house rules assisted their academic success and personal development. In terms of technological support, nearly one third of residents used a residence hall computer or chapter resource room for academic purposes.
• Regarding involving residents: Among those who participated in the activities offered by Residence Life, leadership programs offered by Greek Affairs, Hall Council activities, and RSA programs, approximately 95% residents reported being satisfied with activities.
• Regarding fostering positive contacts between residents and ORL staff: Approximately 80% residents reported they have had positive interactions with hall/house staff (RAs, House staff, Hall Directors, Hall Secretaries, and Student Workers), with hall/house staff’s fair enforcement of university rules, and with hall/house staff role modeling responsible behavior.
The Survey results meanwhile identified some issues. They include:
• The general cleanliness of the bathroom/showering facilities;
• The noise level in the hall/house;
• Resident’s overall level of involvement in the educational programs and activities offered by ORL;
• RCC staff’s ability to help with network connections;
• The significantly lower level of satisfaction with some aspects of residence environment reported by the following four sub-groups of residents: female residents, first-year residents, minority residents, and gay, lesbian, bisexual, and transgender residents.

(3). Key findings from The 2004 National Residence Environment Survey

Significant differences were found between the BGSU RLC students and their BGSU on-campus counterparts not enrolled in a RLC. Specifically, RLC students at BGSU
• Discussed academic, career, and social-cultural issues with other students outside of class more frequently.
• Reported having had more faculty mentorship.
• Reported having used co-curricular residence hall resources more frequently.
• Reported more involvement in such co-curricular activities as arts or music performance/activities, religious clubs/activities, ethnic/cross-cultural clubs/activities, one-time and on-going community service. However, they reported less involvement in fraternity/sorority and intramural/club sports.
• Reported better critical thinking/analysis abilities and more enjoyment of challenging intellectual pursuits.
• Reported more positive peer diversity interactions.
• Reported a stronger sense of civic engagement.
• Were more likely to report that they plan to conduct independent research and to have culminating senior experience (capstone, thesis, etc).
• Were less likely to report health and emotional consequences of alcohol use or report having experienced nuisance negative secondary behavior.
• Were more likely to agree that their residence hall climate was academically and socially more supportive.
• Reported spending less time partying and less time in playing video/computer games.
• Reported spending more time attending classes, studying/doing home work, and volunteering.

Significant differences were also found between BGSU RLC students and their RLC peers at Research Extensive universities with 6-10 living-learning programs. BGSU RLC students

• Reported more ease with academic transition to college.
• Reported having had more frequent course-related interactions with faculty and faculty mentorship.
• Reported more involvement in religious clubs/activities but less involvement in political/social activism and in media activities.
• Reported more growth in cognitive complexity.
• Were less likely to report emotional consequences of alcohol use.
• Were more likely to agree that their residence hall climate was academically supportive.
• Reported less positive peer diversity interactions and a less positive diversity climate of their college campus.
• Were less likely to plan to do research with a professor and to study abroad.
• Reported more satisfaction with faculty/staff interactions.

(4). Key assessment results regarding the SMART Program

The Office of Institutional Research found that the Fall 2002-Fall 2003 retention rate of SMART mentees was 79.4% while the retention rate of comparable non-SMART mentees was 72.5%.
Based on the results from the Spring 2004 SMART Mentee Survey, the SMART mentees perceived the Program to be generally effectiveness in creating conditions potentially improving student retention—social adjustment and involvement in the university community. Specifically,
• 75% SMART mentees reported “Agree” or “Strongly Agree” that because of their mentor they became familiar with resources they needed on campus.
• 83.3% mentees agreed that through SMART, they made new friends on campus.
• Approximately 75% mentees agreed that SMART has helped them feel included in the BGSU community, that SMART has helped them get more involved in the BGSU community, and that SMART staff care about them as individuals.
• 63.8% mentees reported “Agree” or “Strongly Agree” that SMART has helped them with their transition from high school to BGSU.
• Approximately 90% mentees reported that SMART sponsored enough academic, cultural, and social activities in the fall semester. 68.8% reported that SMART sponsored enough leadership activities.
• Except for leadership activities, approximately 60% mentees reported that SMART programs in fall 2003 definitely met their expectations.
• Over 50% students did not attend other SMART activities beyond Study Tables, Kwanzaa, and 1st Home Football game.

(5). Key findings from other assessment activities, e. g., the Fall 2004 Resident Input for the RAs, and assessments conducted by Greek Affairs
• Regarding promoting Greek residents’ civic engagement: in Spring 2004, on average, Greek students performed a total of 12, 348 hours of community service, with each member averaging 7 hours. Greek students raised a total of $ 14, 975 for charity; on average, each Greek student raised $46.
• Regarding involving students in leadership programs: The majority of Greek students attended Greek Odyssey and Emerging Leaders—two major Greek leadership programs.

4. Actions Taken/Program Improvements

• Where residents reported a high satisfaction level with the hall/house services and residence environment, ORL plan to continue the current effective practices.
• ORL will be examining major residents’ needs identified through the various surveys discussed above.
• ORL are now making changes to the 2004 Summer RA Training in order to better meet residents’ needs.
• RAs, Hall Directors, and Hall Staff held discussions surrounding the issues identified through the qualitative data generated through the open-ended question asked in the ORL 2003-2004 Satisfaction Survey. Hall staff expressed how they reacted to the comments; what kind of changes they would suggest to solve a problem; based on the discussions, the assistant directors examined whether overall policy change needs to be made.
• In response to the issue of overnight guest policy, in February 2004, ORL staff and the Resident Student Association co-sponsored a forum to debate this policy. Residence life directors and hall directors participated in the forum and voiced their concerns. By the end of the meeting, the issue remained unsolved. Based on the debate result and after much deliberation, ORL choose not to change the overnight guest policy.
• In response to the issues identified through the ORL Satisfaction Survey, ORL staff met Resident Student Association members to seriously look at the problems and potential solutions.
• Based on the assessment results related to the noise level on a certain floor of a certain hall, the hall directors and RAs of that specific building held formal discussions on how to implement quiet-hours policy more effectively to create a quieter learning environment.
• Given that the SMART mentees perceived the Program to be generally effective in creating conditions improving student retention, i. e., social adjustment and involvement in the university community, SMART staff will continue to sponsor programs to help mentees become more familiar with resources they needed on campus, to help them make new friends, to get them more involved in the BGSU community, and to help them with their transition from high school to BGSU.
• In response to the finding that only 68.8% mentees reported that SMART sponsored enough leadership activities, SMART staff are planning to encourage mentees to participate a formal leadership program—Leaders in Residence.
• To get more SMART mentees to attend every activity sponsored by the SMART Program, the staff will offer well-planned activities and events for Fall Semester 2004 based on mentees’ perceived needs. An initial needs assessment is now being conducted during summer 2004 and will be conducted again at the beginning of the fall 2004 semester. The information from the needs assessments will be used to make changes to the program.