Learning/Service Outcomes

Assessment Methods

Inferences from Assessments

Actions Taken/Program Improvements

Evaluate current organizational communication and workflow processes and procedures for the Operations area of the PACE (Professional and Community Education) Team; identify ways to improve upon existing procedures and/or develop new procedures; and, finally, implement improved and/or new procedures in order to provide the best service possible to the PACE Team and its internal and external customers.

A field consulting project using graduate students in the Master's of Organization Development (MOD) program at BGSU, was conducted during the 2003-04 assessment period. The consulting Team YMAR began working with the PACE staff in early November 2003 and presented their findings and recommendations to the PACE Operations Director on March 23, 2004.

The project utilized action research for assessment & diagnosis purposes. The action research approach utilized in this particular project included the following stages: (1) problem identification, (2) data collection, (3) diagnosis and feedback, and (4) intervention.

Initial meetings and contract development helped to identify the problem. Meetings with the PACE Operations Director and interviews with the PACE Operations staff were used for data collection. Team YMAR reviewed and then condensed their collected data and then worked with their shadow consultant to create an intervention that would address organizational communication and workflow processes and procedures within PACE Operations. The intervention involved the PACE Operations staff conducting a hands-on activity led by Team YMAR that identified an appropriate workflow for PACE Operations projects. Team YMAR then created an organized and easily readable workflow chart for PACE Operations based on the results of the hands-on activity. Finally, the results and recommendations from Team YMAR were presented to the PACE Operations Director on March 23, 2004.

Improvements in organizational communication and workflow processes and procedures were needed to better serve the internal and external customers of PACE Operations. Problem areas were identified and specific suggestions were made by Team YMAR to the PACE Operations Director to address those problem areas.

Organizational communication and workflow processes and procedures have been improved by increasing delegation of appropriate tasks to student employees, development of an improved system for PACE Operations staff to receive their tasks/assignments, scheduling of weekly PACE Operations staff planning and update meetings, and planning to address larger training issues during the summer months when workflow is less intensive.

Reviewing enrollment trends in existing Access and database-related classes.

Observation of conversations with instructors about existing Access and database-related classes.

Class evaluations filled out by students in past classes.

Enrollment trends are quite sporadic, with class sizes as small as 2-3 or as large as 10-12.  8-10 would be considered the optimal number of students for Access classes.  Most often, classes are 5 or fewer students.

Primary inference from analysis of enrollment patterns is that we are not consistently attracting the optimal number of students in Access classes.

There are a few reasons that we have identified as being the causes of this problem:

The concept of relational databases is not well-understood by the computing public.  Many people who would benefit from database/Access training are unaware of the need for training.

The packaging of our existing classes does not lend itself to crisp marketing messages that are more likely to attract students in larger numbers.

Evaluations of our existing classes by our students provides no specific verification of these inferences.  However, students sometimes indicate a desire for more classroom time to be allotted to cover the desired material.  This may indicate that students are beginning classes with less than the ideal preparation and knowledge.  Some students indicate they are unsure that after completing the class that they will be competent in using the software.  This indicates a need to adjust class format and marketing messages.

Meetings were held in the second quarter of 2004 to evaluate the inferences from our various assessment practices.  Those meetings led to recommendations prepared by CTC instructors for the director of the program.  As a result of these recommendations, the following improvements and actions are being taken:

Beginning in the fall, CTC will launch a new concept known as the Total Access Package, which contains three learning modules that lend themselves well to marketing efforts.

Module 1 - Access Foundation will cover relational database concepts and introductory Access skills

Module 2 - Access Level 2 will cover additional Access software skills

Module 3 - Database Project will allow students to apply fully the conceptual skills learned in Modules 1 and 2.

Marketing messages will showcase the fact that Module 1 allows anyone who is unfamiliar with relational database concepts to become acquainted with them.  The message will also emphasize the fact that Module 3 results in the student taking away from the class an actual working database that has been analyzed by a database development professional.

To gain a better understanding of what was involved in running the Women in Science, Math, Engineering, and Technology conferences.  We measured the efficiency of the program delivery by looking at what kinds of problems did we encounter while it ran.  What problems did our instructors encounter?, and we ran a cost-benefit analysis, to measure the expenses incurred to run the program versus the revenue the program generated.

To measure the program's efficiency, we employed several methods of assessment:  post-conference debriefing meetings were held with conference workers in PACE and CEE, to gain a variety of perspectives from people who were "behind the scenes", and instructor evaluation forms were sent out to all those who volunteered to present at the conference, to gain their perspective. 

            To measure the program's cost effectiveness, we completed a formal viability analysis, using detailed budgeting tools built by PACE to examine not only the program's direct operating costs, but also its indirect and variable administrative time-based expenses.  This gave us a much clearer picture of all the costs that must be covered by the revenue generated for the program.

We identified several key points of communication breakdowns between CEE and program instructors; at what class size the quality of the instruction is compromised; at what program size the effectiveness of the delivery is compromised; and at what point, and in which areas, our bottom line is compromised.  

We learned that that the program is at (or may already be beyond) maximum capacity in its current

format:  though demand continues to grow, simply expanding the size of the program will not be

beneficial to participants because we lack staff and resources to adequately service the program at current size or above.  We also learned that the current structure of the program encourages net loss:  costs currently exceed potential revenues, but again, simply increasing the number of participants will not close that gap, rather, the net loss will expand with the program size.

Since programs run in November and February, we will use Summers to plan both programs, in order to increase the efficiency and effectiveness of the delivery.  We will standardize materials (including instructor, registration, and evaluation forms) across programs as much as possible; we will make better use of the Firelands campus, building a more partner-based budget and involving the Office of Educational Outreach more in the planning process; and we will consider diversifying the program format, to more satisfactorily serve existing and future participants (shift the February program to a series of keynote/large group addresses, make better use of DVSS technology to "stream" sessions between BGSU campuses and use fewer speakers). 

In order to improve the bottom line of the program, we will look for ways to decrease expenses (redesign promotional materials, seek less expensive keynote speakers and activities); we will streamline staff efforts to reduce time expended; and we will continue the work begun in Fall 2003 with BGSU's Development office to locate sponsors, donors, and grant funding for the program. 

To better understand the needs of SFS instructors

Instructors are generally very positive about the experience within the confines of State Fire School.  They are often less happy about the support services on campus during their visit.  Based on the feedback that we received in 2003 from the instructors who teach at State Fire School, we made several changes to our accommodations for them and the systems that they followed this year.  Based on 2004 instructor feedback, instructors were very pleased with changes that were made from the previous year and are looking forward to returning next year.  They were most pleased with the new per diem and the ability to stay in a hotel.

1. Instructors were given the option of staying either in the residence halls or a local hotel.
2. Instructors were given a $20 per diem to spend anywhere, instead of declining balance cards that could only be used at the Student Union.
3. Paperwork that used to be handed out to the students by the instructors (during the first hour of class time) was collated into folders for each student, thus saving time for the instructors.
4. We secured burn permits on the properties for live burn training and had the homeowners prepare the properties (remove shingles, etc) rather than requiring the instructors to do it.
5. We put the forms that we require the instructors to fill out (for housing, A/V, apparatus and equipment) online.
6. We met with the IMS staff to ensure that all audio visual was in place, the appropriate classrooms were assigned based on technology needs and that the staff would be willing to troubleshoot.
7. We prepared a newsletter for the opening day so that the instructors would not have to spend class time on announcements.
8. Prepared new instructor handbook.
9. Held new instructor orientation prior to State Fire School.

The Ullman Conference will become financially successful for PACE and the Ullman foundation.

An initial viability study was done to determine potential income and break even point for enrollments.

A final report with revenue and expenses was created for the Ullman board.

This program looked very good on paper based upon numbers in the viability study.

PACE/CEE service costs were covered by the Ullman Conference and a small amount was transferred to the Ullman foundation. 

A smaller number of participants should be anticipated for future conferences.

Additional steps should be considered to decrease costs for this conference so the Ullman Foundation receives a larger income from the event.  Decreases in costs may include looking at marketing, duplication, presenter hotel, etc.

Check the number of students registered for each course.

The desired number of participants in attendance at the Ullman Conference is 170 - 200.  There were 125 participants at the 2004 Conference. 

Look at ways to increase attendance.

Discuss and determine the correct target audience for the mailing list

Add email through OPA as a new marketing medium.

Add Nursing CEUs as an option for attendees.

See that all audio visual components of the conference run smoothly for the presenter and participants.

Check to see if all AV equipment was available for the conference.

Make sure at least one PACE member had adequate knowledge of the equipment and can respond quickly if a problem develops.

Several pieces of equipment were not available for the conference.

We lacked an individual that processed the minimal amount of knowledge about AV during the conference.

Participants were frustrated with the AV components of the conference.

Instructor mentioned the problems but did not seem overwhelmed.

Develop a simple AV form which a presenter submits prior to the conference.

Work with CTLT to develop training classes for the PACE staff on using AV equipment.