• Fitness & Health
• Sport & Exercise Science
• Physical Education
• Strength & Conditioning
• Sports Medicine
• Sport Management
• Dance

Effectively evaluate a teaching personal and social responsibility program

This is an excerpt from Teaching Personal and Social Responsibility Through Physical Activity-3rd Edition by Don Hellison.

Program Evaluation

Evaluating the effectiveness of a TPSR program is no simple task. After-school and community-based programs often serve small numbers of students and have high turnover rates. In fact, the more committed program leaders are to working with underserved youth, the less likely they are to have a program that lends itself to a “clean” evaluation. Although teachers in the schools often have greater numbers of students and more stability (depending on the school), they encounter other obstacles related to the marginalized status of PE in many schools. For these reasons, process-oriented evaluations conducted by TPSR program leaders themselves are common and often quite valuable.

Multiple Sources of Evidence

Depending on a program leader's context and reason for evaluating the program (e.g., for research, funding, oversight, program improvement, or assessing impact), it is usually wise to draw on multiple sources of evidence. In many cases, our program evaluations make use of the kind of information described in the earlier sections of this chapter. Compiling multiple sources of evidence, such as student assessments, program leader reflections, and program records on attendance and retention (depending on the setting), can provide a well-rounded picture of a program and make a case for its effectiveness or areas for possible improvement (or both).

Some version of the following evaluation procedures have been used in many TPSR programs and usually in combination (Hellison and Martinek, 2006; Hellison and Walsh, 2002):

• Include in a journal or on your daily plan observations and feelings about this approach—whether it continues to make sense, whether this is something worth standing for as a PE or PA professional. Try to separate these comments from whether the approach is working.
• Remember that no one can say much about the impact of TPSR if they can't first demonstrate that it was implemented. So consider some form of periodic fidelity check whether it involves a self-assessment or observation and feedback from a colleague.
• Use reflection time and group meeting comments as a source of student perceptions (write them down when possible).
• Keep track of relevant student behaviors, such as the amount of name calling, on-task participation, independent work, and helping others.
• Describe level-related activities and incidents as completely as possible for one lesson early in the program and then again later in the program to help determine the extent of change (either positive, negative, or no change).
• Ask kids how respectful or self-directed they are near the beginning of the year and then again near the end of the year to evaluate change.
• Keep track of behaviors on a regular, or even daily, basis—for example, by marking the appropriate level number next to each student's name.
• Administer anonymous student evaluations of the program. Ask them what they learned about themselves and about relating to others, as well as whether they've improved. See figures 11.10 and 11.11 for examples of anonymous student evaluations that we have given out to our students.
• Consider giving out pre-and post-questionnaires. You may consider constructing your own, or check out two that have been published in academic journals and are available on the TPSR Toolbox Web site (www.tpsr-alliance.org/toolbox). These are the Contextual Self Responsibility Questionnaire (Watson, Newton, and Kim, 2003) and the Personal and Social Responsibility Questionnaire (Li et al., 2008).
• Use written reflections or knowledge tests related to the levels to show the extent to which the students understand them.
• Talk with the students' classroom teachers, administrators, and playground or bus duty supervisors to see whether they believe TPSR is having any effect.
• Find out whether your students' classroom teachers assign conduct ratings. If so, you can get a glimpse into their level of responsibility outside the gym. For example, Don's fourth- through eighth-grade students are rated on self-control in their classroom, so he can look for improvements on their report cards and by talking with their classroom teachers.

When looking at the preceding list, it is important to remember that no one we know of has tackled all of these procedures, and certainly not in a single program evaluation. We recommend choosing just a few data sources or assessment strategies that really address key issues, problems, or program goals.

Culminating Projects

Because of the contexts many TPSR program leaders work in, few are able to conduct the type of program evaluations many academics are interested in (e.g., demonstrating statistically significant decreases in violence or substance abuse). However, many of us have found that culminating activities and group projects can yield compelling products that illustrate some of the unique and meaningful things that can be accomplished in TPSR programs, even if they are hard to measure.

In 1999, Amy Rome (a first-grade teacher at the time) collaborated on a project with Paul in which the two taught yoga and tai chi movements to Amy's first-graders using TPSR. This program involved weekly physical activity lessons and the integration of these lessons (including the TPSR levels) into classroom activities throughout the week. This program was conducted in one of Chicago's public K-8 schools in the Chinatown neighborhood. The student body was made up of poor Chinese immigrants and poor African American students whose families lived in the nearby housing projects. Amy had invited Paul to initiate this program in part because of racial tensions that were an ongoing problem at this school.

At the end of the year, Amy's first-graders had the opportunity to demonstrate what they had learned at a schoolwide brotherhood assembly. In the weeks leading up to the demonstration, the lessons were devoted to planning and rehearsal. When the day of the assembly came, the first-graders (about half Chinese and half African American) worked together seamlessly to demonstrate not only a series of complex and elegant movements but also an extremely high level of composure, self-confidence, focus, and effort. This student performance was an ideal culminating experience to highlight how much the students had learned in this program. It also gave the students the opportunity to serve as role models in their school community for brotherhood as well as personal and social responsibility. Video documentation of the event served as an effective piece of evidence to support the overall evaluation of the program.

In 2001, Stein Garcia (at the time a master's degree student working with Don) and several other program leaders made a short martial arts movie with students from a TPSR Martial Arts Club operated in the after-school hours at another of Chicago's public K-8 schools. This school is on Chicago's West Side and served African American students who were all living at or below the poverty level. Stein, who had the necessary technical expertise and equipment, led this film project, which spanned several weeks. He and the other program leaders worked with the students to write a script, sketch out scenes, and choreograph fight sequences using the skills they had been working on in the club. Of course, the film, titled Defending the Way, had a story line that reinforced TPSR values. The final product was a DVD that included the slick “movie” complete with outtakes and special features such as interviews with the school's principal, the students' classroom teachers, and several of the stars (students) themselves. Although we may not be able to attribute a decrease in dropout rates or improvements in standardized test scores to a program that served only 10 or 12 students per semester, this group project and the product it generated provides compelling evidence of the creativity, hard work, commitment, and responsibility that can be part of a well-implemented TPSR program.

The final example offered in this section is a book project Paul did in collaboration with Diane Coleman (the Memphis PE teacher mentioned previously) in 2004. This project was also carried out with first-graders and extended on the work Paul had begun with Amy Rome in Chicago. By this point, Paul had found that the traditional TPSR daily program format was better suited for students who were at least in the upper elementary grades. To introduce students in the primary grades to TPSR, he created a story that incorporated yoga poses, animal walks, and tai chi-like movements. The story is about a young tiger cub that wanted to be a leader. The tiger cub goes on a journey that involves meeting a number of animals that help the cub learn what true leadership is (i.e., the TPSR levels).

Paul worked with one of Diane's first-grade PE classes for several lessons until they not only knew the various movements in the story but also knew the story itself well enough that they could tell it on their own. At this point, each student was able to choose one character from the story to highlight. In each case, that student was asked to pose for a digital photo demonstrating the pose or movement representing that character (e.g., jumping like a frog). To go along with their photos, students were asked to color pictures of their characters to use as illustrations for the book. Finally, some of Paul's undergraduate students helped to interview the first-graders about what they learned from the story, what they liked most about it, and what they liked the least.

Eventually, all the student photographs and illustrations were integrated with the text of the story into a self-published book. Kid quotes were included as an appendix. The students were thrilled to see themselves in print when Paul read the book to them. A copy of the book was also given to the school's library so the students could show their parents and friends what they had created. The students' classroom teacher was willing to extend the project by having them take one of the themes from the story, caring, as the focal point of acrostic poems they wrote the following month. By the time it was finished, this project had integrated TPSR instruction with authentic assessment strategies and crossed curricular lines while doing it, from the gym to the library and into the classroom. The book along with each student's individual photo, illustration, poem, and interview quotes provided rich and varied data to assess what individual students and the group as a whole had learned along the way.

These examples highlight the potential of creative group projects to provide culminating experiences and generate meaningful artifacts for evaluating TPSR programs. Such projects can result in concrete products such as a performance, a movie, or a book that convey some of the hard-to-measure but extremely important aspects of a TPSR program. These products can augment a program evaluation and provide a unique way of communicating the program's value to stakeholders such as principals, center directors, parents, and funders. Projects such as these provide true authentic assessments of student responsibility because they cannot be successful if students are not willing to engage, try hard, cooperate, and be responsible. The end products often demonstrate the integration of affective development with cognitive and psychomotor development. In fact, all of the examples offered here show the integration of these learning domains as well as the use of higher-order thinking skills in TPSR programs.

To make one final point regarding culminating projects, we remind you of McLaughlin's (2000) comment that “process is product” in programs such as this. Although the performance, the DVD, and the book are wonderful products, the educational value was in the process that led to their creation and the extent to which it challenged and allowed students to be responsible.

More Excerpts From Teaching Personal and Social Responsibility Through Physical Activity 3rd Edition

---