A quasi-experimental design was utilized as both participant and comparison groups were identified. The design was not truly an experimental design as the afterschool programs were not randomly chosen to take part in the study but were selected by the evaluators based on: program’ evaluations, recommendations, and whether the programs met the evaluators’ criteria for high-quality afterschool programming.
The program evaluators adopted a model similar to Stake’s Countenance model as they sought to judge the merit of high-quality afterschool programs. Countenance evaluators identify contingencies between outcomes and antecedent conditions and instructional transactions (Rose & Nyre, 1977). The evaluators collected data to establish a relationship between the outcomes of improved academic and behavioural success and the participation of students from low socio-economic backgrounds in afterschool programming. The evaluators found congruencies between student participation in high-quality afterschool programming, achievement on standardized math tests, reports of improved work habits and task persistence, and reductions in misconduct and drug and alcohol use amongst middle-school students. The evaluators obtained standardized scores and participant observations to make a judgment upon which to recommend a continuation and expansion of high-quality afterschool programming.
The Countenance model allows for an experimental design to be adopted. An experimental design protects against threats to internal validity thus increasing the significance of the findings of the evaluation. Another advantage of the Countenance model is that it focuses on immediate, long-term, personal, cognitive, affective, and societal outcomes (Rose & Nyre, 1977). In this evaluation, behavioural outcomes were considered as well as academic ones. In accordance with Stake’s recommendations, the evaluators considered the judgments of those participating in the programs including teachers and students (Rose & Nyre, 1977).
One of the main weaknesses of the Countenance model is also present in this example as once a judgment had been made regarding the merit of the programs the evaluation was finished. In this evaluation there is no mention of how afterschool programming could be improved in the future except to have it continued and expanded to more communities. If continuing feedback was part of this model, the evaluators could begin looking at why some students chose not to attend, how low attendees can be encouraged to participate, what characteristics seem to be present in high vs. low attendees, and how programs could be improved to further increase positive academic and behavioural outcomes. Participant outcomes were not categorized so that the impact on various groups could be compared so that improvements could be made to meet diverse needs in the future. The longitudinal impact is also not considered with the Countenance model. There is no mention of expanding the study to determine the impact afterschool programming may play on students’ lives once they stop attending and in their adult lives.
Reference
Rose, C. & Nyre, G. (1977). The practice of evaluation. Retrieved September 6, 2009,
from,
http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019
b/80/35/ad/cc.pdf
Rose, C. & Nyre, G. (1977). The practice of evaluation. Retrieved September 6, 2009,
from,
http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019
b/80/35/ad/cc.pdf
Solid application of Stake's model. This is a well-written piece that clearly lays out the process and approaches taken by the evaluation team. You effectively point out the weaknesses in this particular evaluation. The summative nature of the evaluation does leave the reader with many questions about why this evaluation was conducted and what changes, if any, have resulted.
ReplyDelete