Assignment 2

Provus’ Discrepancy model can be used to evaluate educational programs. It would be an appropriate model for the evaluation of the ESC Programming for Children with Severe Disabilities. Evaluators employing this model gather evidence of compliance with established standards, identify any discrepancies between the standards and actual performance, and recommend corrective actions if necessary (Regan, Triggs, Mitsopoulos, Duncan, Godley, & Wallace, 2000).

In order to begin an evaluation using the Discrepancy model, clear and specific program goals must be established. The school board in this case has so far failed to do so as it only mentions that a “child’s program must meet the child’s needs” and includes an inspirational quote that describes how “teachers should lead children to mystery” and help them “unlock the beauty” in what they see (Medicine Hat Separate Regional Division #20, 2009). These hints at the mission of the program are much too vague to be measured or to help define the success and direction of the program. Participating in a Discrepancy evaluation will force the school division to establish some explicit goals for this program.

In addition to determining clear objectives for the program, the first stage of the Discrepancy model is concerned with program design (Rose & Nyre, 1977). The school board has clearly delineated the students who are eligible for the program, the staff that need to be involved, the required time allotment, and the documentation necessary. The program has its regulations clearly outlined but neglects to include information on possible activities that should be engaged in to meet the program goals therefore in addition to program goals, program developers need to ascertain suitable learning activities that can be included in this program.

The second stage of the Discrepancy model involves ensuring that the implemented program is congruent with the school board’s plan (Rose & Nyre, 1977). The evaluators need to ensure that the programs’ regulations are indeed being followed. The evaluators should determine if all children in the division with severe disabilities are able to access the program, whether all children participating in the program do meet the eligibility standards, if staff is properly qualified, and if required time allotments and documentation are being completed.

The third stage is a formative evaluation as it focuses on the program’s process. At this point, the evaluator begins to examine the alignment between the program’s standards and its performance (Rose & Nyre, 1977) in order to refine the program to maximize its instructional effectiveness during the development process (Regan et al., 2000).

It is during the fourth stage that the product of the program is assessed by comparing student attainment with the standards and objectives of the program (Rose & Nyre, 1977). Any discrepancies are determined at this time and the reasons for these discrepancies are investigated (Regan et al., 2000). Steps are then taken to eliminate the discrepancies (Regan et al., 2000). If students’ academic and behavioural achievement has not increased according to the program’s standards then corrective actions can be taken (Regan et al., 2000). The Discrepancy model lends itself equally well to both quantitative and qualitative methods (Regan et al., 2000). In determining if the majority of children have benefitted from their participation, not only should academic and behavioural assessment scores be considered but the children, parents, teachers, and other professional staff should be interviewed to determine their satisfaction with the program.

Finally, stage five involves a cost-benefit analysis (Rose & Nyre, 1977). The program can be compared to other similar programs in order to determine if this program is cost-effective and whether the program needs to be modified in order to improve outcomes for students in the future.



References

Medicine Hat Catholic Separate Regional School Division No. 20. (2009). ECS

programming for children with severe disabilities. Retrieved from

http://www.mhcbe.ab.ca/ on September 12, 2009.


Regan, M. A., Triggs, T. J., Mitsopoulos, E., Duncan, C. C., Godley, S. T. & Wallace, P.

(2000). Provus’ discrepancy evaluation of the Drivesmart novice driver CD-Rom

training product. Retrieved from

http://www.rsconference.com/pdf/RS000047.pdf?check=1 on September 12,

2009.

Rose, C. & Nyre, G. (1977). The practice of evaluation. Retrieved September 6, 2009,

from,

http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019

b/80/35/ad/cc.pdf

Assignment 1 revised with citations

Outcomes Linked to High-Quality Afterschool Programs: Longitudinal Findings from the Study of Promising Afterschool Programs is an outcome-based evaluation that measured the effects of afterschool programs on elementary and middle-school participants. The purpose of this summative evaluation was to determine the effectiveness of high-quality afterschool programs in improving student outcomes as a result of attendance in these programs.

A quasi-experimental design was utilized as both participant and comparison groups were identified. The design was not truly an experimental design as the afterschool programs were not randomly chosen to take part in the study but were selected by the evaluators based on: program’ evaluations, recommendations, and whether the programs met the evaluators’ criteria for high-quality afterschool programming.

The program evaluators adopted a model similar to Stake’s Countenance model as they sought to judge the merit of high-quality afterschool programs. Countenance evaluators identify contingencies between outcomes and antecedent conditions and instructional transactions (Rose & Nyre, 1977). The evaluators collected data to establish a relationship between the outcomes of improved academic and behavioural success and the participation of students from low socio-economic backgrounds in afterschool programming. The evaluators found congruencies between student participation in high-quality afterschool programming, achievement on standardized math tests, reports of improved work habits and task persistence, and reductions in misconduct and drug and alcohol use amongst middle-school students. The evaluators obtained standardized scores and participant observations to make a judgment upon which to recommend a continuation and expansion of high-quality afterschool programming.

The Countenance model allows for an experimental design to be adopted. An experimental design protects against threats to internal validity thus increasing the significance of the findings of the evaluation. Another advantage of the Countenance model is that it focuses on immediate, long-term, personal, cognitive, affective, and societal outcomes (Rose & Nyre, 1977). In this evaluation, behavioural outcomes were considered as well as academic ones. In accordance with Stake’s recommendations, the evaluators considered the judgments of those participating in the programs including teachers and students (Rose & Nyre, 1977).

One of the main weaknesses of the Countenance model is also present in this example as once a judgment had been made regarding the merit of the programs the evaluation was finished. In this evaluation there is no mention of how afterschool programming could be improved in the future except to have it continued and expanded to more communities. If continuing feedback was part of this model, the evaluators could begin looking at why some students chose not to attend, how low attendees can be encouraged to participate, what characteristics seem to be present in high vs. low attendees, and how programs could be improved to further increase positive academic and behavioural outcomes. Participant outcomes were not categorized so that the impact on various groups could be compared so that improvements could be made to meet diverse needs in the future. The longitudinal impact is also not considered with the Countenance model. There is no mention of expanding the study to determine the impact afterschool programming may play on students’ lives once they stop attending and in their adult lives.

Reference

Rose, C. & Nyre, G. (1977). The practice of evaluation. Retrieved September 6, 2009,

from,

http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019

b/80/35/ad/cc.pdf

Asignment 1

Outcomes Linked to High-Quality Afterschool Programs: Longitudinal Findings from the Study of Promising Afterschool Programs is an outcome-based evaluation that measured the effects of afterschool programs on elementary and middle-school participants. The purpose of this summative evaluation was to determine the effectiveness of high-quality afterschool programs in improving student outcomes as a result of attendance in these programs.

A quasi-experimental design was utilized as both participant and comparison groups were identified. The design was not truly an experimental design as the afterschool programs were not randomly chosen to take part in the study but were selected by the evaluators based on: program’ evaluations, recommendations, and whether the programs met the evaluators’ criteria for high-quality afterschool programming.

The program evaluators adopted a model similar to Stake’s Countenance model as they sought to judge the merit of high-quality afterschool programs. Countenance evaluators identify contingencies between outcomes and antecedent conditions and instructional transactions. The evaluators collected data to establish a relationship between the outcomes of improved academic and behavioural success and the participation of students from low socio-economic backgrounds in afterschool programming. The evaluators found congruencies between student participation in high-quality afterschool programming, achievement on standardized math tests, reports of improved work habits and task persistence, and reductions in misconduct and drug and alcohol use amongst middle-school students. The evaluators obtained standardized scores and participant observations to make a judgment upon which to recommend a continuation and expansion of high-quality afterschool programming.

The Countenance model allows for an experimental design to be adopted. An experimental design protects against threats to internal validity thus increasing the significance of the findings of the evaluation. Another advantage of the Countenance model is that it focuses on immediate, long-term, personal, cognitive, affective, and societal outcomes. In this evaluation, behavioural outcomes were considered as well as academic ones. In accordance with Stake’s recommendations, the evaluators considered the judgments of those participating in the programs including teachers and students.

One of the main weaknesses of the Countenance model is also present in this example as once a judgment had been made regarding the merit of the programs the evaluation was finished. In this evaluation there is no mention of how afterschool programming could be improved in the future except to have it continued and expanded to more communities. If continuing feedback was part of this model, the evaluators could begin looking at why some students chose not to attend, how low attendees can be encouraged to participate, what characteristics seem to be present in high vs. low attendees, and how programs could be improved to further increase positive academic and behavioural outcomes. Participant outcomes were not categorized so that the impact on various groups could be compared so that improvements could be made to meet diverse needs in the future. The longitudinal impact is also not considered with the Countenance model. There is no mention of expanding the study to determine the impact afterschool programming may play on students’ lives once they stop attending and in their adult lives.