Evidence Based Education
Rewards
In order to properly establish the efficacy of a pedagogical program, we need to have a peer reviewed meta-analysis. However, no such meta-analysis exists for this program. I searched for studies on Rewards, on Google, the company website, Education Source, and Scholars Portal. I was able to locate 5 studies on the topic and only one of these studies was relevant for studying efficacy. Margaret Shippen conducted a study in 2004, which compared the use of two different explicit instruction phonics programs, on 110 grade 7 students, identified as two years or more behind in their reading levels. Students received 55 minutes of instruction, daily, for 30 days. The study measured results for word reading, reading accuracy, and reading fluency. The author did not calculate effect sizes (ES); however, I did calculate the ES for myself using their raw data. In this circumstance, I used Cohen's average, because the original authors did not provide the N for both the treatment group and the control group. Word Reading showed an ES of -.18, reading accuracy showed an ES of .58, and fluency showed an ES of .54. This resulted in a mean ES of .31.
While on the surface this result seems unimpressive; however, older elementary students and reading disabled students typically have the worst results with phonics interventions. Moreover, typically studies would not use a control group and an experimental group which are both using the science of reading. On top of all these factors, the study had a sufficient sample size and was an RCT. For these reasons, I think the fact that it showed a statistically significant ES at all is impressive.
That all being said, as this is only one study and the results were small, we still cannot claim that the program is evidence-based. However, the program is research based, meaning the principles behind the program are well backed up within the scientific literature. According to the Rewards website the key principles of Rewards are:
Decoding multisyllabic words
Identifying and understanding prefixes and suffixes
Increasing word and passage fluency
Building academic vocabulary
Deepening comprehension
Building self-confidence in reading
Rapid-paced, engaging, and explicit instruction for teachers
Constant teacher-student interaction
To contextualize this information, I have charted the effect sizes of these principles, as they have been found within the meta-analysis literature.
Final Thoughts:
I like that the Reward program is simple and focuses on providing students the most essential missing information through explicit instruction. I also like that the program has a rapid pace. However, some critics of the program, have voiced that the program does not provide enough emphasis on explicit and systematic instruction of phonics. The research here is lacking; however, I think it shows a lot of potential for helping older dyslexic or at risk readers.
Final Grade: B+
Most of the program principles are well evidenced, within the meta-analysis literature. A mean effect size between .30 and .39 was found.
​
Qualitative Grade: 7/10
The program includes the following evidence based principles: direct instruction, phonics, scaffolding, fluency instruction, morphology instruction, sight words instruction, and comprehension instruction.
A Note on Excluded Studies:
The Rewards website claims there is significant experimental evidence supporting their program's efficacy. Unfortunately, within their research section it cites several studies that I could not use in this analysis. At the top of the section it cites a RCT study by the program creators; however, the link to this study does not work and the actual article title is not listed. The Rewards website also cites and describes a study by Engleman, Et al, done in 1999. However, the described study and results are identical to the Margaret Shippen study, leaving me to believe there was some kind of mistake, unfortunately there was also no actual bibliography for me to check the Rewards website citations, either. Lastly, the website lists a 2006 study on Rewards, by Vicky Vachon; however, they do not list a title for the paper and a google search, only brings you to her book on the Rewards program.
Similarly, my additional research also led me to 4 additional papers that I could not use. Klee in 2015, looked at Rewards in 2015, but did not include raw data or effect sizes. Buttler, Et al, looked at Rewards in 2013, but did not include their raw data or calculate an effect size. The Buttler paper did however, conduct a t-test, which found no meaningful effects for fluency, or comprehension, but did find meaningful effects for decoding. Kerl conducted a study in 2018; however, the study was only on 7 students. Autzen conducted a study in 2021; however, their study is no longer within the listed published journal.
​
Disclaimer: Please note that this review is not peer reviewed content. These reviews are independently conducted. Pedagogy Non Grata, does not take profit from conducting any program review found on this website.
​
Written by Nathaniel Hansford: teacher and lead writer for Pedagogy Non Grata
Last Edited 2022-02-10
​
References:
C, Buttler. (2013). The impact of REWARDS on reading skills of students with learning disabilities. California State University. <https://digitalcommons.csumb.edu/cgi/viewcontent.cgi?article=1402&context=caps_thes>.
M, Shippen. (2004). A Comparison of Two Direct Instruction Reading Programs for Urban Middle School Students. Remedial and Special Education. Retrieved from <http://mathenrich.pbworks.com/w/file/fetch/52804182/AComparison.pdf>.
Anita Archer, Mary Gleason, and Vicky Vachon. (No known date). Unidentified RCT Study. Broken link: <https://fcrr.org/FCRRReports/PDF/RewardsPlus.pdf>.
I, Klee. (2015). The Effect Using the REWARDS® Reading Program on Vowel Sounds, Word Part, and Prefix and Suffix Identification in Multi-Syllabic Words: A Case Report. Education Research Quarterly. Retrieved from <https://eric.ed.gov/?id=EJ1061951>.
R, Autzen. (2021). REWARDS: A Reading Intervention Program to Address Grade Level Reading Ability. Undergraduate Scholarly Showcase Proceedings. Retrieved from <https://journals.uc.edu/index.php/Undergradshowcase/article/view/4582/3456>.
R, Kerl. (2018). Timing is Everything: Adapting Rewards Intermediate For 4th And 5th Grade Striving Readers. Hamline University. Retrieved from <https://digitalcommons.hamline.edu/cgi/viewcontent.cgi?article=5436&context=hse_all>.
The Learning Company. (2020). HMH Into Reading Implementation Study Research Study Results 2019-2020. Retrieved from <https://s3.amazonaws.com/prod-hmhco-vmg-craftcms-public/research/Research-Results-Paper-Into-Reading-2019-2020-SY.pdf>.
R, Eddy, Et al. (2020). QUASI-EXPERIMENTAL DESIGN. Cobblestone. Retrieved from <https://s3.amazonaws.com/prod-hmhco-vmg-craftcms-public/research/HMH-Into-Reading-ESSA-Tier-2-QED-Study-Report.pdf>.
Filderman, M. J., Austin, C. R., Boucher, A. N., O’Donnell, K., & Swanson, E. A. (2022). A Meta-Analysis of the Effects of Reading Comprehension Interventions on the Reading Comprehension Outcomes of Struggling Readers in Third Through 12th Grades. Exceptional Children, 88(2), 163–184. https://doi.org/10.1177/00144029211050860
Feng, L., Lindner, A., Ji, X. R., & Malatesha Joshi, R. (2019). The roles of handwriting and keyboarding in writing: a meta-analytic review. Reading & Writing, 32(1), 33–63. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s11145-017-9749-x
Ehri, Linnea & Nunes, Simone & Willows, Dale & Schuster, Barbara & Yaghoub-Zadeh, Zohreh & Shanahan, Timothy. (2001). Phonemic Awareness Instruction Helps Children Learn to Read: Evidence From the National Reading Panel's Meta-Analysis. Reading Research Quarterly. 36. 250-287. 10.1598/RRQ.36.3.2.
Elleman, A.M., Lindo, E.J., Morphy, P., & Compton, D.L. (2009). The impact of vocabulary instruction on passage-level comprehension of school-age children: A meta-analysis. Journal of Research on Educational Effectiveness, 2(1), 1–44. https://doi.org/10.1080/1934574080 2539200
J, Hattie. (2022). Meta-X. Visible Learning. Retrieved from <https://www.visiblelearningmetax.com/influences>.
N, Hansford. (2021). Morphology: A Secondary Meta-Analysis. Pedagog Non Grata. Retrieved from <https://www.pedagogynongrata.com/morphology>.
Graham, Steve, and Michael Hebert. Writing to Read: Evidence for How Writing Can Improve. Carnegie Corporation Time to Act Report. Washington, DC: Alliance for Excellent Education, 2010. Print.
HMH. (2022). Research Into Reading Research Foundations. Retrieved from <https://www.hmhco.com/research/hmh-into-reading-research-foundations>.
-NRP. (2001). Teaching Children to Read: An Evidence Based Assessment of the Scientific Literature on Reading Instruction. United States Government. Retrieved from <https://www.nichd.nih.gov/sites/default/files/publications/pubs/nrp/Documents/report.pdf>.
Voyageur Sopris. (2022). Reading Intervention. Retrieved from <https://www.voyagersopris.com/literacy/rewards/overview>.