Research results can be a fickle thing. I have seen phonics studies with negative effect sizes and Balanced Literacy studies with very high effect sizes. But if we look across the breadth of the literature, we see a clear trend in that phonics outperforms balanced literacy. For this reason meta-analysis is clearly necessary. This allows us to answer the question, what does the majority of the scientific literature say on a specific topic. Of course this is problematic also. Different studies are done across different time horizons, with different demographics, with different tests, by different researchers. 

 

While, meta-analysis is supposed to remove the variability by ultimately increasing the total sample, some of the variability is undoubtedly context specific, and not random. For example, multi-step word problems show very low results in elementary school, but very high results in secondary school. The more we combine and generalize studies, the more we risk taking them out of a context, in which they mean something. 

 

That being said, the variability we see within studies is far too great, within the field of education, for me to believe it is reasonable to accept the results of any individual study in isolation. Personally, I believe the answer for this is meta-analysis that includes as much context specific data, as possible, such as age, subject, and condition. Which is why my website, as of late, has gone through contextual meta-analyses on a variety of topics. However, I also think we have to realize that teaching is a time constricted activity. Teachers only have limited resources and time, so they need to pick strategies/factors which give them the most yield for their time. However, to determine what strategies/factors can best help them reach success in their classrooms, they need to be able to compare the effects of different pedagogies, strategies, and factors. The only way to do this on a high level is secondary meta-analysis. 

A secondary meta-analysis is a meta-analysis of meta-analyses and it comes with all the problems of a regular meta-analysis and then some. A common criticism can be that when you use a secondary meta-analysis you are “comparing apples to oranges”. And of course, this is at least partially true, as the student populations, ages, measures, calculations, etc, all end up being variable. Which means, the contextual specificity of the research is very limited. That being said, there is no more efficient methodology for finding the relative effectiveness of a pedagogy compared to multiple other pedagogies.  

 

This article is an excerpt from a book I am currently writing. For it, I have created a secondary meta-analysis out of the 41 math  meta-analyses and research studies I have reviewed over the last two years. Moreover, I have tried to correct for some of the contextual issues, by creating multiple other secondary meta-analyses that are context specific, which will follow up with in future articles. 

​

Factors Glossary: 

 

Ability Grouping: Grouping students according to their ability levels. Source: Secondary meta-analysis by author, 2022. Link for more information: https://www.teachingbyscience.com/differentiation

 

Accuracy Focused Instruction: Fluency instruction focused accuracy, over speed. Source: Methe 2012 meta-analysis. Link for more information: https://www.teachingbyscience.com/math-fact-fluency 

 

Adaptive Process Tech: Technology that individualizes student instruction via AI. Source: Ran 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/technology-and-math

 

Calculators: The use of calculators on math achievement. Source: Ellington 2003 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/calculator 

 

Cognitive Based Interventions: Includes meta-cognition strategies, schema instruction, self talk, and behavior management strategies. Source: Myers 2021 meta-analysis. Link for more information:

https://www.teachingbyscience.com/learning-disabilities-and-math

 

Collaborative Tech: Software that helps staff and students collaborate/communicate more easily, such as Google Classroom. Source: Ran 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/technology-and-math

 

Constructivist Games: Games based on constructivist math philosophy. Constructivist math philosophy dictates that “Learners are actively engaged in their own learning such that knowledge is assumed to be constructed by learners rather than transmitted. Constructivism closely relates to experiential and discovery learning. However, it adds the construction of personal meaning by the learner as a final step.” Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Contingent Reinforcement: Giving specific rewards to students for specific math achievements, such as candy, for demonstrating mastery of a learning target. *This research was only on primary students, which is important as rewards research shows negative outcomes for older students. Source: Getson 2009 Meta-analysis. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math

 

Cooperative Classroom: A classroom that emphasizes the use of cooperation over competition. Source: Zhining 1995 meta-analysis. 

 

Cover Copy Compare: Have the student copy down a set of math facts. Rewrite the math facts from memory. Then check their work. Source: Methe 2012 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/math-fact-fluency 

 

CRA: Concrete-representational-Abstract, also referred to as graduated instruction. A lesson/unit planning method, in which teachers use manipulatives first, diagrams second, and abstract work last. Source: Methe 2012 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/math-fact-fluency  

 

DI/Conceptual Games: Games that use direct instruction to teach math concepts. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Experiential Games: Games that help students learn math through experiential learning. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Experiential Games: Games that help students learn math concepts through experiential learning. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Feedback: Telling students how they can improve their work. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math

 

Feedback with Goal Setting: Giving students feedback and encouraging them to make goals based on that feedback. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math

 

Feedback with Optional Targeted Instruction: Giving students feedback and offering them additional targeted instruction, to help their individual needs. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math

 

Fluency Focused Instruction: Fluency instruction that prioritizes speed over accuracy. Myers 2021 meta-analysis. Link for more information:

https://www.teachingbyscience.com/learning-disabilities-and-math

 

Game Based Instruction: Teaching math via games. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Homework: Giving students math homework. Source: 2018 Fan meta-analysis. Link for more information: https://www.teachingbyscience.com/math-homework

 

Iterative Teaching: Teaching conceptual and procedural knowledge at the same time. Source: Durkin 2011 meta-analysis. Link for more information: https://www.teachingbyscience.com/conceptual-math 

 

Kumon: Japanese math program that focuses on mastery instruction, fluency and individualization. Source: Un-peer reviewed meta-analysis by author. Link for more information: https://www.teachingbyscience.com/kumon 

 

Learning Management Systems: Computer AI/softwares that provide virtual classrooms which connect with multiple other learning programs. Source: Sayjili 2021 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-management-systems 

 

Manipulatives: Physical objects used to help teach students math. Source: Carbonneau 2013 meta-analysis. Link for more information: https://www.teachingbyscience.com/manipulatives 

 

Mastery Teaching: Teaching content until you are sure the students completely understand it. Source: John Hattie secondary meta-analysis. Link for more information: https://www.visiblelearningmetax.com/Influences 

 

Math Facts Instruction: Directly teaching arithmetics facts through memorization, such as the times-tables. Source: Cason 2019 meta-analysis. Link for more information: https://www.teachingbyscience.com/math-fluency

 

Montessori Style Games: Games made with the private school Montessori brand of inquiry based learning. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Multi-Step Word Problems: Word problems with multiple steps to solve. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems 

 

Number Sense Games: Games that are designed to increase students’ understanding of number sense. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math 

 

Number Sense Instruction: Instruction designed to increase students' understanding of place value, arithmetic, as well as their overall math fluency. Source: Cason 2019 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/math-fluency

 

Number Sense Word Problems: Word problems regarding the domain of number sense. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems 

 

Number Talks: A branded form of math discussion that focuses on teaching students multiple heuristics, via a constructivist approach. Source: A literature review by author. Link for more information:

https://www.teachingbyscience.com/number-talks 

 

Numeracy Instruction: “Numeracy skills may include mathematical-logical thinking, relational reasoning, and specific concepts foundational for number sense such as one-to-one correspondence.” Source: Cason 2019 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/math-fluency

 

Peer Tutoring: Having students help teach other students. Source: Getson 2009 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Problem Based Learning: Teaching math via complex problems that students solve in groups. Source: Haas 2005. Link for more information: https://www.visiblelearningmetax.com/influences/view/problem-based_learning 

 

Problem Solving Games: Games based on the principles of problem based learning. Source: Kacmaz 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/game-based-learning-in-math  

 

Problem Solving Technology: Technology based games that are based on the principles of problem based learning. Source: Ran 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/technology-and-math

 

Scaffolded Examples: Providing students with an easy, moderate, and difficult math exemplar. Source: Getson 2009 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Shema Instruction: Providing students with meta-cognition strategies for solving complex math problems, usually associated with word problems and situational problems. IE: Ask yourself, what is the question asking, what do you know, what don’t you know. Source: Myers 2021 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Single Step Word Problems: Word problems that only require one operation to solve. 

Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems  

 

Speed Based Interventions: Skill and drill activities rapidly done, such as around the world, or flash cards. Source: Getson 2009 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math  

 

Student Verbalization: Students using self-talk, while trying to solve problems. Source: Getson 2009. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Technology: Use of technology for the purposes of math instruction. Source: Ran 2022 meta-analysis. Link for more information: 

https://www.teachingbyscience.com/technology-and-math

 

Use of Multiple Heuristics: Teaching multiple procedures for each type of math problem. Source: Getson 2009. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Visual Representatives: Using diagrams to teach math. Source: Getson 2009. Link for more information: 

https://www.teachingbyscience.com/learning-disabilities-and-math 

 

Word Problems: Concrete math problems. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems 

 

​

Written by Nathaniel Hansford

Last Edited 2022-04-07

References: 

 

Bethany Rittle-Johnson, and Michael Schneider. (2011). Developing Conceptual and Procedural Knowledge of Mathematics. Oxford Press. Page 9.  

 

Rittle, et al. (2007). Common and Flexible Use of Mathematical Non Routine Problem Solving Strategies. Science and Education.  

 

Cowan, R., Donlan, C., Shepherd, D.-L., Cole-Fletcher, R., Saxton, M., & Hurry, J. (2011). Basic calculation proficiency and mathematics achievement in elementary school children. Journal of Educational Psychology, 103, 786–803. doi: 10.1037/a0024556.

​

Schwartz, D. L., Chase, C. C., Chin, D. B., & Oppezzo, M. (2011). Practicing versus inventing with contrasting cases: the effects of telling first on learning and transfer. Journal of

Educational Psychology, 103, 759–775. doi: 10.1037/a0025140

​

Hiebert, J. & Grouws, D. (2009). Which teaching methods are most effective for maths? Better: Evidence-Based Education, 2, 10–11.

​

J, Hatie. (2017). Hattie Ranking: 252 Influences And Effect Sizes Related To Student Achievement. Visible Learning. Corwin. Retrieved from <https://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/>.

McKenna, M. A., Hollingsworth, P. L., & Barnes, L. L. B. (2005). Developing Latent Mathematics Abilities in Economically Disadvantaged Students. Roeper Review, 27(4), 222–227. https://doi-org.ezproxy.lakeheadu.ca/10.1080/02783190509554322

​

Begum, et al. (2020). Effectiveness of Kumon Teaching Method for Academic Achievement of Children in Mathematics. Pakistan Journal of Education. Retrieved from <https://pdfs.semanticscholar.org/3ba0/d3b4a989aa3e30dedcb6dc2d5bf94121fa5f.pdf>. 

​

S, Medina. (1989). Study of the Effects of the Kumon MethodUpon the Mathematical Development of a Group of Inner-City Junior High School Students. Retrieved from <https://files.eric.ed.gov/fulltext/ED331700.pdf>.

Usmadi, et al. (2020). The Effect of Application Kumon Learning Method in Learning Mathematics of Ability Troubleshooting Mathematics of Students. Journal of Physics. Retrieved from <https://iopscience.iop.org/article/10.1088/1742-6596/1429/1/012005/pdf>. 

​

Sulasteri, et al. (2020). The effect of Kumon learning model on mathematics learning outcomes in cognitive style view. Journal of Physics. 

​

NELSON, G.; MCMASTER, K. L. The Effects of Early Numeracy Interventions for Students in Preschool and Early Elementary: A Meta-Analysis. Journal of Educational Psychology, [s. l.], v. 111, n. 6, p. 1001–1022, 2019. DOI 10.1037/edu0000334. Disponível em: http://search.ebscohost.com.ezproxy.lakeheadu.ca/login.aspx?direct=true&db=eue&AN=137730895&site=ehost-live. Acesso em: 25 jul. 2020.

 

S, Parish. (2014). Number Talks. Math Solutions. 

​

J, Hattie. (2018). Hattie Ranking: 252 Influences And Effect Sizes Related To Student Achievement Visible Learning. Retrieved from <https://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/>. 

 

A, Stewarts. (2016). The Impact of Daily Number Talks on the Development of Mental Math Abilities of Second Graders within a Reform-Based Classroom. Lakehead University. Retrieved from <https://knowledgecommons.lakeheadu.ca/bitstream/handle/2453/4235/StewartA2018m-1b.pdf?sequence=1&isAllowed=y>.

 

Gersten, Chard, Jayanthi, Baker, Morphy, Flojo. (2009). A Meta-analysis of Mathematics Instructional Interventions for Students with Learning Disabilities: Technical Report. Instructional Research Institute. Retrieved from <http://3evoie.org/telechargementpublic/usa/gersten2009a.pdf?fbclid=IwAR0c-XjNJoSNy2dDvfWEwOqBl5EqtuFpU5GkW6s4QM7-jpuY90-I85Q5dyI>.

 

M, Duffy. (2017). Can Frequent Use Of Number Talks Increase The

Comprehension, Understanding, And Fluency Of Fractions, Decimals, And Percentages In Alternative High School Students? Hamline University. Retrieved from <https://digitalcommons.hamline.edu/cgi/viewcontent.cgi?article=5349&context=hse_all>. 

 

May, P. L. (2020). Number Talks Benefit Fifth Graders’ Numeracy. International Journal of Instruction, 13(4), 361–374. https://doi-org.ezproxy.lakeheadu.ca/10.29333/iji.2020.13423a

​

Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084

 

J, Hattie. (2022). Meta-X. Visible Learning. Retrieved from <https://www.visiblelearningmetax.com/influences>. 

 

Ran. (2022). A meta‐analysis on the effects of technology’s functions and roles on students’ mathematics achievement in K‐12 classrooms. Journal of Computer Assisted Learning., 38(1), 258–284.

 

Kacmaz. (2022). Examining pedagogical approaches and types of mathematics knowledge in educational games: A meta-analysis and critical review. Educational Research Review, 35, N.PAG.

 

SAYGILI, H., & ÇETİN, H. (2021). The Effects of Learning Management Systems (LMS) on Mathematics Achievement: A Meta-Analysis Study. Necatibey Faculty of Education Electronic Journal of Science & Mathematics Education, 15(2), 341–362. https://doi-org.ezproxy.lakeheadu.ca/10.17522/balikesirnef.1026534

 

Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016). What One Hundred Years of Research Says About the Effects of Ability Grouping and Acceleration on K–12 Students’ Academic Achievement: Findings of Two Second-Order Meta-Analyses. Review of Educational Research, 86(4), 849–899. https://doi.org/10.3102/0034654316675417

 

J, Hattie. (2022). Peer Tutoring. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/peer_tutoring>. 

 

J, Hattie. (2022). RTI. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/response_to_intervention> 

 

J, Hattie, (2022). Teaching to Learning Styles. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/matching_teaching_to_style_of_learning>. 

 

Deunk. (2018). Effective differentiation Practices:A systematic review and meta-analysis of studies on the cognitive effects of differentiation practices in primary education. Educational Research Review, 24, 31–54.

 

E, Emanet, Et al.(2021). The Effects of Student-Centered Teaching Methods Used in Mathematics Courses on Mathematics Achievement, Attitude, and Anxiety: A Meta-Analysis Study. Participatory Educational Research. Vol 8(2), PP. 240-259. 

​

J, Hattie. (2022). Student Centered Teaching. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/student-centered_teaching>. 

 

J, Hattie. (2022). Problem Based Learning. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/problem-based_learning>.

 

J, Hattie. (2022). Collaborative Teaching. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/collaborative_learning>. 

 

J, Hattie. (2022). Student Choice. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/school_choice>. 

Disabilities: A Selective Meta-Analysis of the Literature. Learning Disability Quarterly, 44(4), 248–260. https://doi-org.ezproxy.lakeheadu.ca/10.1177/0731948721994843

​

J, Myers. (2022). A Meta-Analysis of Mathematics Word-Problem Solving Interventions for Elementary Students Who Evidence Mathematics Difficulties. The Review of Educational Research. Retrieved from <https://journals-sagepub-com.ezproxy.lakeheadu.ca/doi/full/10.3102/00346543211070049>. 

 

Cason, M., Young, J., & Kuehnert, E. (2019). A meta-analysis of the effects of numerical competency development on achievement: Recommendations for mathematics educators. Investigations in Mathematics Learning, 11(2), 134–147. https://doi.org/10.1080/19477503.2018.1425591

​

Ennis, R. P., & Losinski, M. (2019). Interventions to Improve Fraction Skills for Students With Disabilities: A Meta-Analysis. Exceptional Children, 85(3), 367–386. https://doi-org.ezproxy.lakeheadu.ca/10.1177/0014402918817504

​

Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084

 

Fan. (2017). Homework and students’ achievement in math and science: A 30-year meta-analysis, 1986–2015. Educational Research Review, 20, 35–54.

 

Methe, S., Kilgus, S., Neiman, C., & Chris Riley-Tillman, T. (2012). Meta-Analysis of Interventions for Basic Mathematics Computation in Single-case Research. Journal of Behavioral Education, 21(3), 230–253. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s10864-012-9161-1

 

Myers. (2021). Mathematics Interventions for Adolescents with Mathematics Difficulties: A Meta‐Analysis. Learning Disabilities Research & Practice., 36(2), 145–166.

​

R, Getsen, Et al. (2009). A Meta-analysis of Mathematics Instructional Interventions for Students with Learning Disabilities:

​

J, Myers. (2022). A Meta-Analysis of Mathematics Word-Problem Solving Interventions for Elementary Students Who Evidence Mathematics Difficulties. The Review of Educational Research. Retrieved from <https://journals-sagepub-com.ezproxy.lakeheadu.ca/doi/full/10.3102/00346543211070049>. 

 

Ellington, A.J. (2003). A Meta-Analysis of the Effects of Calculators on Students' Achievement and Attitude Levels in Precollege Mathematics Classes. Journal for Research in Mathematics Education, 34, 433.