Evidence Based Education
Is Content Knowledge Instruction Essential for Comprehension?
I often see the claim that teaching content knowledge, especially social studies curriculum, is part of the science of reading. Proponents of this claim, often assert that this instruction is essential for comprehension. The argument appears to be built on the notion that complex texts often require background knowledge to be understood. Before I get any further, let me just say, when I originally left teachers college my primary teachable subject was not reading, it was history and indeed some of the most enjoyable years of my teaching career were spent teaching history. Truthfully, I enjoy teaching social studies and history, far more than I ever will reading. That being said, my bias is that teaching a social studies curriculum should first and foremost benefit students' knowledge of social studies, not their reading comprehension skills. Secondly, I would hypothesize that background knowledge is more useful for older students whose reading material tends to be more content heavy. This is not to say that social studies is not important or that the two subjects cannot be synergistically taught. However, I am naturally skeptical of the idea that social studies instruction is essential for improving comprehension outcomes.
Of course, my biases are largely irrelevant in a scientific discussion. What matters is what the research shows. Hyejin Hwang et. al. conducted a meta-analysis on the impact of integrating social studies curriculum with language instruction in 2020.There meta-analysis looked at 35 experimental and quasi-experimental studies on K-5 students. The results of which can be seen below.
Looking at the above data, we see a strong impact for content instruction on vocabulary and comprehension outcomes, especially for primary students. However, the researchers did note that many of the assessments were non-standardized, which leads to inflated results, especially for outcomes like vocabulary or comprehension. If we only use standardized assessments, we get an effect size of .25 for comprehension, which is positive, but barely statistically significant and .61 for vocabulary. The .61 vocabulary effect size looked reassuring; however, the authors noted that the p-value was .42, indicating that the effects were extremely ranging and that the result was therefore not meaningful. Overall this data, would suggest that while teaching background knowledge is not a high yield strategy for improving comprehension or vocabulary outcomes, that it does benefit students, both in terms of their content knowledge and reading capability.
That being said, the authors also included the below chart, which showed the effect sizes for each individual study. And being the nerd that I am, I decided to do some secondary analysis, to see if I could find anything interesting.
The first thing that I noticed was that there was an incredibly wide range of effects within this analysis. For example, while the mean weighted effect found for comprehension was high, 16 out of 48 effects were negative, meaning that teaching content knowledge lowered comprehension outcomes 33% of the time. I also noticed that there was a ton of outlier data, with 7 effect sizes above 2. This phenomenon was likely the result of non standardized assessments. I decided to use an IQR outlier formula and review the results, after removing outlier data.
Once the outlier data was removed it appeared there was a small to moderate effect for teaching comprehension and vocabulary outcomes to elementary students, regardless of age. However, I still was not satisfied. I noticed that a large number of the studies were conducted by Williams, et. al. and that these studies appeared to show much higher outcomes than the rest of the research. I have highlighted these studies below for reference.
The comprehension outcomes in these studies were on average 75% higher than the average reported on standardized assessments and the vocabulary outcomes were 84% higher than the average reported on standardized assessments. Indeed all but one of these studies had an effect size classified as an outlier study with the IQR formula. To make matters more problematic, all of these studies were done with grade 2 students. Which massively inflated the k-2 results. Now none of this is to say that the original meta-analysis authors made a mistake for including this data, or that the experimental authors did something nefarious. But clearly what was happening in the Williams et. al. studies was not being replicated in the other studies, which leads me to believe that it was not the content knowledge instruction in these studies leading to the extreme results. But rather there had to be something unique in the study design or instruction, within these experiments. With this in mind, I reanalyzed the results without the Williams data. The results can be seen below.
Once the Williams studies are removed we see a much clearer trend, with much higher results for content instruction in older grades than in younger grades. Indeed while the results for older grades were moderate to high, the results in younger grades were low to moderate. With this analysis, I also did not control for standardized vs non-standardized assessments and presumably, if we controlled for only standardized assessments the results would be lower. These results appear much more rational and in line with other comprehension research. Logically, it does make more sense to teach content knowledge in later grades, where students need less foundational knowledge and where their curriculum is more likely to require it.
Content knowledge instruction is often advocated for the purposes of improving reading comprehension. However, comprehension instruction has been shown to be more effective in later grades, than earlier ones. For example Filderman et al. conducted a meta-analysis on reading comprehension instruction in 2021, of 64 studies, showing a mean effect size of .67 for grades 9 to 12 and .47 for grades 3 to 8. However, the Filderman et. al. paper did not control for standardized vs non standardized assessments, which might have inflated their results.
Comparatively, Silverman, et. al. conducted a meta-analysis in 2020 on k-5 students, which included 43 studies. Even more useful though, their study controlled for standardized vs non standardized assessment results. And while the custom assessment results were impressive, all standardized assessment outcomes were statistically insignificant. The results of this study can be seen below.
To help make this information more visually helpful, I conducted a secondary meta-analysis of all these results and put them in the below graph. Please note that for the Hyejin meta-analysis, I used the results that excluded the Williams results, as I felt these were the most fair.
Across this data, we see that content and comprehension instruction seems to show higher results in later grades. Of course, this begs this question, “What does this mean for actual instruction”? Just because comprehension and content instruction shows lower results in the earlier grades does not mean it should not be taught. Afterall most of these results are positive. However, I would argue it does mean, we should focus more on comprehension and background knowledge instruction in the later grades and less in the early grades.
As teachers we have a limited amount of time each day, and it makes sense to focus our instruction on the areas that have the most impact for our grade. Moreover, nerdy statistical analysis aside, there is an obviously rational argument for teaching more content/comprehension instruction in the later grades and less in the early grades. In the early grades, books tend to be simplistic. There tends to be no complex themes, symbolism, or meaningful content connection. The opposite is true for older grades. Kids don’t need comprehension instruction or background knowledge to understand the Berenstain Bears, but they do for Shakespeare or Richard Wagamese. Again, this is not to say we should never teach content knowledge or comprehension skills in the primary grades, but it likely should be far less of a focus than phonics or phonemic awareness.
In my personal opinion, the older and more developed our readers are the more content and comprehension instruction we should provide and conversely the less foundational instruction we should provide. For example, I would think there should be a minimal amount of comprehension and content instruction in Kindergarten. The content and comprehension instruction that is given should be delivered orally. Whereas students in grade 7 and 8 should likely be spending the majority of their instructional time on things like content knowledge, report writing, and comprehension instruction. That all being said, within the context of this discussion, I feel obliged to present a nuanced position, because content knowledge is uniquely important to provide for students, in all grades, as it is important part of their holistic development as future citizens. However, I think this type of knowledge should be presented mostly orally in younger grades and without the pre-text that this instruction is meant to improve reading outcomes.
Written by Nathaniel Hansford
Last edited 2022/08/23
References:
HyeJin Hwang, Sonia Q. Cabell & Rachel E. Joyner (2022) Effects of Integrated Literacy and Content-area Instruction on Vocabulary and Comprehension in the Elementary Years: A Meta-analysis, Scientific Studies of Reading, 26:3, 223-249, DOI: 10.1080/10888438.2021.1954005
Silverman, R. D., Johnson, E., Keane, K., & Khanna, S. (2020). Beyond Decoding: A Meta‐Analysis of the Effects of Language Comprehension Interventions on K–5 Students’ Language and Literacy Outcomes. Reading Research Quarterly, 55, S207–S233. https://doi-org.ezproxy.lakeheadu.ca/10.1002/rrq.346
Filderman, M. J., Austin, C. R., Boucher, A. N., O’Donnell, K., & Swanson, E. A. (2022). A Meta-Analysis of the Effects of Reading Comprehension Interventions on the Reading Comprehension Outcomes of Struggling Readers in Third Through 12th Grades. Exceptional Children, 88(2), 163–184. https://doi.org/10.1177/00144029211050860