Recently, Fountas and Pinnell’s literacy instruction program Leveled Literacy Instruction (LLI) has faced intense scrutiny for its efficacy. However, much of the criticism and support of LLI has been qualitative in nature. For example, the Fountas and Pinnell website, has a bibliography of studies on LLI and cites that this research has been positive, as proof of their efficacy. However, followers of my research know that I believe efficacy (in education) can only be determined via quantitative measures and meta-analysis and the Fountas and Pinnell website does not list any quantitative information, in support of their programs efficacy.

I looked for a meta-analysis of the topic; however, I could not find any current meta-analysis of the topic, despite the fact that Fountas and Pinnell’s program is the most widely used English Language program in the world and despite the fact that it has faced intense scrutiny. I decided to do my own meta-analysis of the topic, to try and do my best job of answering the question is LLI evidence-based. Now one caveat that I will make is that this meta-analysis is of course not peer reviewed, so please take my findings with a grain of salt. In order to conduct this analysis, I did a search for LLI studies on the LLI website, on google, Education Source, and Sage Journal. I excluded all studies that did not have a control group and enough statistical detail to calculate an effect size. Next, I calculated effect sizes using the Cohen's d formula (for studies with sample sizes above 50) and Hedge's g (for studies with sample sizes below 50. 

 

Results:
On average, efficacy studies of LLI had an ES of .34, which is in the low range. Meaning that LLI does have a positive impact, but that impact is below the average impact of a literacy intervention and below the average impact of many other literacy programs. Phonics, for example, has an average ES according to both my own meta-analysis and the NRP meta-analysis a mean effect size of .45. This suggests that on average synthetic phonics is far superior to LLI. 

 

Discussion: 

While the mean ES of LLI is .34, there are several caveats that I feel I need to make. Firstly, all of these LLI intervention studies made use of small group instruction and small group instruction has been shown to have greater impacts than regular class instruction. I therefore think, the fact that LLI uses small group instruction likely inflates its results.

Additionally, a large number of the LLI studies were done by the same institute, the University of Memphis Center for Research Policy in Education (CRPE). Indeed, the LLI website only cites one study not done by this institute. These studies, as far as I could find, all showed effect sizes much more positive than all other studies done on the topic. Indeed one CRPE study found a mean ES of .46, which was a outlier compared to the rest of studies on the topic. I also found some strange statistical anomalies in the papers. For example, the 2015 paper by the institute, had 3 effect sizes based on identical standard deviations and results for both their control group, intervention group, and on their pre and post tests. The resulting effect sizes were also extremely high. For this reason, I excluded these 3 effect sizes from my calculations. 

Ultimately I am concerned that the CRPE might have some research bias, in favor of LLI; however, their studies on the surface appeared very well done. They had large sample sizes, were randomized control trials, and posted their raw data.

​

 

Final Grade: C+: The program is not research based, IE most of the principles are not supported by the meta-analysis data.

​

Qualitative Grade: 1/10

The program includes the following evidence based principles: comprehension instruction. 

​

Written by Nate Joseph

Last Edited 2022-07-24

​

References: 

Center for Research Policy in Education. (2012). Efficacy of the Leveled Literacy Intervention System for K–2 Urban Students: An Empirical Evaluation of LLI in Denver Public Schools Study Dates: 2011–2012. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2012.pdf>. 

Center for Research Policy in Education. (2010). Implementation of Effective Intervention: An Empirical Study to Evaluate the Efficacy of Fountas & Pinnell’s Leveled Literacy Intervention System (LLI) Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2010.pdf>. 

 

Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Abilene Independent School District 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-AISD_FULL_REPORT-web.pdf>. 

 

Center for Research Policy in Education. (2015). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Denver Public Schools 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-DPS_FULL_REPORT-web.pdf>. 

 

Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy

Intervention System for Students in Grades 3–5: Data Summary Report for Sandwich Public Schools 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-SPS_FULL_REPORT-web.pdf>. 

 

Heinmann. (2010). Levelled Literacy Intervention, Research and Data Collection Project. Fountas and Pinnell. Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_Research-and-Data-Collection-Project-Report.pdf>. 

 

Gonzalez, et al. (2018). Challenges in Adolescent Reading Intervention: Evidence from a Randomized Control Trial. Mathmatica Policy Research. Retrieved from <https://files.eric.ed.gov/fulltext/ED587404.pdf>. 

 

Lang, Laura & Torgesen, Joseph & Vogel, William & Chanter, Carol & Lefsky, Evan & Petscher, Yaacov. (2009). Exploring the Relative Effectiveness of Reading Interventions for High School Students. Journal of Research on Educational Effectiveness. 2. 149-175. 10.1080/19345740802641535. 

 

Metz, Tracie Jean, "A case study: Effects of using leveled literacy intervention on fourth and fifth grade students' reading achievement" (2014). ETD Collection for Fayetteville State University. AAI3581423.

https://digitalcommons.uncfsu.edu/dissertations/AAI3581423

 

J, Majewski. (2018). The effects of a leveled literacy inter acy intervention  (LLI) on elementary age students reading below grade level. Rowan University. Retrieved from <https://rdw.rowan.edu/cgi/viewcontent.cgi?article=3574&context=etd>. 

J, Hattie. (2021). Visible Learning Metax. Retrieved from <www.visiblelearningmetax.com>. 

N, Hansford. (2021). Morphology. Pedagogy Non Grata. Retrieved from <https://nathanielhansford.wixsite.com/website/morphology>. 

 

Linnea, et al. (2001). Systematic Phonics Instruction Helps Students Learn to Read: Evidence From the National Reading Panel’s Meta-Analysis. Ontario Institute for Studies in Education. Retrieved from <https://www.dyslexie.lu/JDI_02_02_04.pdf>.