How do you make purchasing decisions about programs/materials? Vicki Vinton has a great post titled “What’s the Difference Between a Teacher and a Packaged Program? in case you have not yet read it. (I am not proposing that teachers should be replaced by a packaged program.)
Do you ask questions when you are reviewing supplementary computer reading materials before a purchasing decision? Who do you ask? Friends? Colleagues? Twitter PLN? What do you ask?
I consider myself fortunate as I was trained with a protocol to evaluate the quality of research according to the definition of “scientifically-based reading research” that was used under No Child Left Behind. I believe it helps “weed out silly stuff” pretty quickly.
Do you use a protocol or checklist to guide your review process? I have provided some guiding questions to think about as you read the research report summary (or dig into the actual research) that can also be found in its entirety through the International Reading Association. What additional information will you want to collect and review?
- What was the age/grade level of students in the study?
- Is there a match between the students in the study and students in your classroom?
- Is there a match between the “purposes/goals” of the computer reading programs and the “purposes/goals” of reading in your district and in the Common Core?
- Is there high-quality evidence as a result of the research?
- Does the research list an effect size?
- Was there a control group in the study? Or were students assigned to random groups?
- Is there evidence that the results were sustained over time? (Two years or more later)
- What resources (time, staff, technology) are needed to implement the program? Are the resources cost – prohibitive?
- How much professional development is needed to initiate and sustain the program?
- Is fidelity of implementation described well enough in the research to be replicated?
- Is the effect size .40 or greater? (range John Hattie says should be considered)
- When considering resources and “struggling readers” what effect size is needed in order for the reader to “close the gap” and reach grade level goals?
As you continue reading, please think about your reading process. What do you, as a reader, do to enhance your understanding? (Blog followers: “Are you close reading?”)
Marshall Memo 495 (7-23-13)
4. How Effective Are K-6 Supplementary Computer Reading Programs?
“Despite substantial investments in reading instruction over the past two decades, far too many U.S. students remain poor readers, which has profound implications for these children and for the nation,” say Alan Cheung (The Chinese University/Hong Kong) and Robert Slavin (Johns Hopkins University) in this Reading Research Quarterly article. “Learning to read is a complex task in which many things must go right for a student to become successful… Different students may be failing to learn to read adequately for different reasons. One student may recognize every letter and sound but be slow and uncertain in blending them into words. Another may be proficient in reading words but does not comprehend them or the sentences in which they appear. Yet another may lack vocabulary needed to comprehend texts.”
One-on-one tutoring is the most effective intervention for struggling readers, say Cheung and Slavin, but it’s expensive. What about software packages? “In theory, computers can adapt to the individual needs of struggling readers,” they say, “building on what they can do and filling in gaps” – plus, they’re motivating to students. This article reports on the efficacy of technology products on which there is solid research. The bottom line: effect sizes for almost all products are small (averaging .14) and almost all of them aren’t any better than non-computer approaches. Here are the specifics, from the most to the least effective programs:
– Lexia (Phonics-Based Reading and Strategies for Older Students): the mean effect size for Title I students is .67
– Captain’s Log (BrainTrain): median effect size .40
– RWT and LIPS for first graders at risk for dyslexia: overall effect size .32
– Fundamental Punctuation Practice, MicroRead, Spelling Program, and Word Attack program for fourth graders: effect size .30
– READ 180 for middle schools: weighted mean effect size of .24
– READ 180 for grade 4-6 students: overall effect size .21
– Jostens (an earlier version of Compass Learning): across three studies, the weighted mean effect size was .19.
– Alpine Skier, Tank Tactics, and Big Door Deal for fifth and sixth graders: median effect size .18
– Across 12 studies of supplemental Computer Assisted Instruction, the weighted mean effect size was .18.
– Thinking Reader: median effect size of .14 in vocabulary and .13 in comprehension
– Destination Reading: median effect size .12
– Computer Network Specialist for grades 2-5: effect sizes averaged .10
– Fast ForWord for grades 3-6: weighted mean effect size .06
– Failure Free Reading for third and fifth graders: combined effect size .05
– ReadAbout for fifth graders: weighted average effect size .04
– READ 180 for grade 4-6 students (in another district): overall effect size .03
– Destination Reading, Waterford, Headsprout, PLATO Focus, and Academy of Reading in first-grade classrooms: mean effect size .02 (the study didn’t break down individual programs)
– Leapfrog, READ 180, Academy of Reading, KnowledgeBox for fourth graders: effect size –.01 (no breakdown for individual programs)
“The most important practical implication of the review presented here is that there is a limited evidence base for the use of technology applications to enhance the reading performance of struggling readers in the elementary grades,” conclude Cheung and Slavin. “Within the existing literature, however, the largest effect sizes were found for small-group interventions that supplement first-grade instruction with phonetic activities integrating computer and non-computer activities and occupying substantial time each week.”
“Effects of Educational Technology Applications on Reading Outcomes for Struggling Readers: A Best-Evidence Synthesis” by Alan Cheung and Robert Slavin in Reading Research Quarterly, July/August/September 2013 (Vol. 48, #3, p. 277-299),
Cheung be reached at email@example.com.
What surprised you about the research? How will this information impact your work?
Did you do any close reading? How did you know?
Thanks for your response!