#Research Applied Evenly

#Headlines dealt with “Rule #1.” (Link) P. David Pearson at #ILA19 was a panel member for a Saturday 7 a.m. session titled: “What Research Says About Teaching Reading and Why that Still Matters.” Dr. Pearson proposed several rules for our work and I have been considering this second rule for the last few weeks as I have read across Twitter, blogs, emails, newspapers and journal articles.

Rule 1:  Policymakers have to read beyond the headlines.

Rule 2 is captured here.

Let’s return to

Results are in: Mississippi students No. 1 in the country for reading gains (Link)

What research is reported?

All the research?

Dictionary.com defines research as:

Go check out the article and identify the “research” you find.

. . .

Hmm

. . .

Hmm

. . .

Data

Data

Data

Reporting of “findings” or “results”

Hinted at in this section:

The Mississippi Department of Education attributed the some of the continued success in reading scores to the Literacy-Based Promotion Act, a law that went into effect in 2013 that requires third-graders to pass a reading test before they can be promoted to the fourth grade. This year marks the first where students had to hit a higher bar to move up a grade.

“Mississippi has entered a new era of public education,” said Jason Dean, chair of the Mississippi State Board of Education. “Our significant improvements in teaching and learning have made Mississippi a national leader for improving student success in education.”

The linked article about “a higher bar” took me to this article with this quote in the final paragraph.

Woods is one of dozens of literacy coaches working in classrooms across the state as thousands of third-grade students prepare for their final chance at passing a critical reading exam. Should they fail, the possibility looms heavy that they’ll have to repeat the grade. (Source Link)

Facts

• Picture dated 6/17/19 labeled with coach and students
• Dozens of literacy coaches
• Thousands of third-grade students
• PREPARE
• Final chance at passing a critical reading exam
• possibility to repeat a grade

Questions/ Wonderings

1. Many schools in MS, begin in the second week of August. Was this a summer school program extending the year?
2. Final chance:  How many opportunities had students already had for this test? Beginning when?  How frequently could students retake for another chance?
3. What is this test?
4. Is this test aligned with NAEP?
5. What is the technical adequacy of this test?
6. Have third grade teachers in MS seen the test questions?
7. Does test prep occur during the school year in the third grade classrooms?
8. How many third grade students had to repeat a grade?
9. What are the “significant improvements in teaching and learning”? (Jason Dean)
10. …. (Add your own)

Additional information gathering – Mississippi Dept. of Ed. 2013 Literacy-Based Promotion Act (link)

• Train K-3 teachers, curriculum specialists and other educators
• Research-based instructional strategies
• 2014-15 retention for lowest students unless “good cause exemption”
• Law modified in 2016
• Fall of 2018 literacy coaches were deployed

Additional “digging” on the site, training was in LETRS (subject of IES study link)

Result from RTC study: What did the study authors report about the efficacy of LETRS?

Providing second-grade teachers training based on the LETRS curriculum (with or without the instructional coaches) increased their knowledge of reading instruction techniques and their use of explicit instruction. However, it did not increase the reading test scores of their students {emphasis added}.

The authors estimated effect sizes on reading scores that ranged from 0.03 to 0.08. These estimates were not statistically significant.

Questions about the training:

• How many days of training did the Mississippi teachers have?
• What was the implementation plan for reviewing the instruction in the classrooms?
• What percentage of teachers implemented their training as presented?
• What percentage of teachers were observed for fidelity of implementation?
• What percentage of teachers studied their own implementation of the instructional changes?
• Mathematically, what was the benefit to students in terms of Cost of Teacher Training (K-3) x # of Training Days (cumulative # for all years) / Number of Students (counted only once)?

What do I now know?

So some facts were reported in the initial article.

Some generalizations about student performance were made.

No research was reported.  In fact a prominent journalist claims “There’s no way to know for sure what causes increases in test scores.“

I added in research from a What Works Clearinghouse report on the effectiveness of LETRS.

How can causation or correlation be implied for this “growth of 4 points”?

And what is the significance of “outlier growth” in typical research?

Studying the “growth” for 4th grade students in MS would be an appropriate action from a group who advocates for “science”. 

Rule 2 for RESEARCH was not applied in the original article. You can “judge” whether any research is applied in additional articles on the same topic.

---

---

---

CHALLENGE:

Choose an article, any article, that supports “Science of Reading”. Identify the precise research in the article. Study it. What do you really find?

---

---

---

 

 

Mississippi 2019 NAEP Results Examined – Link