#Research Applied Evenly

#Headlines dealt with “Rule #1.” (Link) P. David Pearson at #ILA19 was a panel member for a Saturday 7 a.m. session titled: “What Research Says About Teaching Reading and Why that Still Matters.” Dr. Pearson proposed several rules for our work and I have been considering this second rule for the last few weeks as I have read across Twitter, blogs, emails, newspapers and journal articles.

Rule 1:  Policymakers have to read beyond the headlines.

Rule 2 is captured here.

Screenshot 2019-12-08 at 10.52.07 AM

Let’s return to

Results are in: Mississippi students No. 1 in the country for reading gains (Link)

What research is reported?

All the research?

Dictionary.com defines research as:

Screenshot 2019-12-08 at 5.03.39 PM

Go check out the article and identify the “research” you find.

. . .

Hmm

. . .

Hmm

. . .

Data

Data

Data

Reporting of “findings” or “results”

Hinted at in this section:

The Mississippi Department of Education attributed the some of the continued success in reading scores to the Literacy-Based Promotion Act, a law that went into effect in 2013 that requires third-graders to pass a reading test before they can be promoted to the fourth grade. This year marks the first where students had to hit a higher bar to move up a grade.

“Mississippi has entered a new era of public education,” said Jason Dean, chair of the Mississippi State Board of Education. “Our significant improvements in teaching and learning have made Mississippi a national leader for improving student success in education.”

The linked article about “a higher bar” took me to this article with this quote in the final paragraph.

Woods is one of dozens of literacy coaches working in classrooms across the state as thousands of third-grade students prepare for their final chance at passing a critical reading exam. Should they fail, the possibility looms heavy that they’ll have to repeat the grade. (Source Link)

Facts

  • Picture dated 6/17/19 labeled with coach and students
  • Dozens of literacy coaches
  • Thousands of third-grade students
  • PREPARE
  • Final chance at passing a critical reading exam
  • possibility to repeat a grade

Questions/ Wonderings

  1. Many schools in MS, begin in the second week of August. Was this a summer school program extending the year?
  2. Final chance:  How many opportunities had students already had for this test? Beginning when?  How frequently could students retake for another chance?
  3. What is this test?
  4. Is this test aligned with NAEP?
  5. What is the technical adequacy of this test?
  6. Have third grade teachers in MS seen the test questions?
  7. Does test prep occur during the school year in the third grade classrooms?
  8. How many third grade students had to repeat a grade?
  9. What are the “significant improvements in teaching and learning”? (Jason Dean)
  10. …. (Add your own)

Additional information gathering – Mississippi Dept. of Ed. 2013 Literacy-Based Promotion Act (link)

  • Train K-3 teachers, curriculum specialists and other educators
  • Research-based instructional strategies
  • 2014-15 retention for lowest students unless “good cause exemption”
  • Law modified in 2016
  • Fall of 2018 literacy coaches were deployed

Additional “digging” on the site, training was in LETRS (subject of IES study link)

Result from RTC study: What did the study authors report about the efficacy of LETRS?

Providing second-grade teachers training based on the LETRS curriculum (with or without the instructional coaches) increased their knowledge of reading instruction techniques and their use of explicit instruction. However, it did not increase the reading test scores of their students {emphasis added}.

The authors estimated effect sizes on reading scores that ranged from 0.03 to 0.08. These estimates were not statistically significant.

Questions about the training:

  • How many days of training did the Mississippi teachers have?
  • What was the implementation plan for reviewing the instruction in the classrooms?
  • What percentage of teachers implemented their training as presented?
  • What percentage of teachers were observed for fidelity of implementation?
  • What percentage of teachers studied their own implementation of the instructional changes?
  • Mathematically, what was the benefit to students in terms of Cost of Teacher Training (K-3) x # of Training Days (cumulative # for all years) / Number of Students (counted only once)?

What do I now know?

So some facts were reported in the initial article.

Some generalizations about student performance were made.

No research was reported.  In fact a prominent journalist claims “There’s no way to know for sure what causes increases in test scores.

I added in research from a What Works Clearinghouse report on the effectiveness of LETRS.

How can causation or correlation be implied for this “growth of 4 points”?

And what is the significance of “outlier growth” in typical research?

Studying the “growth” for 4th grade students in MS would be an appropriate action from a group who advocates for “science”. 

Rule 2 for RESEARCH was not applied in the original article. You can “judge” whether any research is applied in additional articles on the same topic.




CHALLENGE:

Choose an article, any article, that supports “Science of Reading”. Identify the precise research in the article. Study it. What do you really find?




 

 

Mississippi 2019 NAEP Results Examined – Link

6 responses

  1. […] Rule 2 #Research Applied Evenly […]

  2. This rabbit hole you went down is fascinating. And I wonder, how many educators take in the challenge you offer. Data in the headlines is what people see, and I believe what politicians are seeking. This Mississippi situation seems to be a drastic move to save political face. Not how to develop teacher capacity or student learning. I fear so many decisions are made this way. Recently on our district’s home page they are highlighting a new data point based on school growth “from point a to point b”. They don’t define the points. But it does produce lovely colored graphs for every school. What does it really mean? Time to start digging.

    1. Julieanne,
      It quickly became a LARGE rabbit warren – not just a hole! It is so easy to get caught up in “incomplete” information or even “confirmation bias” – definitely complicated. But NOW is not the time to accept anything on face value alone!

  3. […] Rule 2 #Research Applied Evenly […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Tim's Teaching Thoughts

Ideas and Reflections on Teaching

Hands Down, Speak Out

Listening and Talking Across Literacy and Math

Teachers | Books | Readers

Thirty-One Educators Connecting Students and Books

Educator *Speaker *Author*coach

We have the perfect words. Write when you need them. www.carlambrown.com

Curriculum Coffee

A Written Shot of Espresso

Mrs. Palmer Ponders

Noticing and celebrating life's moments of any size.

doctorsam7

Seeking Ways to Grow Proficient, Motivated, Lifelong Readers & Writers

Doing The Work That Matters

a journey of growing readers & writers

Present Perfect

adventures in multiple tenses

The Blue Heron (Then Sings My Soul)

The oft bemused (or quite simply amused) musings of Krista Marx -- a self-professed HOPE pursuing Pollyanna

Middle English

Life as an English teacher leader

steps in the literacy journey

Walking the Path to Literacy Together

arjeha

Smile! You’re at the best WordPress.com site ever

Resource - Full

Sharing Ideas, Strategies and Tools

Joel Pedersen

be that #oneperson

adventuresinstaffdevelopment

All Things Literacy! Brianna Parlitsis

TWO WRITING TEACHERS

A meeting place for a world of reflective writers.

elsie tries writing

"The problem with people is they forget that that most of the time it's the small things that count." (Said by Finch in All the Bright Places by Jennifer Niven. These are my small things that count.

%d bloggers like this: