#Full portfolio of methods
When I return to my cooking thoughts from yesterday, I have to think of methodology and resources. Will I use “glass microwaveable” dishes in the microwave? A double boiler on the stove?
And what about the fudge? Do I really “butter” the pan? Not that nasty cooking spray either! Can I just use parchment paper to line the pan? (Shudder as I think of butter/oleo visible on the 9 x 13 glass casserole plan! Total ICK!)
When do I follow the directions to the letter vs. letting previous experience guide my planning?
Today’s post is considering Rule 4 from P. David Pearson’s presentation as a part of an #ILA19 panel titled: “What Research Says About Teaching Reading and Why that Still Matters.”
So who are the “cousins”? These are some possibilities from the table in “10 Things Every Literacy Educator Should Know About Research.” (Link)
(Note. The information in this table was drawn in part from “Standards for Reporting on Empirical Social Science Research in AERA Publications: American Educational Research Association,” by P.A. Moss, J.W. Pellegrino, B.L. Schneider, R.P. Duran, M.A. Eisenhart, F.D. Erickson, et al., 2006, Educational Researcher, 35(6), 33–40; “Qualitative Analysis on Stage: Making the Research Process More Public,” by V.A. Anfara, Jr., K.M. Brown, and T.L. Mangione, 2002, Educational Researcher, 31(7), 28–38; Literacy Research Methodologies, by N.K. Duke and M.H. Mallette (Eds.), 2004, New York: Guilford; Literacy Research Methodologies (2nd ed.), by N.K. Duke and M.H. Mallette (Eds.), 2011, New York: Guilford; and Educational Research: An Introduction (8th ed.), by M.D. Gall, J.P. Gall, and W.R. Borg, 2007, Boston: Allyn & Bacon.)
The methodology is not set in concrete, but it has to make sense and follow general research principles. All of these involve “science.” ALL. of. these. involve. “science.”
Some seem to over emphasize RCTs – Randomized Controlled Trials. We saw that in the “gold standard” in Reading First. And meta analyses were NEVER allowed but some RCTs just are NOT possible in education. Controlling for every thing in the environment is tough even when two classrooms sit side by side. Equally difficult is the history of single-subject experimental designs. At one point, single-subject experimental designs were the most favored and at other times they were not indicative of “authentic” treatments in classrooms so they were used more infrequently.
Narrowing the field to only one methodology is, in my mind, similar to giving someone a math problem and saying that you can only use addition to solve it. No other process. Just one.
Not helpful. Not logical. Totally restrictive for no real reason.
More productive thinking about the math problem could be multiple routes to solutions with the use of several processes. The solutions could be studied for efficiency or effectiveness . . . or “innovative” status.
What doesn’t count?
Relying on “The Google”
Relying on “Op-Ed” Pieces
Or 
Do the Work.
Stop.
Think.
Does this make sense?
What do you need to add to your repertoire to have a “full portfolio of methods?”
Where will you begin?
When will previous experience guide methodology?
If you have not been following along, here are the posts to date:
#Research Applied Evenly
#Headlines dealt with “Rule #1.” (Link) P. David Pearson at #ILA19 was a panel member for a Saturday 7 a.m. session titled: “What Research Says About Teaching Reading and Why that Still Matters.” Dr. Pearson proposed several rules for our work and I have been considering this second rule for the last few weeks as I have read across Twitter, blogs, emails, newspapers and journal articles.
Rule 1: Policymakers have to read beyond the headlines.
Rule 2 is captured here.
Let’s return to
Results are in: Mississippi students No. 1 in the country for reading gains (Link)
What research is reported?
All the research?
Dictionary.com defines research as:
Go check out the article and identify the “research” you find.
. . .
Hmm
. . .
Hmm
. . .
Data
Data
Data
Reporting of “findings” or “results”
Hinted at in this section:
The Mississippi Department of Education attributed the some of the continued success in reading scores to the Literacy-Based Promotion Act, a law that went into effect in 2013 that requires third-graders to pass a reading test before they can be promoted to the fourth grade. This year marks the first where students had to hit a higher bar to move up a grade.
“Mississippi has entered a new era of public education,” said Jason Dean, chair of the Mississippi State Board of Education. “Our significant improvements in teaching and learning have made Mississippi a national leader for improving student success in education.”
The linked article about “a higher bar” took me to this article with this quote in the final paragraph.
Woods is one of dozens of literacy coaches working in classrooms across the state as thousands of third-grade students prepare for their final chance at passing a critical reading exam. Should they fail, the possibility looms heavy that they’ll have to repeat the grade. (Source Link)
Facts
- Picture dated 6/17/19 labeled with coach and students
- Dozens of literacy coaches
- Thousands of third-grade students
- PREPARE
- Final chance at passing a critical reading exam
- possibility to repeat a grade
Questions/ Wonderings
- Many schools in MS, begin in the second week of August. Was this a summer school program extending the year?
- Final chance: How many opportunities had students already had for this test? Beginning when? How frequently could students retake for another chance?
- What is this test?
- Is this test aligned with NAEP?
- What is the technical adequacy of this test?
- Have third grade teachers in MS seen the test questions?
- Does test prep occur during the school year in the third grade classrooms?
- How many third grade students had to repeat a grade?
- What are the “significant improvements in teaching and learning”? (Jason Dean)
- …. (Add your own)
Additional information gathering – Mississippi Dept. of Ed. 2013 Literacy-Based Promotion Act (link)
- Train K-3 teachers, curriculum specialists and other educators
- Research-based instructional strategies
- 2014-15 retention for lowest students unless “good cause exemption”
- Law modified in 2016
- Fall of 2018 literacy coaches were deployed
Additional “digging” on the site, training was in LETRS (subject of IES study link)
Result from RTC study: What did the study authors report about the efficacy of LETRS?
Providing second-grade teachers training based on the LETRS curriculum (with or without the instructional coaches) increased their knowledge of reading instruction techniques and their use of explicit instruction. However, it did not increase the reading test scores of their students {emphasis added}.
The authors estimated effect sizes on reading scores that ranged from 0.03 to 0.08. These estimates were not statistically significant.
Questions about the training:
- How many days of training did the Mississippi teachers have?
- What was the implementation plan for reviewing the instruction in the classrooms?
- What percentage of teachers implemented their training as presented?
- What percentage of teachers were observed for fidelity of implementation?
- What percentage of teachers studied their own implementation of the instructional changes?
- Mathematically, what was the benefit to students in terms of Cost of Teacher Training (K-3) x # of Training Days (cumulative # for all years) / Number of Students (counted only once)?
What do I now know?
So some facts were reported in the initial article.
Some generalizations about student performance were made.
No research was reported. In fact a prominent journalist claims “There’s no way to know for sure what causes increases in test scores.“
I added in research from a What Works Clearinghouse report on the effectiveness of LETRS.
How can causation or correlation be implied for this “growth of 4 points”?
And what is the significance of “outlier growth” in typical research?
Studying the “growth” for 4th grade students in MS would be an appropriate action from a group who advocates for “science”.
Rule 2 for RESEARCH was not applied in the original article. You can “judge” whether any research is applied in additional articles on the same topic.
CHALLENGE:
Choose an article, any article, that supports “Science of Reading”. Identify the precise research in the article. Study it. What do you really find?
Mississippi 2019 NAEP Results Examined – Link
#Headlines
I remember professional learning with Emily Calhoun where we discussed how the “title” of any book was the promise the author makes to the reader about what the book will be about. A book is bigger than an article in a newspaper or a magazine. Is a headline similar? This lead me to some research about headlines and the types, functions, and even the definitions. Scacco and Muddiman in “The Current State of News Headlines” report four functions of headlines.
The news headline can serve a variety of functions, including story summarization, interest generation, immediacy satisfaction, and attention direction. (Link)
-
- Story summarization
- Interest generation
- Immediacy satisfaction
- Attention direction
Consider this headline. Which function fits?
Results are in: Mississippi students No. 1 in the country for reading gains
Do you know enough to make a decision?
This headline was published in Mississippi Today and according to its website,
Mississippi Today is a nonprofit 501(c)(3) news and media company with a forward-facing mission of civic engagement and public dialog through service journalism, live events and digital outreach. (Link)
Does that descriptor of the publication change your mind about the function of this particular headline?
The first paragraph of this article says,
Mississippi was the only state in the country to improve reading scores, and was number one in the country for gains in fourth-grade reading and math, according to newly released test results.
and yes, it was published the day after the NAEP results were released. (Link)
Has your view of the FUNCTION changed based on a) additional knowledge about the publisher; b) the knowledge of date of publication; and/or c) the first paragraph of the publication?
Which best fits your thinking?
-
- Story summarization
- Interest generation
- Immediacy satisfaction
- Attention direction
Why does it matter?
P. David Pearson at #ILA19 was a panel member for a Saturday 7 a.m. session titled: “What Research Says About Teaching Reading and Why that Still Matters.” Dr. Pearson proposed several rules for our work and I have been considering this first rule over the last two months as I have read across Twitter, blogs and newspapers.
Rule 1: Policymakers have to read beyond the headlines.
I would be remiss if I didn’t note that teachers, administrators, college instructors, parents, and anyone in the U.S. REALLY needs to read beyond the headlines. But careful attention is required particularly in the field of reading, reading instruction, and any “claims” in the headlines about reading pedagogy.
What does this article tell us?
Gains in 4th grade reading in Mississippi.
Only state with gains in 4th reading.
No gains in 8th grade reading in Mississippi.
The gain was 4 points.
“The 2019 results mark the first time Mississippi has met or outperformed national averages.” (In 1992, Mississippi was 16 points below the national average.)
Mississippi scores declined from 2009 to 2013.
And the “credit” for the “increase in scores”:
The Mississippi Department of Education attributed the some of the continued success in reading scores to the Literacy-Based Promotion Act, a law that went into effect in 2013 that requires third-graders to pass a reading test before they can be promoted to the fourth grade.
Do those facts match up with the function of the headline?
For additional practice let’s consider a second view of the Mississippi scores found in this blog post from Paul Thomas last week.
Mississippi Miracle or Mirage?:
2019 NAEP Reading Scores Prompt Questions, Not Answers
Which function matches this headline?
-
- Story summarization
- Interest generation
- Immediacy satisfaction
- Attention direction
And here’s the first paragraph of the blog post.
There is a disturbing contradiction in the predicted jubilant response to Mississippi’s outlier 4th-grade results from the 2019 NAEP reading test. That contradiction can be found in a new article by Emily Hanford, using Mississippi to recycle her brand, a call for the “science of reading.”
What do you believe is the purpose of this headline? Is it similar to the previous article? Or different?
Dr. Thomas then quotes two paragraphs from E Hanford’s own post:
The state’s performance in reading was especially notable. Mississippi was the only state in the nation to post significant gains on the fourth-grade reading test. Fourth graders in Mississippi are now on par with the national average, reading as well or better than pupils in California, Texas, Michigan and 18 other states.
What’s up in Mississippi? There’s no way to know for sure what causes increases in test scores [emphasis added], but Mississippi has been doing something notable: making sure all of its teachers understand the science of reading.
Paul answers Hanford’s claim that “there’s no way to know for sure” with
To be fair, there is a way to know, and that would be conducting scientific research that teases out the factors that can be identified as causing the test score changes in the state.
Scientific research . . .
A broader look at the data suggests that in 1998, Mississippi was only 10 points below the national average. What changed to cause growth between 2005 -2009?
Blog post Source Link
Facts/Questions from this article:
Is the 2013 legislation responsible for any growth? What research supports this hypothesis/generalization?
Is this the role of NAEP data? Should it REALLY be?
What about the 8-point jump in 4th-grade reading in MS from 2002 to 2009 with no explanation?
Original author Hanford used correlation (not scientific) instead of causation (scientific).
Premature?
Irresponsible?
No research?
No evidence?
In any informational text, the information that is included is always critical. But equally important (Hat tip to Katie Clements) is the information that is left out. And the questions that remain after reading the articles. One place for readers to begin in with the promise of the headline, the match between the headline and the article content, and the basic functions of a headline are one entry point.
Did the articles match up to the “hype” of the headlines?
Did they serve the function?
Why is P. David Pearson’s rule about headlines important?
Skinner, K. Results are in: Mississippi students No. 1 in the country for reading gains. Retrieved from https://mississippitoday.org/2019/10/30/results-are-in-mississippi-students-no-1-in-the-country-for-reading-gains/ on December 7, 2019.
Thomas, PL. Mississippi Miracle or Mirage?: 2019 NAEP Reading Scores Prompt Questions, Not Answers. Retrieved from https://radicalscholarship.wordpress.com/2019/12/06/mississippi-miracle-or-mirage-2019-naep-reading-scores-prompt-questions-not-answers/ on December 7, 2019.
#SOL19: Generosity
Face to Face
They come from across the world. 10,000 strong. Teachers, Students, Administrators, Authors. Techies. First time attendees. Veteran attendees. The results from long distance planning to present together. Planning to share a room. Planning, chatting and sharing sessions to attend, places to go, and glorious meet ups. They come by planes, trains, and cars. Some add on early travel and pick up the Poe Historical Tour. Others stay for #ALAN19. So many choices. So many possibilities. 10,000 friends meeting for: conversation, celebration, food, drink, a Read Aloud, laughter . . . and sharing!
CURATED RESOURCES:
5. Twitter Thread of Top 10 Learnings by Kass Minor
4. Interesting Blog: Conference FOMO? I really didn’t mean to make you feel left out (Link)
3. A PhotoStory Blog #NCTE19: A Collection and Reflection (Here)
2. On Gratitude (Link)
- BEST#NCTE19 Notes: Paula Bourque blog post titled Vicarious PD: Sharing the Wealth of #NCTE19 (Link)
Why? All of Paula’s notes are organized and linked! All. 12. of. them.
What other posts should be added?
Thank you, Two Writing Teachers, for this weekly forum. Check out the writers and readers here.
#SOL19: Fueling the Soul
I returned to #NCTE19 in the same site as #NCTE14 to present as part of a different panel group. Excited to rejoin face-to-face friends and colleagues. Exhilarated to learn with new friends and colleagues and just a bit exhausted from the prep and planning to take advantage of every single moment. Celebrating friends. Celebrating peers. Celebrating communities. Celebrating learning. So ready to lean into my #OLW: Celebrate!
NCTE: National Council of Teachers of English. So many folks from so many places. One night around the table, we represented Illinois, Iowa, Louisiana, Virginia, Virginia, Colorado, Pennsylvania, Connecticut, California, Connecticut, and Michigan. That was the night of the Slicer dinner. Two new friends. Many face to face friends. Slicers all who intersect with #G2Great, #TCRWP, #CCIRA and our #NCTE presentation – the four of us together for the first time! Talking. Sharing. A laughing video of a grandson. Sharing of children’s artwork. Shared quotes. Food, drink, conversation, and fun. With just a touch of rain that did not dampen our spirits!
There is nothing like scintillating conversation, learning with peers, celebrating with authors, and after hours gatherings to fuel the soul . . . sparking a joyous celebration of friends, families, and ever increasing meet ups of social media friends. As the world shrinks when we write and speak collaboratively on social platforms, our knowledge base grows exponentially.
As I continue to reflect on my travel and learning while I sift through my notes, I will add three outside sources here.
One of my favorites from NCTE is Kelly Gallagher’s Top Ten Things he heard at NCTE:
Melanie Meehan, co-author of Two Writing Teachers wrote about three sessions here.
Stop and Think Reading List and Resources here.
How do you collect and organize your learning?
How are you refueling your professional soul?
Thank you, Two Writing Teachers, for this weekly forum. Check out the writers and readers here.
#NCTE19: The Beginning
Back to the beginning: Baltimore again. The crew. Face to Face connections. Twitter. Learning, Laughing. Sharing.
NCTE18 – 10 posts
NCTE17 – 4 posts
NCTE16 – 4 posts
NCTE15 – 2 posts
NCTE14 – 5 posts
Celebrating 25 posts already written about NCTE! Anticipating the posts and the learning from the next four days.
The program . . . link
on Twitter #NCTE19
in Baltimore
where my ancestor was born, George Herman Ruth.
In 2014, it was a Friday presentation described here.
In 2019, it will be a Sunday presentation as listed below.
Katelynn Giordano, Betsy Hubbard, Melanie Meehan, and myself
Challenged! Intrigued! Sparking Inquiry Through Collaborative Research
9-10:15 AM, Sunday, November 24 in Room 304
What are you anticipating for #NCTE19?
What is your plan?
What do you plan to learn?
What will you celebrate?
Resource - Full 