#SOLSC21: Radical?
Yes, Radical voices from the middle describes me.
In life, we need a balance. In literacy we need balance. We will never make it to compre-action without skills. But skills are not the end goal. Action as a result of deeper understanding is the goal!
I agree with P. David Pearson.
Noted teacher, leader, researcher. You can read more about him here.
In 2020, he just completed a study of comprehension. Here’s the executive summary.
Headlines:
1. Knowledge is cause, consequence, and covariate of reading comprehension.
2. Language drives every facet of reading comprehension.
3. Reading is an inherently cultural activity.
Pearson, P. D., Palincsar, A. S., Biancarosa, G., & Berman, A. I. (Eds.). (2020). Reaping
the Rewards of the Reading for Understanding Initiative. Washington, DC: National Academy of Education.
Knowlege and language are essential for comprehension and compre-action, a term coined by P. David Pearson and Nell Duke. There’s no one easy, magic, silver bullet for improving reading comprehension. Part of that is because of the definition. Part of that is because of the way comprehension is assessed. Flaws in assessment exacerbate the issues.
Radical middle.
Yes, that’s me.
Let’s get rowdy.
Let’s get to work.
What seems radical about reading comprehension? What do you believe and value? Where will you learn more?
_______________________________________________________________________________________
Thank you, Two Writing Teachers, for this daily forum during the month of March. Check out the writers and readers here.
#Big Picture
During the last five daily blog posts, I have worked my way through the five rules from P. David Pearson and the #ILA19 panel session at 7 a.m. Saturday titled: “What Research Says About Teaching Reading and Why that Still Matters.”
Rule 2 #Research Applied Evenly
Rule 4 #Full portfolio of methods
Rule 5 #Evidence, not a strawperson
Understanding the research in today’s world takes some work, some thinking, and a good hard look at the evidence, the word that appears in both rule 3 and rule 5.
A week ago, this was how I started my first draft for the series. I quickly discovered as I wrote that this look at the Big Picture was the ending of the series instead of the beginning. The REAL beginning was the panel presentation that recentered some beliefs in processes and brought back a review process used by our Statewide Literacy Team in the past.
So let’s get started. “It was a dark and stormy night.” (I love how Snoopy works that into every story!)
Compare these headlines:
- ‘No Progress’ Seen in Reading or Math on Nation’s Report Card
- Screen Time Up as Reading Scores Drop. Is There a Link?
- The One And Only Lesson To Be Learned From NAEP Scores
-
Results are in: Mississippi Students #1 in the Country for Reading Gains
- Mississippi: Miracle or Mirage – 2019 NAEP Reading Scores Prompt Questions Not Answers
Match the quotes with the titles above. 1. #Headlines
_____ NAEP is extraordinarily clear that folks should not try to suggest a causal relationship between scores and anything else. Everyone ignores that advice, but NAEP clearly acknowledges that there are too many factors at play here to focus on any single one.
_____ In reading, Mississippi was the only state to improve in 2019 in 4th grade and Washington, D.C. (considered as a state) was the only one to improve in 8th grade. (The District of Columbia, in fact, showed the fastest gains this year of any state or large school district.)
_____ Todd Collins has raised another important caveat to the 4th-grade reading gains in Mississippi because the state has the highest 3rd-grade retention percentages in the country. . .
_____ Mississippi was the only state in the country to improve reading scores, and was number one in the country for gains in fourth-grade reading and math, according to newly released test results.
_____ Students have actually lost ground since 2017 on both of the NAEP’s main reading content areas: literary experience, such as fiction analysis, and reading for information, such as finding evidence to support an argument. Both grades declined significantly in both areas from 2017 to 2019, but the drop was larger for literary skills.
Which ones seemed pretty obvious?
Which ones took a bit more thought?
And then which two came from the same publisher?
. . .
. . .
#1 Headlines and text that supports or matches the headline.
Answers:
3, 1, 5, 4, 2.
Same Publisher: 1 and 2 were both EdWeek
Of the five articles, where would you expect to see research?
Tip: #2 showed that data was reported but not research in article #4.
What is the best evidence?
When I return to “Results are in: Mississippi Students #1 in the Country for Reading Gains,” I actually have more questions after more reading. Especially after reading this article: “Here’s What All the NAEP coverage missed.”
What if the reading gains are the result of higher beginning points every year?
2.#Research Applied Evenly
What would be worthy of studying?
- Is the gain the result of instruction delivered to the students?
- Is the gain the result of the professional development provided for the teachers since 2013?
- Is the gain the result of the addition of coaches in the lowest buildings (in the fall of 2018)?
- Is the gain the result of the retention policy?
And that takes me back to Paul Thomas’s blog (#5 above). And this updated section:
- UPDATE: Todd Collins has raised another important caveat to the 4th-grade reading gains in Mississippi because the state has the highest 3rd-grade retention percentages in the country:
But Mississippi has taken the concept further than others, with a retention rate higher than any other state. In 2018–19, according to state department of education reports, 8 percent of all Mississippi K–3 students were held back (up from 6.6 percent the prior year). This implies that over the four grades, as many as 32 percent of all Mississippi students are held back; a more reasonable estimate is closer to 20 to 25 percent, allowing for some to be held back twice. (Mississippi’s Department of Education does not report how many students are retained more than once.)
This last concern means that significant numbers of students in states with 3rd-grade retention based on reading achievement and test scores are biologically 5th-graders being held to 4th-grade proficiency levels. Grade retention is not only correlated with many negative outcomes (dropping out, for example), but also likely associated with “false positives” on testing; as well, most states seeing bumps in 4th-grade test scores also show that those gains disappear by middle and high school.
After several questions about “retention” and/or “intervention” and/or “multiple attempts on the state assessment,” maybe this is a focus for research. What data do we have? What data do we need to collect? What other questions bubble up?
- Did students who did not meet the proficiency level have higher absenteeism that proficient students?
- Did any specific classrooms have higher growth than others?
- What do we know about the implementation of the teacher training?
This “study” may require some additional data collection but it could be undertaken relatively quickly to form some general ideas yet this year.
Because I want to reduce the need for intervention, I might also explore this chapter from Regie Routman’s, Read, Write, Lead. (Link) 3.#Best Evidence 4.#Full portfolio of methodology
What I wouldn’t do is:
Give the 4 point credit to ANY of the above areas without study.
Blame teachers for not implementing “enough” or “correctly” without study.
Say that Mississippi has a program that should be replicated in every state because we don’t know the amount of resources that it took to get these results that are not sustained through 8th grade . . . without study. 5. Evidence, not a straw person.
The purpose of this post was to pull together a topic currently in the literacy field, generate some questions, look at the data, and apply the 5 rules from the Research presentation. In less than an hour my questions were generated and this post was written. A beginning application. A beginning look at the Big Picture.
You can do this.
You must do this.
You need to verify the accuracy of what you are reading. Find a partner and get started!
#Evidence, not a strawperson
Today’s post considers Rule 5 from P. David Pearson’s presentation as a part of the #ILA19 panel titled: “What Research Says About Teaching Reading and Why that Still Matters.” (The links for Rules 1-4 are at the bottom!)
Read Rule #5 again.
Does it sound familiar?
The hue and cry that no one is teaching phonics . . . is almost hysterical in light of this report from EDWeek that Reading First (2001-2008) failed to make gains in reading comprehension due to toooooooooooooo (o’s added for emphasis) much focus on skills like phonics. I personally know of school buildings that were spending an hour each day in the primary grades on phonics in the Reading First era. And a few spent more than an hour because of some slick salespersons.
(Link, 2008)
What didn’t work?
Reading First required all 5 pillars from the National Reading Panel
- phonemic awareness
- phonics
- fluency
- vocabulary
- comprehension
with the measure of success being an assessment of comprehension which had the lowest amount of time out of reading instruction. So of course, the data for reading comprehension didn’t improve when phonics and fluency were the most popular and most often tested pillars!
Reading First ended. The Common Core State Standards became the next “great golden goose” and whiplash hit teachers when they were told that “close reading” meant no introduction to the story/book and annotations were now the activity of the day. Cold reads. Master individual standards.
. . .
Phonemic Awareness instruction did not disappear.
Phonics instruction did not disappear.
Fluency instruction did not disappear.
Vocabulary instruction did not disappear.
Comprehension instruction did not disappear.
All five areas were still a part of the CCSS standards. And yes, when writing was finally back in vogue, we did celebrate. Under Reading First, writing was pushed out of the 90 minute uninterrupted reading block. In some instances, writing totally disappeared or appeared briefly as a Monday weekend journaling. One day out of five days for 15-20 minutes was eked out for some assigned writing prompt.
When I hear:
No one is teaching phonics.
Teachers don’t know how to teach phonics.
Teachers weren’t trained to teach reading.
I have to take a deep breath. And sometimes a second breath. And even a third breath.
It isn’t ALL teachers as often reported.
Teachers in Iowa under Reading First were required to have 40 hours of professional development each year in those five pillars. And in our region, we offered similar training for districts that did not qualify for Reading First grants because they also needed the knowledge. It wasn’t withheld from anyone. It was research-based, systematic and explicit.
The very nature of these reports that feel like accusations come from high school teachers (yep, they didn’t have that unless they trained in Reading); teachers who come from alternative licensing (didn’t attend a traditional college licensing program); curriculum/marketing personnel and journalists. No pro/con reporting of more than one side of an issue. Instead, reporting on the state of the nation on the basis of a few states. Sometimes even stating their biases although they have never, ever taught a student to read.
Raising your hackles?
But?
BUT?
BUT?
Me, too!
Just last week, I had one of those parent calls. A parent whose child had an IEP meeting. The child is currently a junior in high school. They (IEP team) wanted to write a goal for phonemic awareness. (I had to literally cover my mouth on the phone to make sure that I was listening and not expressing my opinion.) Phonemic awareness – sound manipulation – no print included for a 17 year old student with three semesters left in the public school system.
Phonemic awareness – which by National Reading Panel research was to take 20 hours of instruction and be done in kindergarten.
Phonemic awareness – because of a data point that wasn’t “mastered.”
“What do you want and need for your child?”
Driver’s Ed. so she can get to and from a job.
Ability to get a job.
Ability to keep a job.
To keep her kind, helpful “I will try anything” attitude.
To continue to grow and learn to be a successful adult in the community.
Math so she can figure out a budget, pay rent, expenses, and be as independent as she wants.
Approximately 270 days of school left for this child. A program of study to complete. A student who has had phonics as a part of her reading goal for 10 years. LETRS trained teachers. Phonics program after program.
EVERY
SINGLE
YEAR
OF
SCHOOL
PHONICS:
-
- kindergarten
- first grade
- second grade
- third grade
- fourth grade
- fifth grade
- sixth grade
- seventh grade
- eighth grade
- ninth grade
- sophomore
- first half of junior year
and now some folks who have never worked with the child and could not pick her out of a classroom believe she should have a goal in phonemic awareness because of a data point.
After 11.5 years of phonics instruction, maybe it’s not the child.
Maybe it’s the crappy timed test and she just doesn’t do well under pressure.
Maybe the nonsense words really offend her sense of meaning.
And it makes me incredibly frustrated.
Don’t tell me the students haven’t had phonics!
Prove it!
Where is your evidence?
(Disclaimer: I understand the frustration of not having student needs met. As a special ed teacher I have taught dyslexic students. So many students needed different approaches and methodology. I have used an array of tools and programs. One example: After parental requests and with the permission of my administrator, I tried “Hooked on Phonics.”)
The current Reading War is based on a bunch of untruths, misrepresentations, and straw man arguments.
Spend the time to check your facts!
If you have not been following along, here are the posts to date:
#Full portfolio of methods
When I return to my cooking thoughts from yesterday, I have to think of methodology and resources. Will I use “glass microwaveable” dishes in the microwave? A double boiler on the stove?
And what about the fudge? Do I really “butter” the pan? Not that nasty cooking spray either! Can I just use parchment paper to line the pan? (Shudder as I think of butter/oleo visible on the 9 x 13 glass casserole plan! Total ICK!)
When do I follow the directions to the letter vs. letting previous experience guide my planning?
Today’s post is considering Rule 4 from P. David Pearson’s presentation as a part of an #ILA19 panel titled: “What Research Says About Teaching Reading and Why that Still Matters.”
So who are the “cousins”? These are some possibilities from the table in “10 Things Every Literacy Educator Should Know About Research.” (Link)
(Note. The information in this table was drawn in part from “Standards for Reporting on Empirical Social Science Research in AERA Publications: American Educational Research Association,” by P.A. Moss, J.W. Pellegrino, B.L. Schneider, R.P. Duran, M.A. Eisenhart, F.D. Erickson, et al., 2006, Educational Researcher, 35(6), 33–40; “Qualitative Analysis on Stage: Making the Research Process More Public,” by V.A. Anfara, Jr., K.M. Brown, and T.L. Mangione, 2002, Educational Researcher, 31(7), 28–38; Literacy Research Methodologies, by N.K. Duke and M.H. Mallette (Eds.), 2004, New York: Guilford; Literacy Research Methodologies (2nd ed.), by N.K. Duke and M.H. Mallette (Eds.), 2011, New York: Guilford; and Educational Research: An Introduction (8th ed.), by M.D. Gall, J.P. Gall, and W.R. Borg, 2007, Boston: Allyn & Bacon.)
The methodology is not set in concrete, but it has to make sense and follow general research principles. All of these involve “science.” ALL. of. these. involve. “science.”
Some seem to over emphasize RCTs – Randomized Controlled Trials. We saw that in the “gold standard” in Reading First. And meta analyses were NEVER allowed but some RCTs just are NOT possible in education. Controlling for every thing in the environment is tough even when two classrooms sit side by side. Equally difficult is the history of single-subject experimental designs. At one point, single-subject experimental designs were the most favored and at other times they were not indicative of “authentic” treatments in classrooms so they were used more infrequently.
Narrowing the field to only one methodology is, in my mind, similar to giving someone a math problem and saying that you can only use addition to solve it. No other process. Just one.
Not helpful. Not logical. Totally restrictive for no real reason.
More productive thinking about the math problem could be multiple routes to solutions with the use of several processes. The solutions could be studied for efficiency or effectiveness . . . or “innovative” status.
What doesn’t count?
Relying on “The Google”
Relying on “Op-Ed” Pieces
Or 
Do the Work.
Stop.
Think.
Does this make sense?
What do you need to add to your repertoire to have a “full portfolio of methods?”
Where will you begin?
When will previous experience guide methodology?
If you have not been following along, here are the posts to date:
#Research Applied Evenly
#Headlines dealt with “Rule #1.” (Link) P. David Pearson at #ILA19 was a panel member for a Saturday 7 a.m. session titled: “What Research Says About Teaching Reading and Why that Still Matters.” Dr. Pearson proposed several rules for our work and I have been considering this second rule for the last few weeks as I have read across Twitter, blogs, emails, newspapers and journal articles.
Rule 1: Policymakers have to read beyond the headlines.
Rule 2 is captured here.
Let’s return to
Results are in: Mississippi students No. 1 in the country for reading gains (Link)
What research is reported?
All the research?
Dictionary.com defines research as:
Go check out the article and identify the “research” you find.
. . .
Hmm
. . .
Hmm
. . .
Data
Data
Data
Reporting of “findings” or “results”
Hinted at in this section:
The Mississippi Department of Education attributed the some of the continued success in reading scores to the Literacy-Based Promotion Act, a law that went into effect in 2013 that requires third-graders to pass a reading test before they can be promoted to the fourth grade. This year marks the first where students had to hit a higher bar to move up a grade.
“Mississippi has entered a new era of public education,” said Jason Dean, chair of the Mississippi State Board of Education. “Our significant improvements in teaching and learning have made Mississippi a national leader for improving student success in education.”
The linked article about “a higher bar” took me to this article with this quote in the final paragraph.
Woods is one of dozens of literacy coaches working in classrooms across the state as thousands of third-grade students prepare for their final chance at passing a critical reading exam. Should they fail, the possibility looms heavy that they’ll have to repeat the grade. (Source Link)
Facts
- Picture dated 6/17/19 labeled with coach and students
- Dozens of literacy coaches
- Thousands of third-grade students
- PREPARE
- Final chance at passing a critical reading exam
- possibility to repeat a grade
Questions/ Wonderings
- Many schools in MS, begin in the second week of August. Was this a summer school program extending the year?
- Final chance: How many opportunities had students already had for this test? Beginning when? How frequently could students retake for another chance?
- What is this test?
- Is this test aligned with NAEP?
- What is the technical adequacy of this test?
- Have third grade teachers in MS seen the test questions?
- Does test prep occur during the school year in the third grade classrooms?
- How many third grade students had to repeat a grade?
- What are the “significant improvements in teaching and learning”? (Jason Dean)
- …. (Add your own)
Additional information gathering – Mississippi Dept. of Ed. 2013 Literacy-Based Promotion Act (link)
- Train K-3 teachers, curriculum specialists and other educators
- Research-based instructional strategies
- 2014-15 retention for lowest students unless “good cause exemption”
- Law modified in 2016
- Fall of 2018 literacy coaches were deployed
Additional “digging” on the site, training was in LETRS (subject of IES study link)
Result from RTC study: What did the study authors report about the efficacy of LETRS?
Providing second-grade teachers training based on the LETRS curriculum (with or without the instructional coaches) increased their knowledge of reading instruction techniques and their use of explicit instruction. However, it did not increase the reading test scores of their students {emphasis added}.
The authors estimated effect sizes on reading scores that ranged from 0.03 to 0.08. These estimates were not statistically significant.
Questions about the training:
- How many days of training did the Mississippi teachers have?
- What was the implementation plan for reviewing the instruction in the classrooms?
- What percentage of teachers implemented their training as presented?
- What percentage of teachers were observed for fidelity of implementation?
- What percentage of teachers studied their own implementation of the instructional changes?
- Mathematically, what was the benefit to students in terms of Cost of Teacher Training (K-3) x # of Training Days (cumulative # for all years) / Number of Students (counted only once)?
What do I now know?
So some facts were reported in the initial article.
Some generalizations about student performance were made.
No research was reported. In fact a prominent journalist claims “There’s no way to know for sure what causes increases in test scores.“
I added in research from a What Works Clearinghouse report on the effectiveness of LETRS.
How can causation or correlation be implied for this “growth of 4 points”?
And what is the significance of “outlier growth” in typical research?
Studying the “growth” for 4th grade students in MS would be an appropriate action from a group who advocates for “science”.
Rule 2 for RESEARCH was not applied in the original article. You can “judge” whether any research is applied in additional articles on the same topic.
CHALLENGE:
Choose an article, any article, that supports “Science of Reading”. Identify the precise research in the article. Study it. What do you really find?
Mississippi 2019 NAEP Results Examined – Link
#ILA19: Research
Third time’s a charm! It was so helpful to dig into additional chapters from this book.
I wrote briefly about the #NCTE18 session here and assessment and vocabulary as well as #ILA18 here about Chapter 16 Comprehensive Literacy Instruction and 8 essential components.
Assessment: Peter Afflerbach Handout
So much to think about from this outline. Some key takeaways to discuss: What do you know about your assessments? What do they claim to measure? How well does the assessment align with your “needs”? What are the challenges?
How do we get quality, informed research in the hands of teachers and administrators around the world?
- Know the source. What Works Clearinghouse
- Know the researchers and their reputations and experience as researchers and practitioners. Reading Hall of Fame is one trusted source.
- Know the goals of research. Nell Duke and “10 Things Every Literacy Educator Should Know about Research”
- Attend the #ILA19 Research session with P. David Pearson and Nell Duke at 7 AM on a Saturday morning in New Orleans!
Smarter Balanced Item Quality Review
Questions continue to be voiced in the media and academic realms about the assessments that will determine whether students are “proficient” on the new Common Core Standards.
Do you recognize the experts listed below?
Do you trust their input?
“Item Quality Review Panel convened on May 20–21—The Item Quality Review Panel convened in Las Vegas to discuss three critical aspects of the item development process: quality criteria, item specifications, and archetypes. The panel members (listed below) who represent content expertise and expertise in services to underrepresented students met as a whole group to discuss the design of the Smarter Balanced assessment and the goals of the Field Test item development. Then, in content-specific groups, the panel members, Smarter Balanced staff and work group representatives, and contractor staff discussed key areas of focus and made recommendations to improve item development.
Dr. P. David Pearson
Dr. Donald Deshler
Dr. Douglas Hartman
Edward Bosso
Dr. Elfrieda Hiebert
Dr. Guadalupe Valdes
Dr. Sandra Murphy
Dr. Alan Schoenfeld
Dr. Bill McCallum
Estelle Woodbury
Dr. Francis (Skip) Fennell
Dr. Guillermo Solano-Flores
Dr. Jason Zimba
Dr. Karen Fuson
Dr. Patrick Callahan
Steve Leinwand”
This information was released in the Smarter Balanced Weekly Update #118, 2013-06-07.
Resource - Full 