I’m a literacy consultant who works with seven districts.
How do I know if I’m being effective?
Doing a good job?
Doing what really works?
I have to start with the original . . . Clint Eastwood . . . same birth year as my dad who always kept me grounded!
A Short Story
I’ve been traveling a lot over the last two weeks. Over three thousand miles in a trip to Kentucky for an adorable grandson’s second birthday, then on to Florida with Mom and an aunt and uncle who is one of my mom’s younger brothers for a nephew’s high school graduation, and then back to Kentucky for some more time with the kids.
Was the trip successful?
Four possible data points might be these:
- The number of miles driven successfully. That is important because it was my first out of state road trip with my new car and then many miles driving a Ford 150 which is about three times the size of my car.
What might constitute a success? No flashing red or blue lights and no major problems. The number of palindromes I noticed on my odometer and particularly the one as I traversed the Missouri River bridge in St. Louis.
What data would not point to a success? Uncle Leo might say it was the number of times I drove over a curb.
- The number of times my GPS and Aunt Shirley’s google maps agreed. Less successful might be our decisions about which to follow when there was a disagreement.
Success? Google maps was definitely more up to date than GPS.
Not a Success? The “shortest” trip was NOT always the ideal route to take.
- The number of card games played.
Success? The variety from hand and foot to pepper.
Not a Success? The number of 9’s and 10’s I had in EVERY pepper hand!
- The variety of experiences and places we went.
Success? Wading in the Atlantic, time with so many precious relatives, driving to the top of Lookout Mountain in Georgia, the flea market, a little homemade wine, the food, the movies, and stories after stories.
Not a Success? Not driving back down Lookout Mountain (remember, not my vehicle!).
Do you notice a possible pattern?
Each data point seems to have more than one side!
If you had to sort these data points, could you find some summative as well as formative measures?
So back to the beginning . . .
I’m a literacy consultant who works with seven districts. How do I know if I’m being effective? Doing a good job? Doing what really works?
We collect a lot of data. We spend a lot of time with data. We spend a lot of time talking about data. But do we EVER really address these questions? Or does each question have multiple data points similar to those listed above. This post is the result of many miles of driving and a push from Elizabeth Moore at Two Writing Teachers when she wrote this post last week, “Literacy Coaches: How do you assess your impact?” Beth talks about using goals, student-centered data, survey data and quantitative data in her post.
I have a ton of quantitative data to share. At our agency we have had team Wildly Important Goals (WIGS) for two years focusing on our K-3 readers and using screener data to determine the effectiveness of our goals. I like to use them also as a beginning point when I reflect on my own effectiveness although they are only a small portion of my K-12 job.
PART ONE:
Here’s my data for four different types of my work in buildings by each month.
- PDC = Professional Development in Core Literacy Instruction K-3
- OCC = Observation/Coaching in Core Literacy Instruction Implementation K-3
- PDI = Professional Development in Research-Based Interventions K-3
- OCI = Observation/Coaching in Research-Based Intervention Implementation K-3
The green boxes show that I met my goals which are also outlined below:
Good:
- I met all four of my goals in December and in February.
- I met my monthly goals 21 times.
- I met my Observation/Coaching Intervention goal in December (after 5 months).
- I met my PD Core and Observation/Coaching goals in January (after 6 months).
- I met my total goal in January (after 6 months).
And to make me feel better . . .
- My annual total for PDI was 94% so it was close.
- Average percentage of goals met is 96.8%.
- Total number of interactions was well above the annual goal just in a different distribution. (146% above the goal)
Bad:
I missed my monthly goal 19 times. (19/40)
I met either one or zero monthly goals in August, March, April, and May. (4 months/10)
There were zeros in four categories across the 10 months. (4/40)
i did not meet my PDI annual goal. (141/150)
Ugly: The hard reality of the data
August was not required for data collection but because it was almost a full month of work I decided to include the data.
I can offer excuses for the spring – horrific sudden death of my nephew and his wife in March and then my brother at the end of April, but the fact is that I only missed one PD session during either of those times – so excuses don’t change the data.
And if you would like to see the data in a larger format – Data Here
PART TWO – How did students do on the screener administered in the fall, winter, and spring?
Data is reported in terms of green boxes for buildings by grade levels if 80% of the students or more met the benchmarks set by the state. (Red if below 60% or fewer of the students met the benchmark criteria.) Districts can choose from several approved screeners but the state of Iowa only pays for one.
Good:
- The total number of grades meeting benchmark by 80% or more by building increased from 7 in fall to 8 in winter with changing criteria.
- The number of grades meeting benchmark criteria by 80% or more (green) building increased for kindergarten from 2 in fall to 4 in winter.
- The number of first and third grades remained the same from fall to winter (3- first, 1-third).
- The number of grades below 60% benchmark criteria decreased from 8 in fall to 3 in winter.
- The number of grades below 60% benchmark criteria decreased from 8 in fall to 4 in the spring.
Bad:
Grades 1 and 3 did not have any buildings meeting 80% benchmark criteria in the spring and kindergarten and second had 2 and 1 respectively.
The spring green (80% benchmark criteria) was the lowest of the three reporting periods.
Ugly:
The 8 grade levels by building meeting 80% benchmark criteria in the winter dropped to 3 for the spring.
The 3 grade levels by building below 60% benchmark criteria at winter increased to 4 in the spring.
What questions arise?
How does this data compare to state-wide Iowa totals?
Which specific buildings have multiple levels of green? or red?
What is working? What is not working?
Is more practice needed across the day (distributed practice)?
Are discrete skills transferring to reading passages?
What about fidelity of implementation? What does that data reveal?
Did we over rely on our winter successes that did NOT appear to transfer to spring benchmarks?
Brave = sharing this data publicly.
It’s not all roses and sunshine. What works in one building doesn’t necessarily transfer to what works in another building.
But, stop!
Is all data equal?
- How many students made growth?
- How many students made significant growth?
- How many teachers changed instruction based on the data?
- How many teachers changed interventions based on the data?
- What if the summative data (Iowa Assessments) shows a different picture of these same students?
- How many students have reading goals for the summer?
- How many students love reading?
- How many students read at school by choice?
- How many students read at home by choice?
- How many students can name their favorite books?
- How many students can name their favorite authors?
- How many students can name their favorite illustrators?
- And how do the students REALLY feel about school?
What data is missing from this snapshot?
Another short story
I am in total grandmother heaven. He meets me at the door, takes my hand, leads me into the living room, and tells me what to do/play/where to sit. “Gramma play.” “Gramma here.” “Gramma ice cream.” Gramma choo choo.” “Gramma dinosaur train.” I can’t even begin to count the number of times that I heard, “Where Gramma go?” during the last two weeks. I count that as a success. To disappear into another room and to be missed makes my heart melt!
Those are all data points that convince me that I’m doing a GREAT job as a grandma. Are they numbers? Are there specific criteria or cut points?
What data points match your school values and core instructional principles? When do you need to make sure that you are triangulating data and not over relying on any one source?
If I had only shown you fall and spring student screener data, you would not have seen the growth that doesn’t seem to have been sustained. That’s why my #OLW “BRAVE” is a part of this post. This is our third year with this process. Because the cut points for benchmarks change annually, we can’t compare each grade level year after year but we can look at trend data to see whether grade levels of students continue to grow as the move up through the grades.
How are you reflecting on successes? The good? The bad? The ugly?
AND who are you reflecting with?
Resource - Full 