Orchestrating Writing Assessments

How do you know that your students are effective communicators?

Do you measure communication?  Do you use writing assessments for that purpose?  If so, what are those writing assessments?  How do you know that your students have made growth in writing?

Those questions and their answers have been responsible for district-wide writing assessments for over ten years in a local district.  Currently, narrative writing at third, and persuasive letters at eighth and tenth grade are assessed with a Six Traits rubric.

The work in this district = 900+ student papers that are all read by at least two scorers:  teachers, administrators, university students, community members, retired teachers, and AEA staff.  Over three days, approximately 100 scorers (30-35 each day) are greeted by the superintendent of schools for a welcome that includes history, data and purpose for the assessment.  Professional development includes increasing knowledge of effective writing instruction, the writing process and the Six Traits before the group begins to look at the rubric and anchor papers.  Each day the scorers must calibrate because that unique group has never been convened before.  “What qualities of the rubric did the NWREL scorers see?” dominates the conversations.  Confidence in the use of the rubric and identifying the traits increases with practice and even a “happy dance” may occur as participants match the NWREL anchor scores.  And then (drum roll, please) the scoring begins . . .

The goal:  adjacent scores.  What does that mean? If Joe scores a trait a 3 and Suzie scores a 4, that is adjacent.  If that “adjacency” has occurred for all six traits, the scoring for that paper is over.  But if Joe scores a 3 and Suzie scores a 5, the paper will be reread by a third reader for that trait (or traits).  If the third reader does not agree exactly with Joe’s 3 or Suzie’s 5, and believes that trait is a 4, the three readers will conference with the paper and the rubric and discuss their thinking.  Imagine, teachers and others, spending time talking about student writing because students are counting on feedback about their writing!

Wow!  Annual scoring of student writing at three grades.  Sound easy?

The support staff prep and post work for scoring writing from three grades of students is phenomenal. The “behind the scenes” orchestration involves year round work!  Winter scoring “work” begins in September when the packets with prompts, draft writing paper, and final copy writing paper are assembled for each classroom at grades 3, 8, and 10. Maintaining the anonymity of students and teachers involves the use of codes.  Recruiting scorers begins.  Later, students write and teachers return papers to central office where packets of five papers are assembled with a quick scan of every page of student writing to remove any possible student identification.  These packets are readied for the scorers.  Reminders to scorers about plans in the face of adverse winter weather, ordering food and snacks for scoring days, and packing up all the materials are just a few of the tasks that precede the scoring.

During scoring days, basic work schedules for key support staff members are put on hold.  Checking scorer registration, last minute substitutes, phone calls to absent scorers are just a few of the early morning tasks after the materials for the day are set out.  Checking the details for the next day also encompass some of the morning.  With luck, there is some office time before lunch.  But the entire afternoon is dedicated to routing scoring packets to readers, collecting and matching score sheets, recording final scores, noting 3rd reader or conference needs and meeting the needs of scorers.  Busy, busy days!

But the work is not yet done!  After the scoring days, data entry (six scores for every student paper) becomes the next task.  Scores are compiled for district, building, grade level (dept.) and  teacher totals. Papers are returned to teachers for February parent-teacher conferences.  Notes are made about the work and filed in preparation for the next round.  And then the process of scheduling for the next year begins.

I work with a small portion of this work, co-facilitator for the scoring days.  I am always amazed by the enthusiasm and dedication of the scorers who are now on the main stage.  They are conscientious about “getting the work done right” and are also eager to learn.  Classroom teachers and administrators often find a gem to add to their instructional repertoire. Many first-time scorers are anxious about this responsibility.  Other scorers have literally scored for at least ten years.  It might be easy for them to become blase about their task, but they remain committed to finding a common language to describe the qualities of writing that they see!

writing assessment

Congratulations on a scoring job well done!  
Good luck with continued instruction!

Will there be changes in the future?  Sure!  With implementation of the Iowa Core, assessments will inevitably change.  Will SBAC be used to assess writing?  Will there be a different writing assessment?  A planful decision will be made as more information becomes available!

Are you assessing writing?  Do you have experience with district-wide writing conversations? What is/ was your role? I would love to hear about your experiences!

6 responses

  1. Glad to hear of this work. I am leading several schools through a similar (schoolwide versus districtwide) though not as intense of a process…a process that not only helps teachers learn about their students, but also allows teachers to learn about the standards and their own thinking and teaching styles.

    Thanks for sharing. Your blog validates my work and also offers ideas for how to bring it up a notch!

    1. Dea,
      Thanks for your comments.
      I think it is absolutely imperative that districts explore a variety of options as they search for ways to measure AND increase knowledge of the standards. In this district, teachers do get independent validation of their students’ writing. Students have two days to draft,a revise and develop a final copy. The topic is the same so that it is easier to score. The students also “win” because two other scorers have read their work and independently assigned scores! I really appreciate the anonymity of the scoring! The assessment is as fair as it can possibly be!

  2. Your question is huge. And the task to assess it equally huge. I am awed at the painstaking process involved here. It does make me wonder how our new common core assessments (which will encompass a great amount of writing) will be scored. Thank you for this enlightening post. I had no idea as to the extent of the work involved.

  3. Julieanne,
    I believe, very strongly, that this is the right work and that we can really provide a great deal of information from district-wide on-demands it if we plan for all the possible issues. My very visible three days are a very minor part of the entire piece. Most only see the three face to face days. They don’t see that I analyze student data in order to see where and how I can change my PD focus to better support quality instruction and enhanced student learning!

    I think these are exciting times in education and that we can be proactive and provide quality assessments that will inform our instruction and decisions.

    Thanks for commenting!

  4. […]  I was intrigued by the fact that Nebraska does have a state writing test (I wonder what kind of orchestration is necessary for a state-wide writing test?) and that all five states (including Iowa) have long had […]

  5. […] wrote about this last year in a post titled, “Orchestrating Writing Assessments“.  Check out the link for the details.  It’s an amazing week of […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

My Zorro Circle

it is what it is

Steph Scrap Quilts

"Our lives are connected by a thousand invisible threads..."

TWO WRITING TEACHERS

A meeting place for a world of reflective writers.

Tim's Teaching Thoughts

Ideas and Reflections on Teaching

Hands Down, Speak Out

Listening and Talking Across Literacy and Math

Teachers | Books | Readers

Literacy Leaders Connecting Students and Books

Dr. Carla Michelle Brown * Speaker * author * Educator

We have the perfect words. Write when you need them. www.carlambrown.com

Curriculum Coffee

A Written Shot of Espresso

Mrs. Palmer Ponders

Noticing and celebrating life's moments of any size.

doctorsam7

Seeking Ways to Grow Proficient, Motivated, Lifelong Readers & Writers

Doing The Work That Matters

a journey of growing readers & writers

annedonnelly.wordpress.com/

adventures in multiple tenses

The Blue Heron (Then Sings My Soul)

The oft bemused (or quite simply amused) musings of Krista Marx -- a self-professed HOPE pursuing Pollyanna

Middle English

Life as an English teacher leader

steps in the literacy journey

Walking the Path to Literacy Together

arjeha

Smile! You’re at the best WordPress.com site ever

Resource - Full

Sharing Ideas, Strategies and Tools

%d bloggers like this: