Brilliant or Insane?: assessment

Thursday, April 24, 2014

7 Characteristics of Quality Summative Assessments

By Reed Gillespie

via wikimedia.com

Recently, much of the talk in educational circles has been on formative assessment--and with good reason--but in the process we've lost focus on creating effective summative assessments.

Today the term "summative assessments" conjures up images of standardized tests, benchmarks, data collection, and exams. In many ways, these have been over-emphasized to the point that we've missed the true value of summative assessments. I need look no further than many of my own tests, which over-emphasized multiple choice questions in an attempt to mimic our state tests, as an example of what summative assessments should not be.

We must move away from summative assessments being synonymous with district, state and national tests. Summative assessments should play an integral part in our educational system.

And how about what summative assessments should be? These seven characteristics comprise a quality summative assessment:

Quality summative assessments should be pre-defined by a group of teachers and involve the students. So as tempting and time-saving as it is, we shouldn't sue pre-crafted rubrics or textbook created tests.
Assessments must provide students with the opportunities to demonstrate what they know.
Challenge students to transfer what they've learned to specific and real-life tasks.
All assessments should correspond with important learning targets and essential questions so they yield true information about student progress.
Summative assessments must provide meaningful feedback to both the student and teacher. By doing so, assessments help increase student motivation to learn. Nonjudgmental feedback motivates students for further effort, while using grades as rewards or punishments decreases student motivation.
Incorporate 21st century skills and content into assessments.
Summative assessments should not be limited to high-stakes tests.

The purpose of any summative assessment is to provide meaningful reports on students’ achievement, and effective summative assessments help students achieve to their highest abilities.

Reed Gillespie writes about education for Brilliant or Insane. Learn more about Reed on our Team page. Follow Reed on Twitter here.

Tuesday, April 22, 2014

Is There Really a Difference Between an A Student and C Student?

By Mark Barnes

via flickr.com

When considering letter grades, most people believe there is a big difference between an A student and a C student. While these marks may suggest that students are excellent and average, respectively, I would argue that, in terms of achievement, the two students could be remarkably similar. Moreover, they might be reversed in their abilities – the C student being excellent and the A student being average. In reality, both measures are entirely subjective.

How is it possible, you may wonder, for grades to be subjective, if they are based on an accumulation of points and percentages? Consider this logic.

Most traditional grading systems work like this:

The teacher creates an activity. The teacher assigns an arbitrary point value to the activity (10, 20, 100 or even 1,000); these values are typically a derivative of the overall points a student can amass in a grading period, also prescribed by the teacher.
The teacher decides how points are acquired on particular tasks – a rubric, points per item within the activity, or gut feeling.
Based on step 3, the activity receives a final score.
The final score is translated to a percentage and a corresponding letter grade.

This seems like a fair, scientific approach to assessment, right? Actually, there are far too many variables in this formula to truly assess learning, if such a thing can ever really be accomplished.

Consider number 1. Was the student given the opportunity to select how she might demonstrate learning? There's plenty of research that suggests that people want to do things when they’re given choices.

Point values may seem fair – a student answers 8 our of 10 math problems, she scores 8/10 for a B. (Of course, in some systems, 80% is a B- or even a C+, another issue with grades.) The teacher chose to measure learning with 10 problems. The student missed two that are most likely quite similar to the other eight? Could it be that after 40 minutes of concentration, the student lost focus on the task and made two careless mistakes? Mental fatigue should never play a role in how we assess learning, yet teachers routinely create a state of fatigue with lengthy summative assessments that often take 45-90 minutes for students to complete.

When contemplating this math example, the subjectivity involved in grading activities in the social sciences is even more salient. No matter how specific a rubric may be, much of grading written work comes down to a teacher’s preconceived notions of the student and/or comparisons to other model papers, which, of course, the teacher selects as the “best” work samples within a group.

Any letter grade, A or C, based in full or in part on these elements is a poor assessment of a student’s abilities and one more in a long line of reasons that we need to eliminate traditional grades from school.

Mark Barnes is an education author and the publisher of Brilliant or Insane. Learn more about Mark's work here. Follow him on Twitter here.

Monday, April 21, 2014

The Case for The Case Against Zeros

By Reed Gillespie

via vancouversun.com

Ten years ago, Douglas Reeves made a compelling case against zeros. Since then, many districts and schools have crafted No Zero policies (some admittedly miss the intended target), but these tend to be the exception. We cannot continue to use grades as punishments, to send "messages," and to "teach students a lesson."

Six Reasons for the Case Against Zeros

If it's worth assigning, it's worth requiring students to do it.
Work completion is often influenced by home life, learned behaviors, economic standing, etc. It's not fair to punish students for factors beyond their control.
Punishing students for failing to complete an assignment doesn't motivate them. In my experience, low grades are more likely to discourage students from making greater efforts.
Often a handful of zeros doom the student for the entire term, causing students to simply quit.
The students we most worry about losing (those who are often deemed lazy or are below grade-level are labeled at-risk) are most harmed by zeros.
Zeros distort final grades, which should be an indicator of mastery.

Critics of No Zero policies will claim that the penalty--a zero--is appropriate to instill proper values within students. This may be true for the highly motivated, mature student, but it's more likely that these students already possess the intrinsic motivation to be successful in school.

It's time for all educators to adopt a No Zero policy.

Reed Gillespie writes about education for Brilliant or Insane. Learn more about Reed on our Team page. Follow Reed on Twitter here.

Tuesday, April 15, 2014

Infographic Shows 7 Things to Remember About Feedback

By Mark Barnes

I've written widely about feedback for learning. In fact, I believe that narrative feedback should completely replace traditional number and letter grades.

This infographic contains 7 key takeaways from an issue of Educational Leadership, dedicated to feedback.

Each "thing to remember about feedback" in the graphic is from an expert on assessment. What are your thoughts about feedback, in general, and these takeaways, in particular?

Mark Barnes is an education author and consultant and the publisher of Brilliant or Insane. Learn more about Mark on our Team page. Follow him on Twitter here.

Thursday, February 6, 2014

The three-step rubric

By Reed Gillespie

Effective rubrics provide clear criteria and an equally clear means of measuring performance. My best rubrics took hours to construct, yet student projects always seemed to fall short of what I envisioned. The Three Step Rubric changed that.

The right assessment tool stimulates conversation
via: learning first.org

Together, the three steps ensure quality work and feedback. First, students create the rubric. Secondly, each student self-assesses, and finally, the teacher assesses the assignment. A peer-editing step can also be added.

Creating the Rubric

As a class or in student groups, use the following questions to guide the creation of the rubric:

What should the assignment include?
What will be assessed?
How will the assignment align with standards?
How will we measure your (student) achievement?

Admittedly, I’m a big fan of using a 4-point rubric (Exceeds, Proficient, Approaching, Needs Improvement). Because of student choice, sometimes a project doesn’t fit into the confines of the rubric. When this occurs, spend time collaborating and discussing with the student how he/she can create an individual rubric.

Student Self-Assessments

After completing the assessment, students use the rubric to evaluate themselves. They must justify and explain their ratings for each category. You’ll find the students to be very accurate with their own evaluations—often even grading themselves more harshly than you would.

Teacher Assessment

After they’ve completed their self-evaluation, it’s the teacher’s turn to provide feedback. Instead of making comments on all parts of the assessment and rubric, only comment where differences between the student grade and your grade exist (this is a great time saver!) This process enables teachers to more intently focus their time on areas where disagreement exists and where feedback is most important.

Advantages

Setting their own goals helps students get started. It helps them organize and the focus on the task. Students visualize. This is what the assignment is supposed to look like.
Students focus on important components and standards.
They can explain and see why the information or skill is relevant.
By allowing student choice, motivation increases.
Rubrics can be personalized, increasing creativity.
Students are able to accurately evaluate the quality of their work, which enables them to discover ways to improve their performance.
Students and teacher work side-by-side, making the feedback more meaningful.

The Three Step Rubric actively engages students in their own learning. By monitoring and reflecting on their work, they assume more responsibility for their learning.

Reed Gillespie writes about education for Brilliant or Insane. Learn more about Reed on our Team page. Follow Reed on Twitter here.

Tuesday, November 19, 2013

Rubrics fail students

While discussing narrative feedback using my SE2R system at a Learning & the Brain conference, the subject of rubrics came up. I shared some of my feelings about rubrics, which are outlined in this blog post, which originated at the results-only learning blog and garnered lots of views and commentary there.

Rubrics are popular assessment tools for many teachers. Like grades and other traditional assessment methods, rubrics fail students.

The problem with rubrics, like letter and number grades, is that they are a one-size-fits-all form of assessment and they are subjective. Rubrics use words like, "seems," "little," "mostly," "adequate," "some" and "weak," to name a few.

Although narrative feedback, the method of assessment that should eliminate grades and rubrics, may use subjective words in some cases, the feedback is far more specific and individualized. If the feedback says, "Very well done overall," this will be followed up with a careful explanation of what was done well. Rubrics don't do this, so even students who meet the learning outcomes don't know how they did it, based on a rubric.

Consequently, learning is lost. For example, on a major project, a part of a rubric may say, "Work displays adequate attention to detail." This statement has virtually no meaning. I'm not sure what "displays" means in this context, and "adequate" is a completely inadequate word in virtually every case. Finally, vague statements like this do not explain what detail the teacher wants. If providing meaningful narrative feedback, I would mention the detail specifically. If it's a science project on hurricanes, I might say or write something like:

"Your project only partially demonstrates that you understand hot towers, based on the lessons and models from class. You need to add two or more captions to your drawing, explaining how the towers power the hurricane. Please make the necessary changes and resubmit."

Not only does this feedback specify what is missing, it asks the student to return to the work, make changes and resubmit. Because a rubric is a one-size-fits-all tool, it does not meet the needs of each individual and, consequently, fails students.

Can you justify your use of rubrics?

Saturday, July 7, 2012

Use the app, Evernote, for formative assessment

The web site and mobile app, Evernote, is a powerful collaboration, content management and formative assessment tool. Check out how one school uses Evernote in a 1:1 computer environment.

Evernote offers the sort of technology integration that helps create a Results Only Learning Environment.

You can learn more about Evernote here.

Don't miss Mark's book ROLE Reversal: How Results Only Learning Will Change Education as We Know It, due in early 2013 by world education leader, ASCD

Thursday, April 12, 2012

Could you stop grading for one unit?

What if you stopped grading, just for one unit of study? Don't put a number, percentage or letter on an activity, project, quiz or test.

How would you evaluate your students? Is it even possible?

What would replace the grading?

Would you speak to your students more? Could they evaluate each other? Themselves? One unit.

Three or four weeks.

Change what you do.

What's holding you back?

Could it be a fear of learning that you never needed the grades in the first place?

Friday, March 2, 2012

Results-only math lesson

Found this very cool video over at Joanne Jacobs' blog. This is an excellent example of results-only learning in a math class (minus the test she mentions at the beginning, of course). I would recommend that Leah follow up with a collaborative activity to review and process the warm-up or even a second pass at the same activity to ensure mastery.

Overall, this is a fine example of how to get students engaged in math without boring them with worksheets.

Saturday, January 14, 2012

Feedback about feedback

As our second grading period draws to a close, I felt it was a prudent idea to remind students how we end the quarter. Even though we discuss reflection, evaluation and feedback often, it's never too much, I think.

A week prior to sitting down for our one-on-one discussions, when we'll reflect on the quarter and decide on a report card grade, I reminded students that they needed to begin the self-evaluation process.

"Remember, the major work we've done for the past nine weeks," I reminded them, heading into a three-day, holiday weekend. Review the feedback you've received and ask yourself how you will answer the question, "What grade should go on the report card?"

This quick reminder of how we do things in a results-only classroom will, I believe, go a long way toward making the evaluation conferences run more smoothly than if I had said nothing at all.

I meet with each student next week. Stay tuned for a report on how it turns out.

Monday, December 19, 2011

Project based learning prepares students for life

While preparing a presentation for the ASCD Annual Conference and Exhibit Show, I came across this remarkable video, produced by Edutopia.org on project-based learning.

What a marvelous example of how project-based learning and assessment can prepare students for life, while helping them perform well on standardized tests -- a bonus that will keep administrators happy. The powerful quotes from great minds, like Howard Gardner, enhance a very insightful production.

Friday, December 16, 2011

What vision do grades provide?

I found this cartoon interesting. It got me wondering how any student might answer a question about vision, based on grades.

Saturday, December 3, 2011

How results-only learning transforms schools

This video, posted to Edutopia.org, is a remarkable example of how ROLE strategies can transform any school into a remarkable learning community.

Although the people at Cochrane Collegiate Academy, featured in the video, don't call their school a ROLE, watch and listen for the results-only concepts shared by staff and students: collaboration, rapport-building, caring, hands-on, engaged and empowered students.

What you don't hear is anything about homework and grades.

All I wonder is with a model like Cochrane, why aren't more school administrators taking notice?

Saturday, October 22, 2011

Feedback and mastery learning

This is a guest post by Justin Vail. A junior high social studies teacher in Indiana, where fellow guest blogger, Joey Till, works, Vail, like his colleague, has committed to results-only learning this year.

How Self-evaluation Drives Students to Mastery Learning

All of my activities are either simple completion assignments or projects. The completion assignments are essentially deconstructed parts of the project.

When I grade a completion activity I sit down with the student and we scan the various parts of the document. If the basic requirements are met, the student moves on to the next activity and I put a 10/10 in the grade book. If the basic requirements are not met, we identify the problem areas and the student makes a second attempt---this is repeated until the student meets the basic requirements.

When I grade the project (most recently a Regional Geography Video Podcast over the major physical features of 8 different regions), I sit down with each student individually for a more in-depth evaluation of their work. We listen/watch the podcast together. Afterwards, I ask the student some or all of the following questions:

What do you think is the best part of your final product?
What parts of the building process will you do again (reviewing how they worked)? What worked?
What is the biggest weakness of the product?
What parts of the building process do need to change next time?
What did you learn from this whole process?
If you were to grade yourself from A+ to F, what grade did you earn?
How can I change the project to make it better?

The self-evaluation process is difficult for some kids. I sometime have to prod the students to express themselves, but eventually they are very honest about their work and their work ethic. In fact, students are often harder on themselves, than I would be. I have evaluated nearly 200 projects this year, and less than 5 times have student's suggested a "grade" higher than what I thought.

Monday, August 22, 2011

Rethinking tests and quizzes

"Today, we'll have a pop quiz." Back in my days as a traditional teacher, I loved this sentence. There was a sense of power and control, when I surprised my students with some sort of test of knowledge over something we'd recently studied or of a chapter in some boring book I had demanded that they read.

By its most basic definition, the pop quiz is nothing more than a stick, used to punish students. It's not much better than the paddle, teachers used back in the days of corporal punishment. Students are asked to complete some basic task, and then they are tested on the knowledge they retained in a short period of time. If they have not completed the task, they fail the quiz.

Tests are not much different. Students study information the teacher provides, and they are given a list of questions created by the teacher, based on the information that was delivered. The teacher has all of the control -- one reason that tests masquerade as objective assessments, when they are, in fact, very subjective.

If we are to reform our failing education system, it's time to rethink tests and quizzes. To simplify things, the first step should be to eliminate quizzes entirely, as they serve no educational purpose. As Jerome Bruner wrote, teachers should help students "experience success and failure not as a reward and punishment, but as information (The Act of Discovery, 1961)."

So, let's concentrate on tests. Although project-based assessment is the best way to evaluate learning, tests can provide the information, of which Bruner writes.

The key to making good use of a test is to use it strictly as a diagnostic tool. In fact, the word "test" should be replaced with diagnostic. Students should be taught that a diagnostic tool, especially one that contains multiple choice items, is only part of assessment.

Because tests are unreliable and subjective, the results should never be part of a grade, and they should be used to evaluate which learning outcomes need more attention from the teacher. Rather than looking at test results in terms of individual students, they should be considered holistically. So, if 40 percent of a class misses number 8 on a diagnostic, there is either a problem with the item, or the learning outcome needs to be revisited.

When tests are used in this fashion, strictly as diagnostic tools, they become one useful part of evaluation of learning.

Look for more on this subject at this post on raising test scores.