Skip to content

Grading by category

May 8, 2013

I decided to try grading by category for the take-home question on my last lecture exam. In a nutshell grading by category involves the following steps:

  1. Instructor reads ~20% of student solutions and determines different categories of solutions based on what solutions students submit.
  2. Instructor prepares a list of categories with descriptions for TAs and other graders.
  3. Graders read each student solution and assign each solution to a category.
  4. Once all solutions are categorized, instructor determines the point value of each category of solution.
  5. Student work is handed back with a category label on top and a separate sheet that describes each category and the points awarded to solutions in that category.

 

The motivation for grading by categories is that 1) it allows instructors to provide students semi-detailed feedback without writing responses on individual papers, 2) it facilitates grading problems in which students submit solutions which may not be anticipated in a grading rubric, and 3) it allows instructors to change the points awarded to a given solution very quickly and consistently for all students who submitted that solution.

Here is the question and categories for the problem I assigned. The category list was missing about five of those categories and did not include point values when I sent it to my TAs and LAs. Below are some advantages and challenges we experienced in our first implementation of grading by category.

 Advantages

  • The LAs found it much faster and much easier to classify solutions by category rather than to decide how many points to award a given solution.
  • I enjoyed separating “interpreting the solutions” from “scoring the solutions”. I enjoy trying to understand student thinking and student difficulties and it was nice to do this without simultaneously making a value judgement about how many points to award each solution.
  • I did modify some point values after all the categories were created. This was fast and easy and required no additional markings on the papers themselves.
  • Papers were labeled with nothing but category letters with the category descriptions and point values posted online after class. This allowed me to let students find their own papers in a pile during class without violating privacy surrounding grades.
  • Focusing on categories helped me identify the common errors more easily than if I had been awarding points to each question. At the end of grading I know lots of students fell into categories C and D in part c which is much more useful for me than knowing that lots of students had earned six points on part c.

 

Challenges

  • My initial pass through 25 solutions did not produce all of the categories we eventually needed. This meant that a substantial number of papers were first read by a grader then set aside to be read a second time by me.
  • My category descriptions need to do a better job of articulating specific errors. My category descriptions focused on what students did correctly and sometimes graders were not sure how to categorize a solution.
  • TAs and LAs had to wait for me to do my initial reading and category construction which slowed down the overall grading process.
  • I did not provide graders with a model solution which led to some confusion as to what were correct solutions. This challenge would also benefit from articulating errors more directly in the category descriptions.

 

Overall, I was pleased with how grading by categories worked. I think it can be especially helpful for new TAs and LAs who are first starting out as graders. Two things I will change next time:

  • Have all the graders start with a set of papers and create their own categories then come together and consolidate categories and decide on common labels before grading the rest of the solutions. This will help us sample a large collection of papers in a short time to create categories. It’s also probably a worthwhile exercise for LAs and TAs to try to describe categories.
  • Involve the TAs and LAs in deciding how many points to award each category. I made these decisions myself this time because I was in a hurry but I think this would be a very worthwhile conversation to have.
Advertisements

From → Teaching

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: