Shared publicly  - 
 
I'm planning the interface for the rubric system, part of the formative feedback system at FlippedTextbook.Com. If you're interested in such things, please look at http://flippedtextbook.com/forum-topic/rubric-system-user-tasks - I could use some feedback on the interface.
5
3
Justin Scoggin's profile photoCindy Sifonis's profile photoKieran Mathieson's profile photoSanford Arbogast's profile photo
7 comments
 
would there be a single rubric for all authors to use or would each author need to create their own? if everyone had access to the rubric then the grading process would be similar for all classes allowing students to be familiar with expectations.
 
I like the single rubric and common rubric concept. I know that I have similar rubric items (e.g., group project rubric items) across several classes (PSY 305, PSY 250).
Sometimes though, a rubric item needs to be tweaked just a bit (maybe in the wording or the scale) between class assignments. Would these changes necessitate a new rubric for each class or would it save variations of the rubric?
 
Thnx for the comments.

I use a content-centric, rather than class-centric approach. The central object is the textbook (or, rather, textbook-like thing).

Every book would have its own set of rubric items. Each class using the book would have the same exercises and same rubrics - subject to customization by the instructor.

Cindy - I'm imagining that individual instructors could override the standard rubric for each exercise with their own, if they wanted to. But I'm not sure what the interface for that would look like.

One strange thing I just thought of - instructors and authors could talk about rubrics, and jointly improve them over time. Wow, I just blew my own mind.
 
+Kieran Mathieson, thanks for including me on this project. Forgive me if my opinions aren't appropriate if I have misunderstood something. I guess you want the computer to rate student work, right? I am not sure how that would work. In any case, my first reaction is that rubrics are meant to provide meaningful formative feedback. The purposes for this are: 1. Act as a guide while students perform the task so they know what standard is expected of them and 2. when the teacher or peer gives feedback using the rubric, it allows the student to know exactly what needs to be corrected and how to do it. If this is your intention then having a rubric that shows "correct" and "wrong" will not help you. The rubric you propose will only help you with items that have right and wrong answers, like spelling and grammar errors. A computer can do this fine. The other items you have identified involve subjectivity: proper, formal, clearly, etc... and therefore need descriptors for the student to know what is expected for him/her. These seem to need some sort of human judgement.

The idea of giving authors the ability to create rubrics as they create exercises is great, as is the idea of an item bank and the proposal you make for each author to build individual rubrics. Ok, this looks great!
 
Thnx for taking a look.

There is no computerized feedback in my approach. ALL feedback is give by humans. If we take skill learning seriously, then assessment has to be done by humans. There is no choice.

If we want students to learn skills, we have to:

(1) give them lots of exercises. Have them create content - write stories, write programs, create Web pages, find bugs, etc. No multiple choice.
(2) give them formative feedback
(3) give them a chance to fix mistakes and resubmit

A class of, say, 30 students might do 40 exercises each, per semester. That's a lot of grading, and it's work that a computer cannot do.

Instead, I'm creating an optimized feedback system for humans to use. Clickable rubrics help that happen. Eliminate every mouse click, keystroke, and wasted cognitive operation possible.

In the first version of the feedback interface, I was able to give formative feedback on a simple exercise in as little as 30 seconds. Half an hour to an hour per day let me keep up with the students. The next version should be even better.

It's a complex project. There's a mildly entertaining short story at http://coredogs.com/article/tale-two-students .
 
I think building different rubric structures for students with different needs will be important. Students with different cultural backgrounds, different language needs, specific educational needs, etc.

So your textbook needs to know about the reader and be able to respond in ways that are appropriate for that student.

This introduces extra complexity into the authoring system whereby you may need ti be able to allocate different rubrics to different 'types' of student need.
 
+Kieran Mathieson thanks for sharing the mildly entertaining short story, I now understand the project much better! (which means I hope my feedback will be more appropriate next time). :)
Add a comment...