Shared publicly  - 
“There's a time when the operation of the machine becomes so odious—makes you so sick at heart—that you can't take part. You can't even passively take part. And you've got to put your ...
Audrey Watters's profile photoJames Petersen's profile photoLaura Gibbs's profile photoDavid Roy's profile photo
The claim is that the robo-graders grade the same as humans. That might be so as regards a final grade but the process is much different. I've been, among other things, a College Board AP Exam Reader. I can say from personal experience, how much thought and effort by many people goes into trying to grade each essay as fairly as possible. Many times, when the reader is not sure about a point or an argument, others are asked to back-read and comment. I recall one essay for which we had four different levels of people, right up to the question leader, trying to find more credit for a student's essay. Try to do that digitally... Of course it's a lot of work and of course it's easier to have them "turn it in" online. That's not the point.
Audrey, this is EXCELLENT - I am really concerned about this awful business, and I hope so much that your commentary here will be widely read. In a long discussion re: robograders here last week, I was so frustrated to learn that the CLA (the study whose results are at the heart of that Academically Adrift book) is robograded... which means that it can accurately assess essays, and only those essays, that are statistically similar to the other essays - but if someone writes something creative, unusual, extraordinary, it will be poorly scored. As a result, CLA as an assessment only works in the aggregate, knowing that it is throwing a lot of babies out with that bathwater. You can kiss creativity goodbye.
I am a teacher, I teach individual students, I prize creativity above all else in what my students write. I personally have no interest in mass aggregates of statistically similar instances. I have to question the logic of anyone who does think such statistically similar aggregates are going to do any of us - students, teachers, schools, colleges - any good at all.
Don't you think the secret is in combining the strenghts of human and machine? Computers are good at what we consider tedious and routine work. Humans (though not all of us) are good at the creative part. Combine them, and you get something better than a robograder: the 6-Million Dollar teacher, better at assessing grades, faster at marking, stronger and with the stamina to check more essays.
+Jeroen Fransen My students don't need mindless assessment - they need USEFUL FEEDBACK, which the robograders cannot provide. There is no shortcut to that - I keep trying to find even a basic grammar checker I can reliably recommend to my students and all the ones I have tested have been worse than useless, so bad (esp. so many false positives; that I cannot even recommend one. I would welcome a grammar checker; I don't need a robograder - and I don't think anyone does.
+Laura Gibbs That's exactly what teachers have been telling me, right after they've complained to me about the mountain of work that essay assessment involves. So with +Joyrite I designed a service that only automatically annotates those errors that can be found reliably (ie reducing false positives). The teacher then uses our web-based service to do the rest. We don't want to sit on teachers' chairs. We do try to learn from teachers' input about the mistakes they find and how they give feedback on them so every release we can lighten their work a little more.
I tested about a half-dozen and they were all, without exception, awful. I'd be glad to run my same test material through your Joyrite service if you have a free trial version. Please note that I am NOT complaining about marking student papers. It's my job and I love it. If teachers do not have time to read and respond to their students' writing, or to set it up so that the students read and respond to each other's writing, then they should not assign the writing.
Went to Joyrite, got to the English page but I wasn't sure how/where to test the service. I've been testing them using the same proofreading assessment I give my students and would be glad to test out Joyrite. I have a great system for annotating and responding to my students which I'm not looking to change, but if there is a grammar checker the students could use for automated feedback that is RELIABLE, I would be delighted.
Sorry to raise expectations, we haven't finished our service for English yet. We're in beta for Dutch and will expand to other languages afterwards. I admire your passion for your work, I recognize that! Would love to learn from your system for annotating and responding, though.
Let's definitely stay in touch - I've only got three more weeks of school and during the summer I am always thinking about ways to improve my class. I would be glad to test out your service when it is ready and I've been meaning to take some time and write up a description of how I do respond to student work (it's actually very simple, because I like things that are simple, ha ha). Anyway, I've circled you here and look forward to hearing more about your project. I would love to find something good; I was surprised that I had zero luck finding a good solution this semester. But I am still looking - my students make good use of spellcheck; I would like for them to have grammarcheck too. :-)
Most of this thread is why I like Google+, because it brings so many people together and can actually solve some potential problems too!

Thanks for this great post, Audrey.
+Jan Zawadzki I think the foundations should be fueling what you guys are doing!!! IMHO :-)
+David Roy Agreed! Plus, I am so grateful to have Audrey's detailed write-up to share with people who have not though through the really dire implications of this stuff, esp. if we go at it without really even thinking about what we are trying to accomplish.
+Laura Gibbs Isn't that the way of most school administrators, though? Move forward without thinking of the educational costs rather than the financial costs?
+David Roy Beg your pardon? I'm a school administrator and I don't fall anywhere near the generalization you describe.
Thanks for the the back-up, +Laura Gibbs . And +James Petersen , I was thinking more of university bureaucracy when making that comment. Sorry if it offended you. Not my intent.
Add a comment...