On Online Education and Assessment
Last night I enrolled in my first online class. It was a course on fantasy and science fiction – literary genres that are near and dear to my heart. This class, which was taught by a professor who is at the top of his field and teaches in a university that consistently ranks as top 50 in the U.S. Even with all of my formal education, I was looking forward to learning many things.
After the first hour of going through the course materials, I had to drop the course.
The reading list offered was excellent and it showed that some thought had gone into the design of the course. Traditionally studied texts like Dracula and Frankenstein were to be studied next to the Grimm’s fairy tales and contemporary science fiction from masterful writers like Cory Doctrow.
Then I went into the “how the class will be evaluated” section.
Not surprisingly, students were to write essays and reader-responses to other student essays. These essays would be evaluated on a three point scale – 1 for mediocre, 2 for average, and 3 for outstanding – and at the end of the term all the points would be averaged and the final grade would be quantified based on the final number. If I recall correctly, the A went from 2.2 to 3.0.
This seems sound enough, with the exception of two major problems.
1. The instructor didn’t offer any criteria as to what made a “bad”, an “average”, and a “great” essay. He simply expected that people would “get better as the term progressed”.
2. The instructor stated that “at least 10 to 30% of the total grades for all papers will be a 1 and no more than 20% of the total grades for all papers will be a 3.” He said that “the great majority of the papers will get a 2, because that’s what’s average”, and concluded by saying that “if all papers are extraordinary, then by definition there’s nothing extraordinary at all”.
The problem with the first stipulation, of course, is that there is nothing to aspire to. This leaves students grasping for straws and bumbling in the hopes of getting it right. But the second stipulation is even more problematic.
By this stipulation, if you consider Wordsworth, Blake, Keats, Shelley, Coleridge, and Byron, then they are all just average writers because none of them is really that more outstanding than the other (maybe Blake is a little bit better). Because all of them are roughly equally outstanding, then , by definition, none of them are. And so, in a poetry class that follows these same standards of assessment, Wordsworth would get a B. Rime of the Ancient Mariner? It gets a 2. It’s just as good as Marriage of Heaven and Hell.
But it wouldn’t stay there. Because we need to have at least 10% of the grades as “poor”, then Khubla Kahn would get a C because it was incomplete.
“But mister Johan sir”, you might contend, “that is an unfair assessment of the situation. You can’t compare the great poets of the 19th century to current students!”
Maybe, maybe not. It depends on who the student is. Regardless, the original point remains: instructors in the humanities shouldn’t have vague criteria based on “what I think is good” that shifts from one paper to the next. For each paper there should be an established set of guidelines to follow. For example, a criteria that says “if a student creates a solid, well-crafted claim and uses reasonable logic that can be demonstrated as valid by enthymemes or by other methods, valid and coherent, well-thought out reasons that demonstrate an in-depth level of knowledge on the issue, and valid evidence from credible sources he or she will get a full score on the “content” section of the essay” is far more specific and helpful for the student than “if it’s extraordinary you get a 3”.
And that is why I dropped the class – I didn’t agree with the instructor’s assessment techniques.
What’s even more troubling is that I have heard from friends and colleagues that many professors use grading systems that are just as arbitrary. There is a rumor in the University of Puerto Rico (where I did my B.A. and two of my M.A.s) that one of the professors simply assigned the highest 25% of the grades an A, the next 25% a B, then a C to the following 25%, and a D to the lowest 25%. In that spirit, some students who managed a whopping 40% passing score in the class got As. I have also heard of other instructors who follow a similar approach, but use a bell curve statistics method (top 2% get As, the next 10% get Bs, 76% of the people get Cs, 10% D, and 2% F). This is even more troublesome as people with 90% passing rates would get Fs sometimes. In another institution I heard that a professor had a dart board in his office, but instead of points he had the labels as A, B, C, D, and F, and that yet another professor threw papers down a staircase and those who landed closest to him got high marks.
Personally, the worst I have actually seen (not things I’ve heard from other people) is instructors who give grades based on “what they think the student deserves”. At the end of the term, this instructor would sit down with his roll book and say “Student A… he deserves about a C. Student B…. a B sounds good for him”, all the way down his list. This is a system almost as bad as the one used by the instructor in the online course I dropped from.
So what’s the point of this commentary? I would like to encourage all university professors, but specially those in the humanities (because that’s my area and the one I care about the most) to take some courses or workshops in assessment techniques. If we are to give students a fair chance, we need to be fair about how we grade their papers. That, in turn, will lead to better educational practices.
In order to protect the privacy of the instructor, his home institution, and the organization that I took this course through, I did not mention any personally identifiable information.
Posted on August 8, 2012, in Education Commentary and tagged assessment, assessment fail, fail, online assessment, online class, online education, online education fail. Bookmark the permalink. 1 Comment.