Lauer Chapter 5

gathered merit ratings from 1 to 9, in addition to comments provided by the readers regarding likes and dislikes about the essays. 300 papers produced 11, 018 comments, which became grouped under 55 headings. The Paul Deiderich study seems appropriate to analyze given it took place at my old school in Champaign, IL. For his study, Deiderich questioned if the reliability of grades on essays could be improved. Even today, this is a pressing issue, mainly with ETS and similar services. He used 300 papers (written by the subjects who were all college students in their first month of school), and the papers covered three universities. The context of the study involved a highly specialized research team with clear goals. For instance, sixty readers graded the essays (turned out to be 53 eventually). These readers included professionals such as English teachers, writers , editors, lawyers, etc. The following five factors were considered: ideas, organization, wording, flavor, and conventions like punctuation. The study specifically looked at the factors that EDUCATED readers consider when grading essays. When collecting data, Deiderich used statistical analysis such as standard deviation, which he compared with percentiles, standard scores, range and letter grades.

The range of readers in this study provides for more accurate comments and scores. All the readers were not, for example, English teachers. The readers were not randomly selected, however, but the study has been replicated with similar results. I know that ETS has little variance in the scores of their essays, so they seem to have a system down that is reliable. Whether or not it is valid is another story. Deiderich used various readers, but I wonder if the variety of his readers was almost too great. Would, for example, a lawyer and a social science teacher grade in a similar manner? Should they? I am not sure that context was considered in this study to a great enough extent. That is, the reliability of grades on essays does not really seem to be the pressing question. Each essay for each subject will be written under a certain set of conventions. What works for one paper might not work for another. Also, I would think using three different universities could produce quite different essays, given that I do not know which universities were used. There does need to be a way to evaluate writing in the way ETS does when we deal with massive numbers of essay writers. That does not mean, however, that the BEST method of evaluation is used, especially when we consider all of the various aspects that go into producing a good essay that is relevant to the context and situation.