Method

In holistic scoring, essays are not given low scores just because they contain many mechanical errors, nor are they given high scores just because they are well organized. The reader considers the overall impression created by the student's writing sample and assigns a score consistent with that overall impression.

The score a reader assigns is defined by a set of criteria that define important traits in all writing: at Southeast, the criteria include focus, organization, development, style, correctness, and if necessary, references. These criteria, in turn, define the scoring scale applied to the essay; Southeast's scale runs from 1 (incoherent) to 6 (clearly excellent).

The evaluation scale is divided into two halves: the top half is labeled "mastery," and the bottom half is labeled "non-mastery." The three scores in each of the halves then represent high, medium, or low levels of mastery or non-mastery.  Bearing in mind all of the writing traits to be considered, a reader first decides whether the essay shows, overall, some kind of mastery or some kind of non-mastery.  This is a yes or no decision. Then, having made that decision, the evaluator decides whether the level of mastery or non-mastery is high, medium, or low.  If the reader decides that the test shows low mastery (or just barely "yes"), he or she gives it an overall score of 4. For medium mastery, the score is 5, and for high mastery the score is 6. In the same way, essays demonstrating high non-mastery (or just barely "no") are assigned a 3. For medium non-mastery the score is 2, and for low non-mastery the score is 1.

Each essay is initially scored by two trained readers in a double-blind process: that is, the two readers do not know each other's score or each other's identity, nor do they know who wrote the essay. If the two readers give the essay "matching" or "touching" scores, the essay's score is figured from the first two scores. Otherwise, in the case of "discrepant scores," a third reader scores the essay; the essay's final score is determined by a combination of the three scores.

Essay Scores are Determined as Follows

	Reader 1	Reader 2	Reader 3	Reader 4
Matching	4			4
Touching	4			4.5
Discrepant	3		3	3
	3		4	4
	3		2	2.5

Criteria

Focus

In order for students to achieve focus in their writing, they must do the following: a) they must address the specific topic presented by the test question, and b) they must present a main point or clear purpose for communicating.

The writing proficiency test contains two specific questions. Clearly, students who do not write an answer to the questions have no chance of scoring well on the test. However, even students who address topics will write poorly if they do not limit their main idea to something they can adequately discuss in the time allotted. This limiting of the topic is often called "focusing" because both the writer's attention and the reader's attention are zeroed in on a particular aspect of a broad subject.

This "focusing" often takes the form of a thesis statement, a sentence that states the main idea of an essay. Generally, the thesis statement occurs in the introductory paragraph. When writers open their essays with anecdotes, statistics, or other attention-getting material, the thesis statement is often placed at the end of that paragraph or the beginning of the next paragraph.

Organization

Once writers settle on a main idea, they must think about the most effective way to organize their materials in order to convince their readers that the main idea is a reasonable one. Thus, the supporting information must be presented in some sort of logical progression. Obviously, if readers cannot follow the discussion, they will have no reason to accept the main idea.

Planned essays are usually divided into three parts: a beginning (introduction), a middle (body), and an end (conclusion). What goes into each of these three parts depends on the main idea to be developed, the evidence available, and the writer's strategy. Whatever plan of organization is used, the materials must be logically ordered and presented, and each step in the plan must be clearly signaled by the appropriate transition words or phrases.

Development

When the body of writing is only one paragraph long, the writing is not an essay. Instead, it is one paragraph with its beginning and ending improperly separated from the middle. The middle of an essay will have at least two to three paragraphs, and each of these paragraphs will present one major step in a logical plan.

These middle paragraphs usually open with the main idea to be discussed in the paragraph. (This sentence is usually called the topic sentence.) Without stating their main ideas, these paragraphs are likely to lack organization, and more often than not, they become simply a collection of unsupported major ideas that lead nowhere.

The supporting material in these paragraphs must be specific or concrete details that support the writer's point of view or main idea. This material, which illustrates or explains the broader topic sentence, must be presented in a logical order.

Style

Sentence coherence, diction, and tone combine to compose the element of style. Coherence is a result of sentence patterns, pronoun reference, and transitional connectives. In non-technical terms, coherence refers to the impression that the writing "flows" and that the essay is "of a piece." Diction signifies the appropriate choice of words; the words used must be accurate, appropriate, and effective in conveying the writer's intended meaning. Tone is the emotional attitude of writers toward their subject and audience. Whatever the writer's approach to the subject, the tone must be consistent and appropriate to the writer's overall purpose.

Correctness

Correctness covers the areas of grammar, spelling, punctuation, and manuscript preparation. Correctness is important because, without it, the reader may get the wrong information. Because they create the most confusion, the most serious errors are flaws in sentence structure, such as fragments, comma splices and fused sentences, and errors in agreement, such as subject-verb agreement and pronoun-antecedent agreement problems.

Correctness is no substitute for a thoughtful paper; it is better to have clearly stated generalizations that are supported by convincing specific details than to have a perfectly correct paper that makes no point or that does not support the point with concrete detail. Proofreading is an indispensable, but last, step in writing.

References

Writing that makes use of outside source materials is called "referential" writing. In Part II of the proficiency test, students demonstrate how logically, insightfully, and elegantly they can incorporate into their own essays paraphrases and direct quotations from the outside materials provided during the test.

Every time writers use material that is not their own or that is not common knowledge, they must indicate where they got the information. This is true regardless of whether the writers are paraphrasing or directly quoting the source material. In this testing situation, no particular style of documentation is required, although students must document each use of source material in a consistent and accurate manner.

Holistic Scoring Scale

 score in Southeast's holistic scoring scale is described below. Each description begins with the numerical score and a short characterization of that score. Then, each of the holistic scoring criteria is listed with a description of the level of achievement on that criterion for that score.  An asterisk indicates a criterion applicable only to Part II of the writing proficiency exam.  For scores 3, 4, and 5, the description concludes with a link to a sample essay with commentary.

While the descriptions provided indicate the characteristics of achievement at each level for each writing trait, it is important to remember that a given essay probably will not score in the same category for all criteria; an essay awarded a 5, for example, might rate a 6 on focus and a 3 on style. Test evaluators award scores on the overall impression of the writing using the following scale:

Holistic Scoring Scale: 6 (Clearly Excellent)

A score of 6 designates a clearly excellent piece of expositional or referential writing.

• Focus: the main idea is very clearly stated, and the topic is effectively limited.
• Organization: a logical plan is signaled by highly effective transitions; the essay's beginning and end are effectively related to the whole.
• Development: all major ideas are set off by paragraphs which have clearly stated or implied topics; the main idea and all major topics are supported by concrete, specific evidence.
• Style: sentences relate to each other and to the paragraph topic and are subordinate to the topic; word and phrase choice is felicitous; tone is consistent and appropriate.
• Correctness: there are no major mechanical errors (e.g., agreement) and only a few minor errors (e.g., spelling).
• References: source material is incorporated logically, insightfully and elegantly; sources are documented accurately, elegantly and emphatically.

Part I Sample Topic

Part I Sample | Score 6

Part II Sample Topic

Part II Sample | Score 6

Holistic Scoring Scale: 5 (Still Impressive)

A score of 5 designates a still impressive piece of expositional or referential writing.

• Focus: the main idea is clear, and the topic is limited.
• Organization: a logical plan is signaled by some transitions; the essay's beginning and end are clearly and effectively related to the whole.
• Development: almost all major ideas are set off by paragraphs which for the most part have clearly stated or implied topics; the main idea and all major topics are supported by concrete, specific detail.
• Style: paragraphs are built on logically related sentences; word and phrase choice is consistently accurate; tone is nearly consistent and appropriate.
• Correctness: there is only one major mechanical error or a few minor errors.
• References: source material is incorporated logically and proficiently; sources are documented accurately.

Sample Essay for Part I | Score 5

Sample Essay for Part II | Score 5

Holistic Scoring Scale: 4 (Adequate)

A score of 4 designates an adequate piece of expositional or referential writing.

• Focus: the main idea is clear or clearly implicit, and the topic is partially limited.
• Organization: a logical plan is signaled by transitions; the essay's beginning and end are somewhat effective.
• Development: most major ideas are set off by paragraphs which mainly have stated or implied topics; the main idea and almost all major points are supported by concrete, specific detail.
• Style: sentences in paragraphs are subordinate to topics; word choice is almost always accurate; tone is sometimes appropriate.
• Correctness: there may be a few major and minor mechanical errors.
• References: source material is incorporated logically and adequately; sources are documented accurately for the most part.

Sample Essay for Part I | Score 4

Sample Essay for Part II | Score 4

Holistic Scoring Scale: 3 (Developing)

A score of 3 designates a developing piece of expositional or referential writing.

• Focus: the main idea is unclear, and the topic is only partially limited.
• Organization: there is an attempted plan which the reader must infer; the essay's beginning and end may be ineffective.
• Development: some major ideas are set off by paragraphs which may have stated or implied topics; some major points in paragraphs are supported by concrete, specific detail.
• Style: sentences may not be subordinate to topics; word choice is generally accurate; tone is often inappropriate.
• Correctness: some major and minor mechanical errors are present.
• References: source material is incorporated but sometimes inappropriately or unclearly; documentation is accurate only occasionally.

Sample Essay for Part I | Score 3

Sample Essay for Part II | Score 3

Holistic Scoring Scale: 2 (Rudimentary)

A score of 2 designates a rudimentary piece of expositional or referential writing.

• Focus: the main idea is unclear, and the topic is unlimited.
• Organization: there is no clear plan; the essay's beginning and end are not effective.
• Development: few major ideas are set off by paragraphs; few paragraphs have stated or implied topics; supportive detail is imprecise, unclear or redundant.
• Style: sentence relationships at times are confusing; word choice is frequently inaccurate; tone is inappropriate.
• Correctness: many major and minor mechanical errors cause confusion.
• References: source material is inappropriately or unclearly incorporated; documentation is infrequent.

Part I Sample Topic

Sample Essay for Part I | Score 2

Part II Sample Topic

Sample Essay for Part II | Score 2

Holistic Scoring Scale: 1 (Incoherent)

A score of 1 designates an incoherent piece of expositional or referential writing.

• Focus: the subject and main idea are unclear; no apparent attempt has been made to limit the topic.
• Organization: there is no discernible plan; no attempt is made to compose an effective beginning and end.
• Development: major ideas are not set off by paragraphs; only one, if any, paragraph has a stated or implied topic; little or no supporting detail is used.
• Style: sentence relationships must be inferred; word choice is often confusing; tone is inappropriate or distracting.
• Correctness: many varied major and minor errors occur, making the paper difficult to read.
• References: source material is never incorporated or incorporated inappropriately or unclearly; documentation is inaccurate.

Part I Sample Topic

Sample Essay for Part I | Score 1

Part II Sample Topic

Sample Essay for Part II | Score 1

Sample Graded Papers

Using a topic that has been discontinued, we have included here a set of three actual (anonymous) student papers with their scores, along with an analysis of their strengths and weaknesses. There are many ways of obtaining a particular score, so these should not be taken as templates or models to follow. Still, they provide some concrete examples of the ways that the scoring criteria are applied in practice.

Part I	Part II
Sample Paper (Score 3)	Sample Paper (Score 3)
Sample Paper (Score 4)	Sample Paper (Score 4)
Sample Paper (Score 5)	Sample Paper (Score 5)