|
In order to test effectively, the examiner requires not only a sound knowledge of the characteristics
of evaluation, but also a firm understanding of the possible errors that can occur throughout the
evaluation process. Errors in evaluation fall into several categories.
Personal Bias Error
Personal bias is indicated by the tendency of an examiner to rate candidates or a particular group of
candidates the same.
Central Tendency Errors
Central tendency errors are indicated by a tendency to rate all or most candidates as average. The
examiner really “feels” that the performance of most candidates is not as good as it should be and therefore
underscores a candidate’s good performance. On the other hand, the examiner is reluctant to cope with the
possible emotional response of a candidate or a recommending instructor. This results in padded or inflated
assessments of poor performance. This error may also occur because an examiner does not want to put effort
into making a decision. An average mark is easier to defend.
Generosity Errors
Generosity errors are indicated by a tendency to rate all individuals at the high end of the scale
and are probably the most common type of personal bias. This could be caused by an examiner’s desire to be
known as a nice person.
Severity Errors
In this case, all or most candidates are graded at the low end of the marking scale. Examiners
may feel that the published standards are too low and score the test against their own set of standards. This
type of examiner feels that few people can fly as well as they can.
Halo Effect
This occurs when an examiner’s impression of a candidate is allowed to influence the assessment of
performance. Halo error can result in rating an applicant too high or too low. One form of halo error is the
error of leniency. Leniency has it's source in an examiners likes, dislikes, opinions, prejudices, moods and
political or community influence of people. For example, when testing a friend, acquaintance, or high profile
individual, an examiner may give undeservedly high marks or, conversely the error of stereotype.
Stereotype
As with the error of leniency, the error of stereotype has its source in likes, dislikes, opinions,
prejudices, etc. In this case, however, an examiner may allow personal opinion or prejudice to influence the
assessment of the candidate and award undeservedly low marks or high marks.
Logical Error
Logical error occurs when an examiner assumes that a high degree of ability in one area means a similar
degree of competence in another. This is especially true if the two items being assessed are similar or
related. A good mark on one or two items does not mean the candidate is also qualified on all items. The
full test must be completed and marked.
Error of Narrow Criterion
This may occur when an examiner has a group of candidates to test. The examiner may, under this condition,
rate each applicant against the others within the group instead of against the
standards. If the group to be tested is above average, a candidate who is of average ability may be awarded
an undeservedly low mark. If the group of candidates to be tested is below average, then a candidate who
performs the best within this group may be awarded a higher assessment than actually deserved.
Error of Delayed Grading
This type of error occurs when there is a delay in the assessment of an item, resulting in a tendency to
award average marks due to the lack of information and/or poor recall. The use of the top or bottom end of
the marking scale would be avoided. By not making an assessment immediately after the event, examiners may
award assessments based upon an overall impression of the flight test. This results in an erroneous assessment
and a flight test report that is of little value to the training system.
Standards Error
All the errors we have discussed result in a standards error. However, if an examiner is not thoroughly
familiar with established standards, as outlined in the applicable flight test guides, it is virtually
impossible to conduct an evaluation to that standard.
While these errors may appear obvious on paper, they may not be under flight-test conditions, especially
as the judgement of the examiner may be obscured by a combination of two or more. Examiners must, therefore,
be aware of these errors to consciously prevent them from influencing the validity of the tests they
conduct.
|
