The following post appeared in the August 30, 1996 Judgenet Digest. Scott Bickham is the exam director for the BJCP. ------------------------------------------------------- From: Scott Bickham Date: Fri, 30 Aug 1996 08:11:41 -0400 (EDT) Subject: Re: Tasting portion of the exam jdecarlo@mail04.mitre.org (John A. DeCarlo) asked: > 1) What is the break-down of grading the tasting portion? As you know, the tasting portion of the exam is worth 30% of the total score. The tasting portion itself is broken down as follows: Scoring: 20% Beer 1: 20% Beer 2: 20% Beer 3: 20% Beer 4: 20% The scoring of the four beers is split evenly between perception, descriptive ability, feedback and communication (readability and completeness). With this system, it is extremely difficult for examinees to score below 50% on the tasting portion of the exam. Note that only the scoring and perception rely on the scoresheets of the proctors, and the graders are experienced enough to make adjustments if the proctors are weak or if there are biases in the exam beers. There is *no* ideal scoresheet for the beer, but the proctors' are used to see if the examinees are in the ballpark. When scoring exams, I generally make notes of the significant flavors and flaws that the proctors find in the beers and then see how many of these were identified by the examinees. Often, as with the Shelton exams that Peter Garofalo and I are grading, a significant number of examinees will find additional flaws such as oxidation or diacetyl in one or more of the exam beers. In that case, points are not deducted since flavor thresholds vary and those flaws may in fact be present. The scoring is done by adding up the absolute deviations between the examinees scores and those of the proctors. I generally also compare the proctors' scores with the average. They are usually very close, but if there is a large difference, an adjustment may be made. For example, on the Cincinnati exams, the proctors scored one of the beers 8 points higher than the average. However, the average of the four examinees who scored above 80% on the essay portion was only a point less than the proctors, which to me indicates that the proctors scored this beer accurately. To get a quantitative measure of scoring ability, a chart is used. If the average scoring deviation is 2-3 points, the examinee is given 18-20 points, i.e. Master level. If the average deviation is 7 points (the recommended amount), 12 points, i.e. recognized level, are given. In between differences are scaled accordingly, and no fewer than 9 points are given.