When the London Amateur Brewers (LAB) asked if we would be interested in joining the judges for their recent regional home brewing competition, we jumped at the chance.
Like most people, we’ve observed the results of judged competitions with bemusement in the past: “How did that win!?” Although we knew it wouldn’t be on the scale of CAMRA’s Champion Beer of Britain we thought it might give us some insight into how such decisions get made.
Arriving at Beavertown’s new Tottenham Hale brewery at 8:45 am on a Saturday, the first thing that struck us was how reliant it all is on good will — on unpaid volunteers with the necessary skills giving up half their weekend. (Our only qualification for judging beer is enthusiasm which is why we were each paired with someone more experienced who had achieved some level of BJCP accreditation.)
Ah, yes — the BJCP. Judging a competition with something like 150 entries without using some kind of rules, however arbitrary, would be impossible. The BJCP system breaks those entries down into categories and provides rigid style definitions: they say Cream Ale ought to have ‘sparkling clarity’, for example, so even if your attempt tastes incredible, it will lose points if it is hazy and has lumps floating in it.
In fact, some beers which didn’t really taste all that great scored reasonably well because they were clear, correctly conditioned, and formed and retained a decent head. Flavour isn’t everything, at least not in this context.
The emphasis on ‘style’ works reasonably well with fairly common types of beer such as bitter or IPA. The judges had lots of experience drinking commercial examples and knew what to expect. ‘California Common’, on the other hand, presents more of a challenge, as does ‘Festbier’, only a handful of which are ever found on sale in the UK, and usually past their best — judges were largely reliant on the written descriptions from the BJCP to decide if a beer was ‘correct’.
There was also a sense that some entrants struggled to find a category for their beer and, shoe-horning it into the closest fit, found themselves docked points for not being ‘true to style’. Others seemed to have attempted one style but accidentally brewed another (e.g. it had gone sour, or turned out darker than expected) and were trying it on.
All in all, the BJCP system rewards conformity and ‘cloning’ while punishing creativity.
Now, let’s be clear: the judges try their absolute best, within the rules they’ve signed up to, to make sure great beers get the recognition they deserve. There was lots of agonising and soul-searching, with arbitration from stewards and senior judges, and we heard variations on “I’d drink that all day if I could, but it’s not a [STYLE]” throughout the day. But final decisions in each category were made by re-tasting the top scorers and ranking them without close reference to style guidelines: which one really was best?
The toughest part, arguably, is completing feedback forms for each beer. If the comments aren’t honest, then what’s the point? But if they’re too blunt, then they might be discouraging, and the LAB don’t want to put people off brewing or competing in future. Generally, it was possible to say, even where a beer scored poorly, that it was a problem with ‘trueness to style’ rather than poor brewing technique. There was, however, no nice way to say, “This beer made me retch and forced me to spit it into a bucket.”
Some good beers no doubt got knocked out, just as some good teams have been knocked out of the World Cup, but those that made it to the final judging table for ‘Best of Show’ were undoubtedly deserving. Here, a couple of category winners simply didn’t stand up to scrutiny: the best saison was better than the best lager, in absolute terms, for example. The overall winner, after much impassioned argument and careful consideration, was a flawless black IPA with commercial potential.
On balance, we might enter a BJCP-rules competition if we thought we’d brewed a particularly good example of a standard style, but not if we’d brewed anything remotely unusual.
And, from now on, we’ll certainly have a little more sympathy with the judges at national and international competitions.
11 replies on “On Judging”
The feedback is probably the most important part of the whole event for the entrants, so even if it is difficult for judges and they may get ‘feedback fatigue’ it’s vitally important that it is carried out to the best of their ability at the time.
I don’t understand this part of your post:
“If the comments are honest, then what’s the point?”
Because surely the entire point of judging a beer is to be honest? We have yet to meet any amateur brewers who want people to lie about how good/bad their beer is. Constructive feedback is essential. Obviously it is nice to hear that people enjoy something you have created. But it is absolutely no use at all to a brewer to pretend you like their beer if you think it has faults. You are doing them an injustice by not telling them what you think is wrong. In a competition the entrants have submitted their beers in order to get feedback (and paid a fee for the privilege) so they are entitled to whatever useful comments a judge can provide.
I appreciate that judging a large number of beers is not easy and that there will always be, as you said, good beers that lose out. I also understand that despite having strict BJCP guidelines there will inevitably be some level of subjectivity involved. But what I do not understand is how some judges have scored entries against their own guidelines for a style. We had bizarre feedback on some of our entries which was out of sync with the actual BJCP guidelines, e.g implying that a saison must be orange in colour and if it isn’t orange it is ‘out of style’. Or stating that an American IPA is over-hopped for the style – I’ve drunk many (fresh) American IPAs on the west coast of the US so I know how incredibly hoppy the style can be. Not everyone likes extremely well hopped beers – but that does not mean they are outside of the BJCP style guidelines. This sort of inconsistency in judging is off-putting for entrants.
Is it true that none of the entries were chilled prior to judging? Probably didn’t do the Kolsch or lager entries any favours.
“Is it true that none of the entries were chilled prior to judging? Probably didn’t do the Kolsch or lager entries any favours.”
They weren’t chilled, and we wouldn’t normally drink them at room temp. (They weren’t warm.) Having said that, we quite often *tasting* beers which are supposed to be served chilled at room temp at least once, if we can, because chilling can help to conceal downright faults. It was, at least, a level playing field within the category.
From what I recall, the biggest problem with lagers was (a) lack of carbonation (chilling might have helped with this a bit, I suppose) and (b) being entered as one very specific subset of lager with colour/strength/hoppiness that would have fit better into another category.
Brewing “to-style” can be a fun, creative, and rewarding part of the homebrewing hobby. I like to enter competitions as a test of my skill as a brewer and do it knowing what is needed to get a good score- execute the brewing process as flawlessly as possible using a good recipe for the style. Its also a way to learn beer styles. Nothing wrong with that, is there?
I personally think it helps tremendously to take on this type of test of brewing skill to help when it comes to brewing out-of-style. Styles exist because there are common sets of taste parameters that work well together. Learning those can help with it comes to playing around with those sets.
“I don’t understand this part of your post…”
That would be a typo… Fixed now.
Ok. Now I get it. I thought you were trying to make a point that I couldn’t make out.
We’re hoping to invest in a state-of-the-art stainless steel proofreading unit from Germany next year.
There was no option for chilling at the event, so the only beers that were chilled were those that were transported up on the morning by the brewers themselves. If they were judged early, that was great, otherwise they too would have warmed up by the time they were judged.
A comment on the sensitivity of feedback:
The scarcest thing in home brewing is getting honest, critical feedback from people with a good palate. If you are at the level where you are entering competitions, your friends and acquaintances will happily guzzle away your brews with no complaints.
It is common for a home brewer to enter a beer with faults. They are checking to see if their suspicions are confirmed by the judges.
Lastly, the BJCP guidelines are imperfect and have a lot of holes. Recently an update has been published for review that is much improved.
But isn’t judging of homebrewing just a case of one hobby interacting with another? Trueness to style is like excelling in novelty mini putt golf. Not a great deal of broader application even if fun for those involved. As for the new rules, the world waits like an excited child on Christmas Eve for explosion of the Kentuky Common “style” examples in a few years – perhaps as much they anticipate the circus coming to town, the one with the sharpshooter who uses a mirror.
“Trueness to style is like excelling in novelty mini putt golf. Not a great deal of broader application even if fun for those involved.”
Oh, I don’t know about that — isn’t it learning to walk before you run, if you’ve got hopes of going into commercial brewing as we suspect do some of these entrants?
That might be the case of the “styles” were not mainly experimental as they are now but, yes, it would be relevant as a line on a CV for someone seeking a position as a brewing assistant where following rules mattered. But that’s a tiny part of the whole.
[…] Boak & Bailey’s Beer Blog – On Judging […]