Jamie Goode: Confronting bias in wine judging

By , 1 December 2021



How to make allowance for different alcohol levels, extraction techniques and wooding regimes? Image: OMTWS.

Wine comes in many different styles. Different people have different tastes. You could even go so far to say that there are even national distinctions in taste, although, of course, this might be a generalization. Only the other day I was at lunch with friends from the wine trade, and one of the wines we shared was a highly rated Saint-Emilion. It was an expensive bottle of wine, but we didn’t really like it. Who’d buy that, one person asked. It’s for the Americans and Chinese, another person countered. A third person was quite upset about what we were saying – he liked the wine.

Personal preferences are one thing, but where do these fit if your job is to assess wine critically? The answer is complex, but it’s an interesting question to consider.

First of all, if you are a buyer, you have to think of your customers. You know what you can sell, and at what price. So what you like is sort of secondary – it’s good news if your palate preferences resonate with your public, but that isn’t essential. The exception here would be importers whose agency business is built around their taste and their style of wines. In some ways, their palate preference and stylistic sensibilities are part of their branding, and people come to them or avoid them on this basis. There are a few importers in the UK who I trust because of this: if they bring the wine in, I know I’ll probably like it.

But what of critics? Here it is a more complicated job, and I can only really speak from my own experience. If people show me wines, or send me samples, they want my honest opinion, but that isn’t as simple as saying that I score wines on the basis of how much I like them.

Of course, liking is part of assessment, but a critical assessment is much more than simply a measure of hedonic valence. When a novice comes to wine, hedonic valence is all they can manage: I really, really like this wine, but I can’t tell you why, and nor can I tell you why I don’t like this other wine, but I just know I don’t. Part of being a competent critic is having experience; having a database of previous wines tried, of benchmarks, of context gained from lots of tasting. This means you can take a step back from your initial reaction to a wine, and try to understand it a bit – to listen what it has to say. And it’s important to be able to put the wine in context.

For me, I know I have some stylistic biases. I’m really drawn to lighter, more elegant reds. I love saline, mineral whites. I like wines that aren’t too obvious. I’m not really a fan of new oak when its apparent in the wine. I don’t like over-ripeness or over-extraction. I love alternative elevage: concrete, clay, large oak. I don’t like apparent volatile acidity. And I could go on.

But when I assess wines I try to set these aside. Let’s take South Africa as an example. The sorts of wines that Duncan Savage, Eben Sadie, Peter-Allen Finlayson, David Sadie and Dominic Rall make really move me – I could add many more names to the list, but this is just off the top of my head. There’s a shared aesthetic sensibility here, and I usually score these wines highly.

But what about full bodied Stellenbosch Cabernet Sauvignon? The sort of wine that fetches high prices and gets many collectors excited. How do I respond to a wine like this as a critic? Here, I have to set some of my own preferences aside if I’m to assess the wines professionally. So I might taste a bunch of them, and score them as to how much I think they are a good example of this style. I don’t want to be scoring a wine for what it is not, which is a common mistake among wine judges. They might be faced with a flight of wines where they disapprove of the riper, richer examples and award a wine for what it doesn’t have (ripeness and richness) rather than because it’s a particularly good wine. Say I’m presented with the new release of Kanonkop’s Paul Sauer. This is not a wine I would ever buy, because I find there’s too much extraction and new oak (although these wines age spectacularly), but I usually score it highly because it’s a good example of its style. There comes a point, though, where I think a wine has deviated too far and can no longer be a good example of its style because that expression is illegitimate. I don’t think there’s ever a sound reason for a Stellenbosch red at 15.9% alcohol with pruney fruit, lots of vanilla from new American oak and a VA of 1.3. [OK, maybe there might be, but I just can’t imagine it happening.]

All this is true up to a point. I think it is possible, at some level, for critics to allow their own palates to shape their work. But there’s only a small skew in this way that works. People who read my stuff know the sorts of wines that really excite me, and they can tell from my writing which they are. But I will score highly wines that I think are really well made, that aren’t in my preferred style. I won’t punish people for not sharing my tastes. But if you read what I write, and look at which wines I do write up versus those that I don’t, you will get a good feel for what my tastes are, even though I try not to let it shape my scoring.

Do you agree? Or do you think critics should be letting their personal tastes shape their scoring more?

  • Jamie Goode is a London-based wine writer, lecturer, wine judge and book author. With a PhD in plant biology, he worked as a science editor, before starting wineanorak.com, one of the world’s most popular wine websites.

Help us out. If you’d like to show a little love for independent media, we’d greatly appreciate it. To make a financial contribution, click here. Invoice available upon request – contact info@winemag.co.za


8 comment(s)

Please read our Comments Policy here.

    Christian Eedes | 1 December 2021

    Hi All, I’ve made the argument before that we tend to confuse “reliable/repeatable” with “valid” when it comes to making wine judgements. I’m often drawn to a wine in one context but not in another (as I believe we all are) which makes giving the same score to a wine every time you encounter it at least a little difficult.

    Also, I think we privilege the intention of the winemaker too much – what his or her creative agenda is and where he or she is attempting to position a particular wine in the market. These are often very difficult to know and therefore what the critic brings to bear in considering said wine is equally if not more important.

    GillesP | 1 December 2021

    I agree and support the column of Jamie although this brings me to a question: why does it seems most SA wine critics find best of classic Bordeaux and perhaps Burgundy white and red not to their liking ? I certainly see a trend here which is the opposite of many other countries.

    Colin | 1 December 2021

    It is one of my big gripes with this website and Christian in particular. He will never admit it, but it is clear from the way he scores – on the few occasions where it is done blind – that he prefers bigger, riper styles over lighter, more elegant styles. In particular in the reds. The recent scores for the CWG wines showed as much again.

    And whilst it is perfectly acceptable to prefer one style over another, as Jamie writes here – as a critic one should score the wine based on how good it is, not on your preference.

      Duncan | 1 December 2021

      Funny, I’ve always taken Christian’s preference to lean towards relative elegance and lightness (relative being the operative word: my casual observation is that Tim James tends to rate more restrained wines slightly higher than Christian) albeit in wines that are classically ‘well made’.

      Not that a critic having preferences is a bad thing (as the piece eloquently demonstrates).

        Colin | 1 December 2021

        I agree with you with the sighted ratings that there’s a preference to rate syrah – mostly the kind with a fair bit of wholebunch character – highly. I was mostly referring to his blind ratings. See the CWG ratings in particular.

      Gareth | 1 December 2021

      I find this perspective really interesting, because I too feel that Christian has a natural bias, however my view is completely different.

      From what I recall, he seems to reward lighter, more acidic crunch reds (particularly Syrah) while perhaps being a little miserly with his scores of the bigger Cabs and B style blends.

    Mike | 1 December 2021

    Absolutely! We need an emoji here please Winemag, of a gif, one of hands clapping or whistle-blowing or the band playing! There are wine critics and then there are wine critics…

    Kwispedoor | 1 December 2021

    We have very similar preferences, Jamie. But one sentence is key for a professional judge: “I won’t punish people for not sharing my tastes.”

    A pro judge should never veer too far off course in terms of stylistics, for instance pre-tasting briefs that aim to bring any advantage or disadvantage to a certain aesthetic, instead of rating the inherent quality of what’s in the glass. Or over-correction regarding lesser liked styles – for instance where you add undeserved points to an oaky red because you know you don’t like oak and you are afraid of punishing the wine unfairly. Arguably, tasters should also not know what category of wines they are tasting, in order to avoid any bias.

    If those sort of things are avoided and the judge tries to be ultra objective, his/her personal preferences will still influence his/her assessment and score, because it’s hardly possible to eliminate it completely. That very small personal bias that necessarily finds a way through in the end is part and parcel of such a complex process. Thus, there’s no need to let personal taste knowingly shape your assessment, because it’ll happen on a small scale in any event.

    Of course things might change if one is not judging professionally. The readers of a blog or a website might in fact just be looking for, and following, a taster with an aesthetic bias.

Leave a Reply

Your email address will not be published. Required fields are marked *

Like our content?

Show your support.