Tim James: Yet another observation on sighted/blind tasting

By , 28 September 2018



Tim Atkin SA Special Report 2018Talking of Tim Atkin and his high scores (we were, weren’t we?) … I had the privilege of attending his tasting of 158 (of the 160) wines he scored at least 95 points for his 2018 Special Report. His preface to the tasting booklet described it as “the most amazing line up of South African wines ever assembled in one place”.

I’m pretty sure he’s correct. There were, for example, twice as many wines (and I can’t think of any that weren’t good) as there were at the last Platter five-star tasting. And if a tiny asteroid had happened to pick on the Delaire Graff wine cellar – where the event was taking place – it would have disastrously wiped out the best of South Africa’s winemakers as they poured their wines (and participated in the gossip and chat that’s a vital part of the pleasure of occasions like this).

As I wandered around I was guilty of envy. Envy of Tim Atkin. And, though I risk the boringness of raising one of the recurrent, inevitable debates in modern wine commentary, I’ll tell you why. I’d just come through a lengthy process of tasting wines for the Platter guide, according to the curiously hybrid Platter methodology: first, all the wines tasted at home, at leisure and with access to all possible information about the wines; then the wines put forward by the team of tasters as five-star nominations (plus those scoring close to the magic 95-point mark) tasted blind by panels of three. 

What I’ve envious of is Tim Atkin making his judgement entirely on a sighted basis, without the lottery element that is inevitably part of a biggish line-up of wines judged blind and quickly (though, as I’ve said before, the methodology for the Platter final tasting is about as good as it can be). I suppose another enviable part of Tim’s report is that it expresses the aesthetic and skill of a single taster, whereas as the Platter team inevitably have differences of preference and ability that get expressed in their judgements.

I’m not permitted to say anything about this year’s work for Platter until the Guide is published in November, but the basis for my envy of Tim’s sighted-tasting protocol is apparent in previous Platter results. It’s not so much the sins of commission that have worried me in Platter five-star listings, as mostly the awards go to sufficiently plausible wines. It’s the sins of omission. This problem is partly signalled by the comparatively small number of 95+ pointers in Platter, compared to the Atkin Report: my worry is the great wines that slip away through the blind panel tasting. I’ll cite just one example of many in the last three years of Platter scores: for the splendid range of Alheit white wines, in the 2016 Guide – no five stars; 2017 – two; 2018 – one.

It’s not unreasonable, in my opinion, to question the procedures of a guide with that sort of hole in it. But such errors – if you admit that’s what they are – are inevitable in biggish blind line-ups, I fear, however hard one tries to avoid the problem.

I thought it would be interesting, in this light, to compare the results in Tim’s Report with those in the 2018 International Wine Challenge, of which he is co-chairman – and which he claims to be “the world’s most rigorously judged blind tasting competition”. Despite vintage differences, I managed to find comparisons to justify my contention – and my envy of Tim for his “sighted tasting” protocol for his Report.

Certainly there are some positive correlations: the four South African IWC trophy winners did rate 94 or 95 in the Atkin Report (by Tim’s subsequent sighted judgement). But, as I say, it’s the holes that are the telling thing. And I noted a few KWV winners of mere bronze medals at the IWC scoring 95 in the Report: Perold Tributum 2014 and Mentors Petit Verdot 2016. There are quite a few with one medal-placing difference, up or down (entrants achieving less than “Commended” status are not named; and as no actual scores are given for the IWC one can’t see precisely how significant the differences were).  Also obviously relevant, and drastically reducing the scope of comparison, the overwhelming majority of South Africa’s top producers don’t enter the IWC.

There might well be other problems in the way Tim judges wines for his Report – I suspect the wines are not all judged in the same way, for example, with some tasted at leisure with the producers, others rapidly in big line-ups – but it’s an impressive document with a thoughtful introduction, and for what it’s worth I agree with his judgements probably more often than not (when I have tasted the wine, or the same vintage, which is far from all the time). My envy remains, that he can take his sighted judgements all the way through. The report is available for GBP 20/ZAR 395 from Tim’s website

  • Tim James is one of South Africa’s leading wine commentators, contributing to various local and international wine publications. He is a taster (and associate editor) for Platter’s. His book Wines of South Africa – Tradition and Revolution appeared in 2013.


3 comment(s)

Please read our Comments Policy here.

    Stewart Prentice | 10 September 2022

    Surely this piece in FT today shows the inherent folly and almost utterly subjective nature of blind tastings.

    Blind tasting is a game, and rarely a fair one

      Mike Froud, Top Wine SA | 10 September 2022

      Stewart, Jancis Robinson’s lovely piece is about identifying wines, whereas Tim James is talking about rating them. Robinson is a staunch supporter of tasting ‘blind’ in order to score wine properly, objectively, but guessing what type it is or where it comes from is just a bit of fun.

    Tim Atkin | 28 September 2018

    Thanks, Tim, for a very fair and largely positive article. It was good to see you and so many of South Africa’s best wine writers at my tasting. You were all welcome.

    There is indeed a fair degree of correlation between the IWC results and my report, as well as a few discrepancies. I don’t judge all of the wines at the IWC. I’m a co-chair, not the chairman. But I reckon we get things right most of the time. Blind tastings always throw up anomalies and discrepancies as you say, but every wine at the IWC is judged a minimum of two times.

    Just two things I’d like to correct, if I may. It’s true that I do taste most of the wines in my report with producers, but anyone who has seen my programme would disagree that it’s “at leisure”. I put in very long days when I ‘m in South Africa. I’m not saying it’s not enjoyable, but it’s no cake walk.

    Ideally, I’d like to taste all the wines in my report with producers, but this isn’t possible given the numbers involved. This year, 285 wines (out of 1,986 in my report) weren’t tasted with producers (either at their wineries or in group tastings); instead, they were assessed over three days at Spier in what I call my Discovery Tasting. So that’s fewer than 100 a day. So to say that these were tasted “rapidly in big line-ups” is untrue. I take these tastings very seriously, as they always yield some interesting new finds.

Leave a Reply

Your email address will not be published. Required fields are marked *

Like our content?

Show your support.