Tim James: Yet another observation on sighted/blind tasting

Talking of Tim Atkin and his high scores (we were, weren’t we?) … I had the privilege of attending his tasting of 158 (of the 160) wines he scored at least 95 points for his 2018 Special Report. His preface to the tasting booklet described it as “the most amazing line up of South African wines ever assembled in one place”.

I’m pretty sure he’s correct. There were, for example, twice as many wines (and I can’t think of any that weren’t good) as there were at the last Platter five-star tasting. And if a tiny asteroid had happened to pick on the Delaire Graff wine cellar – where the event was taking place – it would have disastrously wiped out the best of South Africa’s winemakers as they poured their wines (and participated in the gossip and chat that’s a vital part of the pleasure of occasions like this).

As I wandered around I was guilty of envy. Envy of Tim Atkin. And, though I risk the boringness of raising one of the recurrent, inevitable debates in modern wine commentary, I’ll tell you why. I’d just come through a lengthy process of tasting wines for the Platter guide, according to the curiously hybrid Platter methodology: first, all the wines tasted at home, at leisure and with access to all possible information about the wines; then the wines put forward by the team of tasters as five-star nominations (plus those scoring close to the magic 95-point mark) tasted blind by panels of three. 

What I’ve envious of is Tim Atkin making his judgement entirely on a sighted basis, without the lottery element that is inevitably part of a biggish line-up of wines judged blind and quickly (though, as I’ve said before, the methodology for the Platter final tasting is about as good as it can be). I suppose another enviable part of Tim’s report is that it expresses the aesthetic and skill of a single taster, whereas as the Platter team inevitably have differences of preference and ability that get expressed in their judgements.

I’m not permitted to say anything about this year’s work for Platter until the Guide is published in November, but the basis for my envy of Tim’s sighted-tasting protocol is apparent in previous Platter results. It’s not so much the sins of commission that have worried me in Platter five-star listings, as mostly the awards go to sufficiently plausible wines. It’s the sins of omission. This problem is partly signalled by the comparatively small number of 95+ pointers in Platter, compared to the Atkin Report: my worry is the great wines that slip away through the blind panel tasting. I’ll cite just one example of many in the last three years of Platter scores: for the splendid range of Alheit white wines, in the 2016 Guide – no five stars; 2017 – two; 2018 – one.

It’s not unreasonable, in my opinion, to question the procedures of a guide with that sort of hole in it. But such errors – if you admit that’s what they are – are inevitable in biggish blind line-ups, I fear, however hard one tries to avoid the problem.

I thought it would be interesting, in this light, to compare the results in Tim’s Report with those in the 2018 International Wine Challenge, of which he is co-chairman – and which he claims to be “the world’s most rigorously judged blind tasting competition”. Despite vintage differences, I managed to find comparisons to justify my contention – and my envy of Tim for his “sighted tasting” protocol for his Report.

Certainly there are some positive correlations: the four South African IWC trophy winners did rate 94 or 95 in the Atkin Report (by Tim’s subsequent sighted judgement). But, as I say, it’s the holes that are the telling thing. And I noted a few KWV winners of mere bronze medals at the IWC scoring 95 in the Report: Perold Tributum 2014 and Mentors Petit Verdot 2016. There are quite a few with one medal-placing difference, up or down (entrants achieving less than “Commended” status are not named; and as no actual scores are given for the IWC one can’t see precisely how significant the differences were).  Also obviously relevant, and drastically reducing the scope of comparison, the overwhelming majority of South Africa’s top producers don’t enter the IWC.

There might well be other problems in the way Tim judges wines for his Report – I suspect the wines are not all judged in the same way, for example, with some tasted at leisure with the producers, others rapidly in big line-ups – but it’s an impressive document with a thoughtful introduction, and for what it’s worth I agree with his judgements probably more often than not (when I have tasted the wine, or the same vintage, which is far from all the time). My envy remains, that he can take his sighted judgements all the way through. The report is available for GBP 20/ZAR 395 from Tim’s website. 

• Tim James is one of South Africa’s leading wine commentators, contributing to various local and international wine publications. He is a taster (and associate editor) for Platter’s. His book Wines of South Africa – Tradition and Revolution appeared in 2013.