by Andrew Chalk
In wine-searcher on June 6th, Kathleen Wilcox claims that critics score organic wines higher, all else equal, than conventionally-farmed wine. She doesn’t perform the research. Rather, referring to a study by Magali A. Delmas and Olivier Gergaud.
She either did not read, or did not understand, that study as, unfortunately, it is fatally flawed. Not blemished, but faulty to the point that none of its conclusions regarding wine quality and farming practice are sustained. And, had the article been submitted to the mainstream academic economics literature (rather than the journal Ecological Economics), I doubt it would have withstood the scrutiny of the independent referee.
THE FLAWED STUDY
In their study, Delmas and Gergaud attempt to explain critics’ (e.g Wine Advocate, Wine Spectator) scores of a wine as the result of multiple factors. They use the standard statistical technique to do this, known as multiple regression. They have a massive sample of over 128,000 wines. So far, so good. Unfortunately, they are missing the price for all the wines. As a result, they cannot tell if higher scores for organic wines are a result of them being organic, or simply being more highly priced. All their efforts may simply be showing that more expensive wines get higher scores -- something we already knew. Indeed, the way that their study is constructed, if the price of all organic wines were doubled, it would not affect their scores! Simply because price is omitted from the study's regressions.
I spoke to one of the authors about this and he did not dispute my basic point. I asked for their data in order to try to add price data (to at least a subset) in order to get meaningful results. They declined to make it available (an unusual thing in modern academic research where peer review is critical and widespread).
The editor of wine-searcher should have insisted Wilcox conduct a serious review of the literature before naively believing the first study she found that matched her prejudices. Had she Googled “does organic agriculture lead to higher wine scores” she would have found that the fourth result (out of the 13.3 million returned) was the piece where I originally analyzed the Delmas and Geraud paper (and stated the above in more detail).
The editor could also have corrected her claiming “multiple studies” supported the Delmas and Gergaud result, when she only cited two -- and the other one was by the same authors! In fact, it is essentially the same (flawed) study, on a different data set.
There definitely needs to be research into the relationship between agricultural technique, wine quality, and wine price, but this kind of research and its uncritical acceptance by writers out of their depth, is not helping.