On Irresponsible Science Journalism: No, those studies do not show that a market in Wuhan was the source of the Coronavirus

International scientists on Saturday released two major studies which one participant said made it “extraordinarily clear” a market in Wuhan, China was the source of the coronavirus which fueled the Covid-19 pandemic – and not a Chinese government laboratory, a theory championed in the US by rightwing campaigners, columnists and politicians.

The question of where Covid-19 came from and how it spread has proved divisive...

In August last year, a US intelligence review of the issue proved inconclusive.

The New York Times first reported the new studies, which it said had not been published in any journal.--"Coronavirus came from Wuhan market and not Chinese lab, twin studies say"--The Guardian, 26 February, 2022. 

For those who may read my blog for the first time, I am not a rightwing campaigner.  I have been a critic of former President Trump since 2015. I am also not a virologist or epidemiologist, however, I am a philosopher of science and my original interest in the field is evidential arguments. I will suggest below that the claims in The Guardian and in The New York Times are examples of irresponsible and sloppy science journalism. Before I get to explaining why I think that, here is a flavor of The New York Times reporting:

The studies, which together span 150 pages, are a significant salvo in the debate over the beginnings of a pandemic that has killed nearly six million people across the world. The question of whether the outbreak began with a spillover from wildlife sold at the market, a leak from a Wuhan virology lab or some other event has given rise to pitched debates over how best to stop the next pandemic.

“When you look at all of the evidence together, it’s an extraordinarily clear picture that the pandemic started at the Huanan market,” said Michael Worobey, an evolutionary biologist at the University of Arizona and a co-author of both new studies.

Several independent scientists said that the studies, which have not yet been published in a scientific journal, presented a compelling and rigorous new analysis of available data. "New Research Points to Wuhan Market as Pandemic Origin" The New York Time, updated 27 February, 2022

One of the papers with the suggestive title "The Huanan market was the epicenter of SARS-CoV-2 emergence," is the main source of the claims reported by the The Guardian and The New York Times that I will argue are examples of irresponsible and sloppy science journalism. The paper, which I downloaded on February 27, and that I will be discussing can be found here. I heard about the paper through a tweet by Marion Koopmans shared by my colleague John Grin (since unshared after learning of my concerns below). Dr. Koopmans is a leading virologist and public health scientist in the Netherlands. Professor John Grin and I both work in a political science department; he is a former physicist, who works in public policy on issues related to transition economy. I consider both high quality sources of science related material. And so when I decided to read the paper, I assumed I would be convinced by the findings.

Michael Worobey (the lead author of the paper) and his colleagues have dome some impressive modeling and statistical analysis to argue that the source of the spread of the pandemic is probably to be found in the Wuhan market. The paper has, however, a peculiar step, which deserves some scrutiny, and which weakens the claims by the media that this article rules out a lab leak.

As is clear from the suggestive title of the paper and the media reporting I have quoted above, the political significance of this paper is it that it is presented as ruling out that the virus leaked out of the lab of the Wuhan CDC. Since outside access to it has been denied (here is the Guardian's reporting; here's the New York Times), this has generated concerns over a cover up.

What the media reporting fails to remark is that this possibility is addressed in only one sentence shown in a picture below.* 

So, naturally, I was curious to see what source (9) is. It's the Chinese CDC! (See the other picture.)

Now, clearly the authors of the report treat the Chinese CDC as a reliable source, and they may well have good reasons to do so. Both papers rely heavily on data from it, it seems.

But, if the point is to rule out a lab leak (and, hence, a cover up), this line of reasoning will not do at all. For, rather than testing whether the data they do rely on can distinguish empirically between the evidence for the Wuhan market and against a lab leak, the paper rules out a lab leak by fiat on the authority of those whose impartiality is questioned by critics of China. So this paper does not settle the matter at all.

Notice that I am not claiming this is poor science. The paper uses what it calls "spatial analyses" to "show that the earliest known COVID-19 cases diagnosed in December 2019 were geographically distributed near to, and centered on, this market." And this it undeniably does compellingly and with interesting techniques. And so the paper may well be published after proper peer review.

But in the media Michael Worobey's paper is presented as settling the politically salient issue (lab leak vs market). And his own comments seem to contribute to this perception in the context in which they are presented. I assume he consented to this because in my experience science journalists tend to be careful with their scientific sources in checking how they are presented. Yet, for the purposes of the controversy (lab leak vs market), the underlying logic of the paper  is something like this, "given that we assume on authority of an interested party that the lab cannot be the origin, we will study how data on the initial spread relates to the Wuhan market." 

I really hope I have missed something and that I am wrong about this because I do not want to contribute to the undermining of trust in science reporting of quality media. But even leaving aside the part of the story that has made it a central issue in our contemporary culture wars, there are many public health reasons to get clear on what the source of the original problem is. And it is a mistake to consider and irresponsible to claim that it is settled based on these studies. 

 

 

*The sentence occurs on p. 19 of my downloaded copy of the paper. Somewhat unusually the document does not have page-numbers. Reference 9 can be found on p. 65 in the "Supplementary references."

Comments

[David Duffy]

"this possibility is addressed in only one sentence" - the abstract of Worobey et al clearly states the actual argument: "these analyses provide dispositive evidence for the emergence of SARS-CoV-2 via the live wildlife trade and identify the Huanan market as the unambiguous epicenter of the COVID-19 pandemic."

They cannot, I think, exclude hypotheses where the CCDC labels samples where virus was present as coming from the southwestern corner of the market, where one author had seen live raccoon dogs being sold, or that two infected laboratory workers shop for live animals there. The paper has also to be read along with their other paper - Pekar et al, and the Gao et al paper that preceded them, which address the question of where the A and B lineages arose. Also see the N&V article
https://www.nature.com/articles/d41586-022-00584-8

The Guardian article, at least, reads to me as well measured.

[ERIC SCHLIESSER]

David, The Nature piece -- which I had not seen, thank you, -- quite rightly notes: "Some virologists say that the new evidence pointing to the Huanan market doesn’t rule out an alternative hypothesis. Namely, they say that the market could have just been the location of a massive amplifying event, in which an infected person spread the virus to many other people, rather than the place of the original spillover."
As far as I can tell nobody denies that the market is the most likely place that amplified the spread. And while that's super important to establish empirically, it does not speak to the politically salient question whether this originates in a lab leak. This is why I call the reporting irresponsible and misleading, and I stand by that for now.

[David Duffy]

Dear Eric - the distribution of sars-cov-2-positive environmental samples within the market (p15) is what drives the "dispositive evidence", with "five SARS-CoV-2-positive environmental samples [being] taken from a single stall known to be selling live mammals in late 2019...highly suggestive of infected animals having been present at the Huanan market at the beginning of the COVID-19 pandemic...the odds of detection [of a positive environmental sample] decreased by 2.8% per meter away from a live mammal vendor and by 6.6% per meter from a known human case (for N=6, Table S8), indicating that estimated effect on sample positivity due to proximity to live mammal vendors was farther-reaching than that of stalls with human cases".

The alternative hypothesis that the virus entered the market via a human carrier has to explain this particular intra-market pattern. Maybe everyone who shops there goes to see the animals, and the state of the cages etc helps the virus survive on surfaces. The dataset is not exactly ginormous, so I don't regard this as knock-down conclusive.

[ERIC SCHLIESSER]

Dear David, look, first this is a classic case of underdetermination.

Second, the paper exhibits a tendency to treat confirmatory evidence as "dispositive," while treating lack of evidence as defeasible. Let me offer two examples: (I), as it notes when the Wuhan market was closed down "Some 457 samples from 188 individual animals corresponding to 18 mammals species were screened for active SARSr-CoVs infection via qRT-PCR from “within and outside Huanan Market”, and no positive SARS-CoV-2 samples were identified." (p13) It does not explain why this is a problem for the argument, or why we should set it aside. Rather, it goes on in the same paragraph to conflate this testing with a much broader (and less pertinent) testing program (which also did not provide positive evidence) and explains (quite plausibly) with it can be ignored. Again, as the Nature piece notes (and I am grateful for you to call my attention to it), "none of the studies contains definitive evidence about what type of animal might have harboured the virus before it spread to humans." That's the core of the evidential problem, and this is why it is so odd that on the authority of a stake-holder (the Chinse CDC--to which some of the authors have disclosed ties), the lab-leak is not even tested as a competing hypothesis or a null hypothesis (which with their spatial analysis they could do).

(II) The paper has a tendency to discount what it calls "putative index cases" unrelated to the Hunan market, (p.3) while acknowledging that all the cases (including the ones connected to Wuhan market) are unlikely to be the first one. I don't see how that avoids the perception of cherry-picking.

So, yes, I think this paper leaves no doubt that the market was the source of initial spread. I think it's plausible that there was a spillover event there. But as far as I can see only the Nature article reports the grounds for caution.

[Lorna Salzman]

In Viral (Chan and Ridley) and elsewhere there are firm statements indicating that despite tests of thousands of animals at the market, not one showed any signs of a virus. Has any contrary evidence appeared? Saying the virus spread from the market is not the same as saying that it ORIGINATED there. How can scientists do a study without stating clearly that a market
visitor (a lab worker or housewife or shopper) could have brought it there originally.? If I were editor of a peer reviewed journal, I would insist on this being included in the study before publishing it. Why didnt they?