« Plato, Ibn Rushd, Eugenics, and Being an (Accidental) Straussian | Main | On Gardening, Communism, Eugenics and Competitive Emulation in Thomas More's Utopia. »

10/05/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

D. Des Chene

As an exercise in statistical reasoning, I don’t think it proves much. On the other hand, as propaganda given a scientific sheen it will no doubt work well enough.

The sample size is very small (no doubt because it’s genuinely laborious to write even fake papers). Moreover, one crucial piece of information is missing: the background acceptance rates of the journals in question. If the Journal of Disingenuous Scholars accepts everything that comes over the transom, then it’s not surprising that one of the “grievance studies” papers was published.

To show in a statistically responsible manner that pandering has a significant effect it is not sufficient to show that papers that pander get accepted. You have to show that the acceptance rate is significantly higher than the rate one would expect given background acceptance rates. Otherwise all you have provided evidence for is lax reviewing.

One would like to know that basis of the following claim: “We improved this ratio [of reaching the peer-review stage] from 0% at first to 94.4% after a few months of experimenting with much more hoaxish papers”. That last percentage is approximately equivalent to getting 19 of every 20 papers accepted. How you get that number from a sample well under twenty is a mystery.

Furthermore: notice that the acceptance rate at the outset was 0%. Presumably the first papers they wrote pandered as much as the later papers. Only the obviousness of their being hoaxes changed. Isn’t it possible that the increase in success rate owes as much to “academic virtue signalling” as to pandering? After all, that may well have been the only quality that varied through the trials. Social constructionsm would be irrelevant.

One might suppose that the papers are such that in a best of all worlds at most one paper would have been accepted; that seven (or 10, as they estimate) were accepted is evidence that we do not live in the best world. If, on the other hand, the noise level is high enough, then again it has not been proved that pandering has an effect, but only that the reviewing process isn’t filtering out enough noise.

The comments to this entry are closed.

Here's a link to my past blogging (and discussions involving me) at: New APPS.

Categories

Blog powered by Typepad