« Plato, Ibn Rushd, Eugenics, and Being an (Accidental) Straussian | Main | On Gardening, Communism, Eugenics and Competitive Emulation in Thomas More's Utopia. »

10/05/2018

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

D. Des Chene

As an exercise in statistical reasoning, I don’t think it proves much. On the other hand, as propaganda given a scientific sheen it will no doubt work well enough.

The sample size is very small (no doubt because it’s genuinely laborious to write even fake papers). Moreover, one crucial piece of information is missing: the background acceptance rates of the journals in question. If the Journal of Disingenuous Scholars accepts everything that comes over the transom, then it’s not surprising that one of the “grievance studies” papers was published.

To show in a statistically responsible manner that pandering has a significant effect it is not sufficient to show that papers that pander get accepted. You have to show that the acceptance rate is significantly higher than the rate one would expect given background acceptance rates. Otherwise all you have provided evidence for is lax reviewing.

One would like to know that basis of the following claim: “We improved this ratio [of reaching the peer-review stage] from 0% at first to 94.4% after a few months of experimenting with much more hoaxish papers”. That last percentage is approximately equivalent to getting 19 of every 20 papers accepted. How you get that number from a sample well under twenty is a mystery.

Furthermore: notice that the acceptance rate at the outset was 0%. Presumably the first papers they wrote pandered as much as the later papers. Only the obviousness of their being hoaxes changed. Isn’t it possible that the increase in success rate owes as much to “academic virtue signalling” as to pandering? After all, that may well have been the only quality that varied through the trials. Social constructionsm would be irrelevant.

One might suppose that the papers are such that in a best of all worlds at most one paper would have been accepted; that seven (or 10, as they estimate) were accepted is evidence that we do not live in the best world. If, on the other hand, the noise level is high enough, then again it has not been proved that pandering has an effect, but only that the reviewing process isn’t filtering out enough noise.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)

Here's a link to my past blogging (and discussions involving me) at: New APPS.

Categories

Blog powered by Typepad