Sister blog of Physicists of the Caribbean. Shorter, more focused posts specialising in astronomy and data visualisation.

Thursday 15 October 2015

Open reviewing isn't that much better than normal reviewing

I'd be wary of disclosing the identity of reviewers. There's a risk that they won't want to be seen as supporting unconventional research, thereby establishing a false consensus. However I do think that publishing the author-reviewer correspondence as well as the research itself might be a good idea.

I'm surprised that the difference in quality is so low (5% better using the alternative method), but I haven't read the original report yet.

https://www.timeshighereducation.com/news/open-peer-review-better-quality-than-traditional-process
at
Labels:

5 comments:

  1. David Lazarus15 October 2015 at 16:07

    I'm proud to be a non-mainstream scientist by education and philosophy.

    ReplyDelete
  2. Jordan Henderson15 October 2015 at 16:24

    Then there's this amazing story of the Japanese Mathematician who just quietly put his (possibly) miraculous proofs on his website for public review.

    http://www.scientificamerican.com/article/math-mystery-shinichi-mochizuki-and-the-impenetrable-proof/

    It appears that he didn't submit them for peer review because he recognized that nobody would be able to review them in any reasonable time frame.  It's been 3 years and the other experts are still mystified.

    ReplyDelete
  3. David Lazarus15 October 2015 at 16:54

    Jordan Henderson​ - Excellent article.

    ReplyDelete
  4. Rhys Taylor15 October 2015 at 17:19

    David Lazarus We definitely need people working on non-mainstream ideas... just not me. :)

    ReplyDelete
  5. Rhys Taylor15 October 2015 at 20:48

    So I glanced at the paper, and the open peer review journal also publishes the referee reports and author correspondence. Looks like it doesn't make that much difference to the quality of the reports. Still, ideologically I think it's a good idea, so that the general public will know just how carefully (or not) the referees are being.

    It could also be there's a difference in review standards from subject to subject; this study only looks at biology and medicine.

    ReplyDelete

Subscribe to: Post Comments (Atom)

Back from the grave ?

I'd thought that the controversy over NGC 1052-DF2 and DF4 was at least partly settled by now, but this paper would have you believe ot...

  • Death of the Flying Snakes
    In the last batch of simulations, we dropped a long gas stream into the gravitational potential of a cluster to see if it would get torn...
  • The Return of the Failures
    Another day, another paper on how exciting Ultra Diffuse Galaxies are.  At first, these large, faint galaxies were just wholly unexpected, a...
  • Proving a negative is perfectly possible
    Of course you can prove a negative. In one sense this can be the easiest thing in the world : your theory predicts something which doesn...