Ross Mounce

This is just a quick post of appreciation for [PCI Registered
Reports](https://rr.peercommunityin.org/). I've recently joined the PCI
RR community as a
'[recommender](https://rr.peercommunityin.org/about/recommenders)'.

<figure class="wp-block-image size-full">
<a
href="https://rossmounce.co.uk/wp-content/uploads/2023/11/image.png"><img
src="https://rossmounce.co.uk/wp-content/uploads/2023/11/image.png"
class="wp-image-2279" loading="lazy" decoding="async" width="259"
height="194" /></a>
</figure>

  
One thing that spurred me to join is a rather unsatisfactory experience
I had as a peer-reviewer, reviewing a manuscript where the experimental
design was deeply insufficient. I won't reveal for which open access
journal I reviewed, or the full content of my peer review report but the
gist of it was (quoting from my report, my words):

> *"...this \[name of OA journal redacted\] manuscript has an inadequate
> sample size to draw conclusions from. Sampling only 152 journals out
> of a total population of \~4594 SCIE-indexed biology journals, equates
> to just a 3.3% sample size (152/4594). I see no reason why the authors
> couldn't have taken at least a 10% sample (over 459 journals). Prior
> work on open access citation advantage such as Ghane et al (2020) did
> their work on a sample of 1881 journals indexed in DOAJ (DOAJ at the
> time indexed a total of \~11,000 journals; Ghane et al (2020) achieved
> a greater than 10% sample size).*
>
> *Furthermore, the choice of how which journals were sampled was by
> their own account in a non-random and arbitrary manner. Students
> merely \_chose\_ a journal to download data on. This is highly
> subjective and prone to bias. As an example of one possible bias
> introduced: if I were a student, I might be tempted to choose a
> journal with a low article volume so that it would be less work/time
> to complete the assignment..."*
>
> Ross Mounce (2023)

TL;DR the experimental design was deeply flawed. Although I appreciate
the authors put a lot of work into it, the conclusions of their work
were just not valid, given the problems of their experimental design.

The experience as a reviewer was like arriving at the scene of a car
crash six months too late to stop the accident. To change this piece of
work into one with an acceptable experimental design at this point would
be a LOT of new work and might require throwing away much of the
existing data.

It's clear to me that a lot of researchers are now trying their hand at
metaresearch, research on research, journalology, and open access
studies --- whatever you call it. That's great. But I implore folks to
seriously consider getting their experimental design peer-reviewed
BEFORE they start their study, and *PCI Registered Reports* is one of
the communities that will provide that service for you, for free!

I would love to see more stage 1 registered reports in metaresearch, to
catch improperly designed studies before they get carried out,
written-up, and submitted. Reducing research waste AND increasing the
quality of research that ends-up getting published --- what's not to
like? It might save you a lot of anguish.


This is just a quick post of appreciation for PCI Registered Reports. I’ve recently joined the PCI RR community as a ‘recommender’.  One thing that spurred me to join is a rather unsatisfactory experience I had as a peer-reviewer, reviewing a manuscript where the experimental design was deeply insufficient.

Messaggi di Rogue Scholar

Kudos to PCI Registered Reports