> It also leads to good research being improved, in a very similar way to how code review can improve good code.
But the parallel breaks, as unlike with code reviews, there is no protection against (and the people below are real people I have encountered in my 10+ year research experience):
* A reviewer writing eloquent reviews on every single paper they ever reviewed, explaining why they should be rejected, as a gimmick at this point.
* Reviewers not taking time constraints into consideration. Doing this 2 year experiment would indeed make for a better paper.
* Reviewers not reviewing in a timely manner. Yes, you said you volunteered to review this. It has been 9 months since.
* Reviewers requesting you to cite their papers, however unrelated.
* I quote a full review I once had: "This paper needs more equations."
When code reviewing in a team, these issues get sorted out. In academia, peer reviewing is in a state of complete anarchy.
I agree, those are all existing problems. But I would say the don't detract from the benefits.
Maybe peer review should be time limited, but have the reviewers comments be immediately available for the authors to respond to, rather than having to wait for them all to come in.
But the parallel breaks, as unlike with code reviews, there is no protection against (and the people below are real people I have encountered in my 10+ year research experience):
* A reviewer writing eloquent reviews on every single paper they ever reviewed, explaining why they should be rejected, as a gimmick at this point.
* Reviewers not taking time constraints into consideration. Doing this 2 year experiment would indeed make for a better paper.
* Reviewers not reviewing in a timely manner. Yes, you said you volunteered to review this. It has been 9 months since.
* Reviewers requesting you to cite their papers, however unrelated.
* I quote a full review I once had: "This paper needs more equations."
When code reviewing in a team, these issues get sorted out. In academia, peer reviewing is in a state of complete anarchy.