This piece on the history of peer review at the Royal Society and the problem of unconscious bias originally appeared on the RS Publishing Blog, 10 Sept. 2018, as part of Peer Review Week 2018.
Peer review cannot be done by everyone. It can only be done by people who share certain levels of training and subject-expertise, and have a shared sense of what rigorous experimentation, observation and analysis should look like. That shared expertise and understanding is what should enable alert peer reviewers to reject shoddy experimental methods, flawed analysis and plans for perpetual motion machines.
But as we have increasingly come to realise, any group of people with shared characteristics may display unconscious bias against outsiders, whether that means women, ethnic minorities, or those with unusual methods. While peer review should exclude poor science, it should not exclude good research on the basis of the individual traits or institutional affiliation of the researchers, nor should it dismiss innovative approaches to old problems.
However, it seems socio-cultural and intellectual criteria have often been mixed together in the peer review process, and history can help us to understand why.