1936: LNG Filon on the importance of journal reputation

“Research qualifications are now more and more insisted upon for appointments to academic and other posts, and appointing bodies have often no means of discriminating between important and trivial research, except the particular medium of publication. The publications of the Society have always been recognized as of exceptionally high standard, and special significance has been attached to papers published in them. Should such discrimination between publications become obsolete or even weakened, a spate of trivial papers may easily outweigh, in the minds of lay persons, a few really valuable contributions, with results ultimately detrimental to the best interests of Science.”

So wrote mathematician (and fellow of the Royal Society) Louis Filon, in the summer of 1936.

Closing paragraph of LNG Filon’s memo to the Royal Society, RS CMP/14 9 July 1936

The Royal Society’s Publication Committee had just recommended a series of changes to the arrangements for improving the physical quality of the Society’s journals (including moving to the University Printers at Cambridge), but Filon was expressing  concern about the intellectual quality of the journals. Long before the impact factor or metrics, Filon noted how useful it was for grant and appointment panels to be able to use the reputation of a journal as a proxy for the quality of its contents.

As a former vice-chancellor of University College London, Filon was well-versed in academic politics. He would have chaired plenty of committees scrutinising the CVs and publication lists of hopeful academics, and heard the arguments between academics (most of whom would not be specialists in the candidate’s field) as they tried to distinguish quality from “a spate of trivial papers”.

The context of academic politics and prestige informed the memorandum Filon sent to the Royal Society (considered at a meeting on July 9, 1936). The real point of the memo is his concern that papers published in Society’s Proceedings were no longer reliably high quality (or, as Filon puts it, “of critical importance”).

He blamed the growing amount of “routine research” produced by the growing numbers of postgraduate and postdoctoral researchers, i.e. “young and comparatively untrained men”. According to Filon, much of this work was of “secondary importance”, and would not formerly (before the Great War) have been “either offered or accepted” by the Royal Society.

Filon felt that too much of this apparently “routine research” was now being published by the Royal Society. This was increasing the bulk and cost of the publications, and also risked damaging the reputation of the Society’s journals, and hence their usefulness as markers of quality for appointment panels.

The editorial processes of the Society’s journals at this time relied entirely on the fellows of the Society. Papers had to be submitted via a fellow (a ‘communicator’), and would be refereed by one or more fellows. Filon felt that fellows who were laboratory heads were finding it “difficult to refuse the request by a student or a member of his staff to submit a paper”, and were thus failing in their duty to the Society as communicators.

He also worried that fellows acting as referees might “not unnaturally, hesitate to recommend the rejection” of a paper “vouched for by a Fellow of some reputation”.

The Society’s council presumably agreed with at least some of Filon’s concerns, for they sent a reminder to all fellows about the duties of communicators and referees, and in 1937, revised the guidance on these matters in the Society’s standing orders.

It should be noted that Filon never claimed that the allegedly “routine research” of early-career researchers should not be published. But he felt that work that was “sound so far as it goes”, or involved “the accumulation of data” or “the elaboration of minor details” should be published elsewhere, not by the Royal Society.

In the age of print and paper, this was not an unreasonable stance: the Society could not afford to publish all the “sound” work that was being produced. In the digital age, things are different, and in 2014, the Society launched Royal Society Open Science, which uses objective peer-review” and aims to publish “all articles which are scientifically sound and useful to the community”.

Filon’s 1936 memo is one of the earliest pieces of evidence we have yet found linking editorial processes (including, but not only, the role of referees) explicitly with  intellectual quality. (For an extended discussion of the roles of refereeing at the Royal Society, see Moxham and Fyfe 2017)

It is also the earliest evidence we have yet found of a recognition that journal brand or reputation was being used as a proxy for the quality of the papers published in it. (The Journal Impact Factor was not launched until 1975, see Archambault and Lariviere 2009).

Historical source reference: Royal Society Council Minutes, vol. 14, 9 July 1936.

 

Who are the referees?

As a rule, referees had to be Fellows. Only in 1990 (earlier?) did the Society officially allow non-Fellows to referee papers. [If there are earlier examples of non-Fellows could add here.] This meant that there was a finite number of available referees. The number of Fellows changed over time but in the mid nineteenth century, after reforms to reduce the size of the Fellowship, the number of Fellows was around 400 to 500. In reality, a very small proportion of the Fellowship was involved in refereeing. Those who were most active were typically past, current or future Council members. In other words, fellows who were active in one area of the Society’s service tended to be active in other areas too. During his Secretaryship, for example, George Gabriel Stokes was the most active referee, followed by his co-secretary William Sharpey. The Society’s Fellowship in the nineteenth century, when the number of practitioners in science in Britain was growing, represented only an elite group of the scientific community. At the same time, the majority of the papers submitted were from Fellows. Refereeing was thus about judging one’s peers, rather than making judgements on outsiders. Overtime this changed, and increasingly in the early twentieth century more non-Fellows submitted papers to the Society. Refereeing was still conducted by Fellows only but now many authors were not known personally to the Society’s Officers and Council. The Fellowship had grown, and now more Fellows were involved in refereeing: in the 1880s, 8% of the Fellowship were involved in refereeing; this rose to 30% by the 1930s. The workload, however, became even more uneven. The top most active 20% of referees produced just under 40% of referee reports in the nineteenth century, while the same % group in the mid-twentieth century produced 50% of referee reports. Getting a paper through the refereeing process at the Society signified acceptance by representatives of an elite national learned body of judges

1925: Printed referee report forms (used since the 1890s)

In the 1890s, the Royal Society had introduced a set of 7 questions for referees, in the hope of structuring the reports (which were sometimes extremely long-winded!). These were originally hand-written into the covering letter, but were quickly turned into a printed standardised report form, sent to each referee with the manuscript to be evaluated. Referees were encouraged to return their reports within 14 days – a deadline that was routinely breached.

By the early twentieth century, these report forms included clear instructions for referees, including advising them of the confidentiality attached to the papers referred to them (see image). It was routine for the author’s name to be written on the form: refereeing was single-blind, not double-blind. At this time referees were always Fellows of the Society (and their names and reports were kept confidential), but the majority of papers came from those outside the Fellowship.

The report forms made it possible for a referee to present an extremely succinct report, as was the case with Professor H. Lamb’s report on this 1925 paper by ‘Mrs. H. Ayrton’. Hertha Ayrton’s work in electrical engineering had previously been published with and exibited to the Society, but her status as a married woman had prevented the Royal Society accepting a fellowship nomination certificate in her name in 1902. (Her husband was also a well-known electrical engineer, and Fellow of the Royal Society, William Ayrton.)

The printed forms were also an attempt to standardize the refereeing process, or to at least advise referees on how to write an effective report. The Society never officially instructed referees until this date; referees were automatically expected to know how to write a report. Guidance on this continued to develop. By 1926, ‘Instructions to Referees’ was part of the Society’s Standing Orders.

Source: Box RR, 1925-1926, Royal Society Archives, London.

Printed referee report form (used since the 1890s)

In the 1890s, the Royal Society had introduced a set of 7 questions for referees, in the hope of structuring the reports (which were sometimes extremely long-winded!). These were originally hand-written into the covering letter, but were quickly turned into a printed standardised report form, sent to each referee with the manuscript to be evaluated. Referees were encouraged to return their reports within 14 days – a deadline that was routinely breached.

By the early twentieth century, these report forms included clear instructions for referees, including advising them of the confidentiality attached to the papers referred to them (see image). It was routine for the author’s name to be written on the form: refereeing was single-blind, not double-blind. At this time referees were always Fellows of the Society (and their names and reports were kept confidential), but the majority of papers came from those outside the Fellowship.

The report forms made it possible for a referee to present an extremely succinct report, as was the case with Professor H. Lamb’s report on this 1925 paper by ‘Mrs. H. Ayrton’. Hertha Ayrton’s work in electrical engineering had previously been published with and exibited to the Society, but her status as a married woman had prevented the Royal Society accepting a fellowship nomination certificate in her name in 1902. (Her husband was also a well-known electrical engineer, and Fellow of the Royal Society, William Ayrton.)

The printed forms were also an attempt to standardize the refereeing process, or to at least advise referees on how to write an effective report. The Society never officially instructed referees until this date; referees were automatically expected to know how to write a report. Guidance on this continued to develop. By 1926, ‘Instructions to Referees’ was part of the Society’s Standing Orders.

 

1925-1926: Standardised letter to referees
1925-1926: Report on Ayrton
Ayrton report by H. Lamb

1951: Rosalind Franklin at the Royal Society

Rosalind Elsie Franklin (1920-1958) is becoming more known for her contribution to X-ray crystallography and the discovery of the DNA double helix. Brenda Maddox’ excellent biography, ‘The Dark Lady of DNA’ is key reading for anyone interested in Franklin, women in science, or the DNA-discovery saga. But because Franklin was never made an FRS, her times at the Royal Society have often been overlooked. In fact, she both visited, spoke and published with the Royal Society, as this example of a 1950s referee report shows. Note the questions referees are asked, and that the referees in question are JD Bernal and Dorothy Hodgkin, both huge names in the field by 1951. Franklin’s paper was well received by both, as you can see, and published by the Royal Society. Today, a photograph of a young, smiling Franklin hangs to the right of the main staircase when you walk into the Royal Society. Despite her lack of FRS status, her work was recognized by the Society in the fifties; – and today through the Rosalind Franklin award and lecture.

JD Bernal writes in support as a referee. Click for a larger version.
Dorothy Hodgkin writes in support as a referee. Click for a larger version.