Viewpoint

The Worship of Peer Review

Peter Meredith

Scientists write, that being an essential part of the gaining or rehashing of knowledge. The work is no use until it reaches an audience, at which time it becomes part of the body of knowledge. At times we may wonder how reliable that knowledge is. The old philosophy had it that nothing was "proved" until the experimental observations had been repeated by another person in another place at another time with essentially the same "result".

That result, of course, was dependent not only on the observations but on their interpretation, depending on what the thinker's mind could conceive of for possible explanations. All too often, on reading work of 50 or 100 years ago, we can now see an alternative explanation of the observations that was not conceivable at that time. The observations were then explained by choosing the most attractive fit between hypotheses A, B and C; it is now obvious to us that hypothesis D fits better. And no doubt in another 50 years we would choose hypothesis E that we still can't conceive of.

Thus new knowledge has two parts: the observations and the interpretation. To give future thinkers the best chance, the circumstances surrounding the observations must be precisely described, and in manner that is not too culture-dependent. We currently have problems with perceptions of everyday circumstances described contrastingly on the two sides of the Atlantic Ocean, such as in "the crops were sprayed with the active compound bi-weekly". That editor couldn't see any problem, but this reader could and the Oxford Dictionary recommended avoiding the ambiguity. There's lots of this sort of thing in print. Peer review doesn't seem to remove it.

The Paper

That example, like most "papers" in journals, had been through peer review. I have avoided or manipulated peer-reviewed journals for many years. Peer review is a very time-consuming process that perhaps achieves some of three objects: it saves the editor having to be much of an expert; it saves the educators having to worry so much about training jejeune students to communicate well; and it supports the cosy coterie of established knowledge that dislikes being questioned. We may think of the "old boys' network", "the alumni", the "trades union of those with appropriate background". A closed shop of the mind.

This thought-conforming process is unnecessarily wasteful. There are journals in this world with fully professional technical editors. Send a contribution to them and the probability is of a fast response, accepted or rejected, with only an occasional referral about specific points. Quick and clean, but requiring competence of both writer and editor.

My experience is that observations are rarely false but that interpretations often contain muddled thinking, while description of the obtaining of the observations leaves much to be desired, consisting all too often of a ritual listing of who supplied the chemicals and references to someone else's analytical method without even stating the principle of that method. The principles behind the methodology can have a big influence on the final interpretation of results, as we realise on hearing heated "discussions" at meetings of professional bodies.

At present, to conform, one is expected to publish "papers" that are peer reviewed. In so doing, one also reviews the literature of the past and makes judgements about those "papers". The non-peer-reviewed publications are also considered in these introductions to the present work, with equal weight. We do not discriminate there except in our choice of what to include and what to exclude. We slant the choice to build our case for our current hypothesis -- let's be honest about it, it's like being an adversarial lawyer.

The Chapter

Within the profession of "science" there are higher levels of writing to aspire to. Just think of those around you who have written "a chapter" in a book. That certainly will have been dealt with by a technically competent editor and will rarely have been peer reviewed. If peer review is such a wonderful thing, why have we suddenly abandoned it at this level? I suggest it's because we are properly using a competent editor.

I'm not suggesting that journal editors are not technically competent, but that they are being misused as sorting offices for shuffling manuscripts to and from reviewers. We have developed a system that has outlived its purpose or has been captured by administrators playing the numbers game. They are either counting (reviewed) papers or, even worse, numbers of pages in those papers. And now look at the articles we get that "statistically" rank journals for "quality" and then make numerical assessments of the "quality" of the science of one country or organisation or discipline versus another.

Publish or perish was heaven sent for the bean counters. Go to the annual meeting of a large international scientific society, and listen to the editor-in-chief spouting statistics about numbers of manuscripts, numbers of pages per volume and, almost always, "what are we going to do about the poor quality of a proportion of submitted manuscripts?". Get tough, of course. Throw the problem back to academe.

The Book

The peak of publishability is to have written "a book". The editor often isn't technical in the science sense but is engaged to produce a book that will sell. That's what book publishing is about. So you can get away with murder in your book and no-one will question if the statements of "fact" should have been peer-reviewed.

Every journalist knows that a good sub-editor or editor can vastly improve anything that has been written, though in technical subjects there is a danger of over-improvement leading to loss of technical content or loss of flavour. A scientist inherently has been trained to say things once and precisely, whereas the graduate in arts/literary/history has been taught to say substantially the same thing several times over with shades of meaning that give a warmer tone of understanding without precision. (Let's not get into a discussion about the legal fraternity...)

You have the future of your book to face, of course. The near future is concerned with the comments of book reviewers in journals and magazines. Reflect on reviews of that book recently published from the Auckland alternative. The farther future is what your fellow scientists will think of the book; whether they will use it and, more importantly, quote from it. You may even make a pittance from your book or your chapter, but it is unlikely to pay for the effort. Pride is one of the driving engines of scientists, and a book is something tangible to be proud of.

Let us have fully professional editors operating in a draconian manner. Reputation will soon sort the good publications from the not-so-good. Let us largely scrap this wasteful peer review system that often is little more than a trade union type of restrictive practice and otherwise is a crutch for what teaching should have done. We will then also avoid that ever-present danger of intentionally or unknowingly stealing ideas. Reviewers are in a powerful position and some will not readily relinquish that power. What do you think? Is it better that we communicate with society more, and in a less restrictive but more competent manner?

Peter Meredith is a research scientist with Ilam Potato Sciences.