In Queen Mary: nobody expects the Spanish Inquisition John Allen and I wrote:
The same double standard follows, now, in our School of Biological and Chemical Sciences. For example, one of the “metrics” for research output at professorial level is to have published at least two papers in journals with impact factors of 7 or more. This is ludicrous, of course—a triumph of vanity as sensible as selecting athletes on the basis of their brand of track suit. But let us follow this “metric” for a moment. How does the Head of School fair? Zero, actually. He fails. Just consult Web of Science. Take care though, the result is classified information. HR’s “data” are marked Private and Confidential. Some things must be believed. To question them is heresy.
We hope to report back on our Head’s one-to-one interview with himself. After all, we have his word, and that of College senior management, that the restructuring is proceeding with complete fairness and transparency. Perhaps he’ll use a mirror?
This passage has been scrutinized already by two Employment Tribunals who have put to themselves the task to determine whether Professor Evans, who is the Head of School referred to above, met the “Research Output – quality” restructuring criterion, which is described immediately below in his own words in his restructuring proposal document.
Research Output – quality
The following minimum thresholds will be used for high publication quality between 1st January 2008 and 31st December 2011 for staff at each academic level.
Professor/Reader – 2 published items
Senior Lecturer/Lecturer – 1 published items
Publications must be in journals rated in the top 5%. The top 5% of journals will be assessed as either attaining an ARC quality rating of A* or an impact factor ≥ 7. Publications must be as a significant author (defined as sole, first, last or corresponding) and contain original research or review. Evidence of a publication being in press before 31st December 2011 will be accepted.
One advantage of this straightforward measure of quality is its objectivity. In April 2012, a search in Web of Science (see below why the search was restricted) for the publication record of Professor Evans returned no evidence of any paper published by this author after the 1st of January 2008 in a Journal that met the criteria listed above.
I asked Professor Evans about this problem given his written declaration that “all staff in SBCS” would be subject to these criteria, including himself. It looked strange that he would have written a set of criteria that he would so evidently fail, hence the insistence to find out what was going on. Repeated requests for this information were declined citing confidentiality.
It was during the Employment Tribunal hearing (Dr F Missirlis vs Queen Mary University of London, Case Number: 3202937/2012) that I first understood how Professor Evans claimed to have met the criterion listed above. He relied on two papers:
Predictive ecology: systems approaches
Matthew R. Evans, Ken J. Norris, Tim G. Benton
Phil. Trans. R. Soc. B (2012) 367, 163-169
Modelling ecological systems in a changing world
Phil. Trans. R. Soc. B (2012) 367, 181–190
–
These data were used to assess whether we met two “Research Output” criteria. The first one being:
Research Output – quantity
The following minimum thresholds will be used for high publication quality between 1st January 2008 and 31st December 2011 for staff at each academic level.
Professor – 11 published items
Reader – 9 published items
Senior Lecturer – 7 published items
Lecturer – 5 published items
Publications must be cited in Web of Sciences (document types categories – ‘article’, ‘proceedings paper’ or ‘review’). Publications not on Web of Science will not be included, nor will publications in Web of Science document categories not listed above, for example ‘Meeting abstract’, ‘Letter’, ‘Note’, ‘Editorial Material’, ‘Book review’ or ‘Correction/addition’. Evidence of a publication being in press before 31st December 2011 will be accepted.
The rules were applied to a subset of 61 academics. (Professor Evans’ claim at the time that they had been applied to all staff was false). Each member of staff only received their own table under the heading “confidential”. A significant number of errors was discovered and this caused stress.
One member of staff had produced a publication containing original research or review in a journal rated in the top 5%, which was listed in Web of Science category ‘Editorial Material’. He inquired why this publication was not listed in his table and he was told that it was not listed because the category ‘Editorial Material’ was not part of the “metrics”.
The table above requires some explanation. For example, if you look under ‘notes’ I have collected six zeros. That is because in this column the six papers I produced during my first four years at Queen Mary are considered for “quality”. None of my papers was published in a journal rated in the top 5% so I get rated as zero quality.
In other tables, each publication in a journal rated in the top 5% received the score of 1 under the same column, subject to the author having made a ‘significant’ contribution.
You can now consider the table Professor Evans produced for himself. He listed his publications from Web or Science: a collection of eleven zeros.
He then added the two publications mentioned above. The journal had classified one of the two as “Introduction”, which Web of Science identifies as ‘Editorial Material’.
fanismissirlis 
Paul Jump reports today in the Times Higher Education on the Employment Tribunal Judgment in Dr J F Allen vs Queen Mary University of London. He says “The tribunal also rules that… Professor Evans met the metrics threshold”.
https://www.timeshighereducation.co.uk/dismissal-was-unfair-but-academic-sparked-it-himself
This conclusion predisposes the Tribunal, which also found that John Allen’s dismissal was unfair because Evans and QMUL acted unreasonably both in removing all his teaching and in failing to address his grievance in respect with these decisions. The Tribunal, however, didn’t accept that it was within John’s right to refuse further teaching prior to reaching a conclusion on the former matters. The Tribunal found that our Lancet letter contributed to all of the above.
Therefore the true position is very important. If ‘editorial material’ was excluded by the metrics, then the fact is that Evans failed to meet the metrics threshold. Queen Mary managers continue to insist otherwise. Hence, a Tribunal that would have found that the Lancet disclosure was substantially true, would have likely reached a very different conclusion as to the reasons that led John to resist.
I am reading again the full “restructuring proposals” and I spotted another relevant passage:
“Research Outputs: Only including outputs that can be accessed via Web of Science is justified as this academic citation index is the most widely accepted and provides an objective, independent and transparent means to determine output numbers. In order to appropriately recognise the range of outputs that may be considered to represent peer-reviewed research activity that may vary across the broad subject range covered in SBCS both full journal articles and proceedings papers are included, but other outputs such as abstracts and editorial comments etc are not included.”