It was 28 November 2012 when the Levelt (Tilburg University), Noort (University of Groningen) and Drenth (University of Amsterdam) committees published their joint investigation report: Flawed science: The fraudulent research practices of social psychologist Diederik Stapel.
Stapel was found to have committed fraud in at least 55 publications at all three universities. In 2011, he was suspended by Tilburg University and shortly afterwards, he returned his PhD title to the rector of the University of Amsterdam.
Although the damage caused to science was enormous, it was immediately clear to everyone which literature had been tainted: all fraudulent articles were revealed by name, including the names of co-authors.
Enormous damage
That is not what happened at this university. In the investigation report of the Committee for Academic Integrity on fraud in the work of former Leiden psychologist Lorenza Colzato, the Executive Board redacted the titles of the fifteen fraudulent publications and the co-authors’ names ‘in order to protect the co-authors’.
What do the investigators of Stapel’s misconduct have to say about this?
‘I don’t approve of the anonymisation’, says Pieter Drenth, Chair of the Drenth Committee and Professor Emeritus of Psychology at the Vrije Universiteit. ‘Openness and the availability of information are very important conditions for the correction of science when it comes to violations of scientific integrity.’ He says that ‘everyone who has published work in collaboration with Colzato’ is now considered suspect. ‘The only way to solve that is through openness.'
Chris Klaassen, a member of the Drenth Committee and Professor Emeritus of Statistics at the University of Amsterdam, shares his view. ‘The publications in which integrity has been violated should be revealed.’
Jules Pieters, a member of the Noort Committee and Professor Emeritus of Applied Psychology at the University of Twente, speaks of ‘large-scale fraud’ in the Colzato case. He also believes there is insufficient reason for anonymisation. ‘In the Stapel investigation, we considered disclosing the titles as a matter of course, because all the authors mentioned by name are responsible for the content of an article’, he says. ‘So that would also have been the obvious thing to do in Colzato’s case.’
Jacques Hagenaars, a member of the Levelt Committee and Professor Emeritus of Statistics at Tilburg University, also thinks that keeping the titles of the articles confidential is ‘wrong’. Although the guidelines do allow for anonymisation, this is ‘fundamentally at odds with the principle of transparency’, he says. ‘Transparency is an essential feature of science. Secrecy has no place in the scientific field.’
Flawed research, wrong theories
He also finds it problematic to leave the task of correcting science to the journals in which the fraudulent articles were published. ‘In the Stapel case, they either did not respond, or they were very slow to do so’, says Hagenaars. Not only do the committee members criticise the anonymisation of the publications, they also think that much more of Colzato’s work should be investigated for fraud than just the post-2015 publications.
‘We were more thorough in our approach than Leiden’, says Drenth. ‘Three committees analysed all 137 of Stapel’s publications.’ He thinks the same should be done for Colzato’s work. ‘Everything has to be investigated, otherwise you fall short. You can’t just assume that the rest of the publications are fine. The response is that it takes a tremendous amount of time, but choosing not to carry out a full investigation will have serious consequences: scientists will make mistakes if they base their work on flawed research, wrong theories may persist and fraudulent articles may be included in meta-studies.’
Ed Noort, Chair of the Noort Committee and Professor Emeritus of Israelite Literature: ‘A partial investigation leaves open the possibility that tainted publications will continue to circulate in scientific discourse. So, in the interests of science, we chose to investigate Stapel’s entire body of work.’
Hagenaars also believes that ‘as an institution, you have an obligation to investigate everything’, which is what happened in the case of Stapel. ‘That takes a tremendous amount of time, but given the enormous importance of the matter, it’s imperative that the university makes a great effort.’ Hagenaars notes that some universities have a different view. ‘Other university institutions have kept similar fraud cases under wraps. Partly out of concern for innocent co-authors, but mainly out of fear of reputational loss, I’m afraid.’
The experts do admit that it was much easier to investigate all of Stapel’s work: the fraud was investigated using a statistical technique for which the empirical data underlying the articles were not necessary.
This method was developed by Klaassen. ‘The results were too good to be true. The reason for this is because Stapel made up the underlying “empirical data”. People are very bad at making up that kind of data.’
Reputational damage
The situation with Colzato is different, Klaassen explains. ‘It’s not that data has been made up, but that unwanted empirical data has been omitted without mention. That means that every article has to be compared with all the underlying data derived from experiments, because it is highly unlikely that the publications themselves show any evidence of fraud. The question is whether these underlying data can be retrieved at this point - especially for the older articles for which the archiving is inadequate.’
Stapel was sentenced to a community punishment order of 120 hours for committing scientific fraud after the University of Groningen and Tilburg University filed charges against him. That was ‘because of forgery and fraud’, says Ed Noort. 'And to put an end to the culture in which institutions keep such practices to themselves to avoid damaging their reputation.’
Leiden University has not filed a forgery report against Colzato, says university spokesperson Caroline van Overbeeke. However, according to Noort, Leiden University should have done just that. ‘If an offence has been established, I believe charges should be filed.’
Klaassen is also in favour of going to court. ‘These fraudsters have a major and extremely negative impact on the lives of their PhD candidates and staff. I don’t understand why no legal action has been taken.’ Pieters agrees. ‘Maybe it will still come to that.’
Not thrilled
Drenth is less emphatic on the matter. ‘Between correct science and completely fabricated research, there is a huge grey area of sloppy science’, he says. ‘All this bad and impure science does not necessarily fall under the legal definition of forgery. That’s why I’m a bit apprehensive about labelling all fraudulent articles as such. We should leave that to the lawyers.’
In the meantime, Colzato continues her work as a researcher with impunity. Since October 2019, she has been working at TU Dresden at the Faculty of Medicine.
‘That shouldn’t be allowed to happen’, says Pieters. Noort calls the appointment ‘unwise’ and Drenth is also ‘not thrilled about it’. He would be ‘very hesitant’ to employ her: ‘You can’t just allow a person who has done something like that to give lectures and do methodological research. Someone like that should be working in a different environment.’
Murky past
Len de Klerk, a member of the Drenth Committee and Professor Emeritus of Urban Planning at the University of Amsterdam, says that not all companies and institutions make a fuss about a murky past. My work experience - largely outside the university context - has taught me that there are employers who think: no criminal conviction means no reason for impediment, provided that the applicant is a perfect fit for the job description.’
Whether the TU will have to terminate her appointment after all depends on the information that Colzato provided to the university management, Drenth thinks. ‘It’s possible that she told the Board in Dresden everything and was hired anyway. In that case, there is no reason to dismiss her.’
However, that is a very unlikely scenario, he thinks. ‘As rector, I would consider firing her if she had been dishonest.’
Professor Emeritus of Psycholinguistics Willem Levelt, who was in charge of the entire investigation into Diederik Stapel, responded to questions by e-mail and only agreed to participate in the publication if Mare would include his text in full. This is what he wrote:
‘I can’t pass judgement on the Colzato case. I haven’t looked into it and I’m not going to. However, I do have an opinion on how to generally deal with an author’s body of work after serious publication fraud has been discovered. In that case, it’s important to subject the author’s entire body of work to an integrity investigation. Failure to do so leads to two undesirable situations. Firstly, all the author’s valid publications will remain suspect and lose all their value to science. Secondly, the author’s (other) fraudulent publications won’t be recognised as such or retracted, thus continuing to play a disruptive role in research and applications. In particular, valuable time and attention may be wasted on fruitless replication and follow-up research, or applications may prove ineffective or even harmful.
'Who is responsible for carrying out the integrity investigation? I think it’s the academic institutions where the author was employed when they published their work. The results of such an integrity investigation should be public and should lead to the retraction of the contested publications. It’s understandable that the academic institutions in question find this difficult. A specialised integrity investigation is expensive and time-consuming. Still, accreditation doesn’t only concern the quality of publications, but also how the institution deals with the integrity of research and publications. This also sets the standards for its own staff and students.’