Toxicologic Pathology – a Sage title – has issued expressions of concern for six papers that were among the subjects of an investigation by Azabu University that concluded in November 2022.
The expression of concern, dated March 7, 2024, includes a list of the six articles and reads:
Toxicologic Pathology was contacted by the author, Shin Wakui, requesting retraction of these articles. Despite multiple attempts at gaining further information on the reasons for retraction, we have not received anymore communication regarding this case.
This expression of concern will remain in place until further evidence is provided to Sage.
The authors did not share that report when they contacted the journal in November 2023, a Sage spokesperson told Retraction Watch. “At this point it’s too soon to say how the report might impact any future action with the article,” Sage told us.
Wakui’s Azabu email address bounced, and we were unable to find other contact information for him.
Collectively, the papers have been cited just shy of 100 times, with just a handful of those citations coming after the university report was published in 2022.
A journal and publisher have retracted three papers about abortion, including one that has been used in court cases to support the suspension of FDA approval for mifepristone, aka an “abortion pill.”
For one of those articles, initially flagged by a reader, “an independent reviewer with expertise in statistical analyses evaluated the concerns and opined that the article’s presentation of the data in Figures 2 and 3 leads to an inaccurate conclusion and that the composition of the cohort studied has problems that could affect the article’s conclusions,” according to the notice.
The notice also said Sage “confirmed that all but one of the article’s authors had an affiliation with one or more of Charlotte Lozier Institute, Elliot Institute, and American Association of Pro-Life Obstetricians and Gynecologists, all pro-life advocacy organizations, despite having declared they had no conflicts of interest when they submitted the article for publication or in the article itself.”
One of the peer reviewers, Sage learned, “was affiliated with Charlotte Lozier Institute at the time of the review,” leading the publisher and journal editor to determine “the peer review for initial publication was unreliable.” That referee also reviewed the other two now-retractedpapers, according to Sage.
James Studnicki, the lead author of the three papers, told Retraction Watch the retractions were “a blatant attempt to discredit excellent research which is incongruent with a preferred abortion narrative.” He told The Daily Wire, a conservative news outlet that was first to report on the retractions, the move was “completely unjustified.” The Daily Wire notes that “The Supreme Court is set to hear arguments in March on the legality of restricting the abortion pill based on [Judge Matthew] Kacsmaryk’s ruling, proceedings that will certainly be impacted by the retractions.”
Sage had subjected one of the papers to an expression of concern in August 2023, saying they were investigating “potential issues regarding the representation of data in the article and author conflicts of interest” after being alerted by a reader. As News From The States reported then, the notice came after Chris Adkins, a professor at South University who teaches pharmaceutical sciences, raised concerns with Sage. As News From The States noted in August:
Kacsmaryk leaned hard on a 2021 study that was designed, funded and produced by the research arm of one of the most powerful anti-abortion political groups in the U.S. The judge cited this paper — which looked at Medicaid patients’ visits to the emergency room within 30 days of having an abortion — to justify that a group of anti-abortion doctors and medical groups have legal standing to force the FDA to recall mifepristone.
In a point-by-point response to Sage’s critiques of the paper sent to the publisher in November and now shared with Retraction Watch, Studnicki and colleagues pointed out they had noted their affiliations in the original manuscript and the then-proposed retractions “misrepresent ICMJE disclosure standards,” referring to the International Committee of Medical Journal Editors’ guidelines. They also call some of the post-publication peer reviewers’ critiques “factually incorrect” and “unfounded.” They conclude:
No single specific finding in any of the three papers has been explicitly challenged, let alone invalidated.
There is no evidence of a major error, miscalculation, fabrication, or falsification.
There is no breach of any of the COPE guidelines that could permit Sage to retract any of our published papers.
The retraction of any of these papers, let alone all three, is demonstrably unwarranted.
Adkins told Retraction Watch he is “pleased the journal approached my concerns with legitimate and serious consideration.” He continued:
It is reassuring that my initial concerns with the 2021 Studnicki et al. article were verified and affirmed by other experts. Despite the length of time spanning my initial communications with the journal and today’s retractions, I understand that thorough investigations and re-review processes take time. Given that these now-retracted articles have been excessively cited by parties involved in ongoing federal judicial cases, now positioned before the SCOTUS, Sage’s retractions should help our courts remain informed by the highest standards and quality in scientific and medical evidence.
The publisher Sage has retracted 209 articles from an engineering journal after an investigation found “compromised peer review or 3rd party involvement,” according to a company spokesperson.
At that time, the company marked 318 additional papers “with more complex issues” with expressions of concern as it continued investigating. All of the papers retracted today previously had expressions of concern.
The 209 articles were retracted with five different notices. Some articles “contain indicators of third-party involvement” and the corresponding authors “were unable to provide a satisfactory explanation,” one read.
Others “had been accepted as a result of a peer-review process that did not meet the standards and expectations at Sage” and a post-publication peer review “highlighted fundamental concerns within the articles.”
For another batch with compromised peer review, the corresponding authors asked Sage to retract their articles after the publisher contacted them.
The notice for a fourth grouping of articles stated that the corresponding authors did not respond to Sage’s attempts to contact them, and:
Due to concerns around unauthorised third party involvement, overall questions about the integrity of the research and authenticity of the author byline, Sage retracts these articles.
After reading the Expression of Concern on this article, co-author Enas Abdulhay wrote to Sage to explain that they did not author this publication and did not know the first author.
Due to concerns about the authenticity of the author byline and overall integrity of the research, Sage retracts this article.
The Sage spokesperson told us:
Editorial and ethical oversight and peer review management for this journal are being managed within Sage. Additionally, our Research Integrity team is continually working to proactively investigate misconduct and respond accordingly when issues arise. More information about our approach to bulk retractions can be found in this commentary piece.
When I began my graduate work almost 15 years ago, retractions of papers in academic journals were rare, reserved mainly for clear misconduct or serious errors. Today, rarity has given way to routine, with retractions coming more often and increasingly in bulk.
Sage is not immune to large-scale retractions, nor are we passive observers of their growth. As Retraction Watch wrote, we were “one of the first publishers to recognize large-scale peer review manipulation and begin retracting papers in bulk nine years ago.” Recently, we issued some major retractions; just in the last few months, we put out 37 from Proceedings of the Institution of Mechanical Engineers and 21 from Concurrent Engineering. And there are more to come.
While we don’t celebrate this type of action, the news is not all bad. The high numbers of retractions at times reflect a problem of industrialized cheating, but also, as in our case, a belief that rigorous scholarship – robustly reviewed by researchers who are experts in their fields – can and should improve the world. Sage was founded on this principle, and it guides everything we do.
We take our role of vigorously correcting the academic record very seriously because we believe in the scholarly process. We also know that every part of the process is managed by humans with biases (conscious or unconscious), agendas, heavy workloads, and – at times – dubious incentives.
As research integrity manager at Sage, I work to safeguard the credibility of the research published in more than 60,000 articles every year across more than 1,100 journals. In my role, I see a lot of unethical practices: peer review rings, where researchers unfairly influence the review process; paper mills that produce mass-fabricated research papers, and the brazen trend of selling authorship or entire papers on private or public forums. When it comes to preventing and correcting this type of action, much goes on behind the scenes.
What goes into a bulk retraction?
Behind every set of bulk retractions are teams of trained individuals using specialized tools, carefully going through several steps of a standardized process in line with industry best practice, often in concert with multiple parties representing various institutions.
My team at Sage comprises specialists who can recognize and resolve ethical breaches to help ensure that the articles published in our journals are based on sound, rigorously reviewed science. A large part of this lies in the unglamorous work of prevention – guidance and policies that aim to make publication impermeable to large-scale unethical activities. This means keeping authors and editors abreast of best practices and training editors and journal managers on research integrity matters such as paper mills or political pressure on research.
Of course, enforcing these policies and practices to perfection is possible. But just as there are those who are using all the tools at their disposal to game the system, we’re using everything at our disposal to uncover questionable practices. We use a number of tools to spot common breaches of publishing ethics, and we’re constantly fine-tuning our methods to stay current with evolving trends in this field. For example, one of these tools scrutinizes submissions and reviews for signs of suspicious activity. Clues often come in the form of non-institutional email addresses or peculiar IP addresses used during manuscript submission. We’ve also recently joined a pilot program as part of the STM Integrity Hub, a cross-publisher initiative, which detects unusual activity and misconduct across publishers such as duplicate submissions across journals and publishers – a tactic often employed by paper mills.
When misconduct isn’t caught before publication, my team commits considerable time and resources to thoroughly – sometimes painstakingly – investigate so that we can make the best decisions following industry standards from organizations such as the Committee on Publication Ethics (COPE). This process is not always linear or straightforward but there are steps common to most incidents.
We begin by determining the the scope of the issue by listing affected submissions and published articles. This can be a challenge, as sometimes the list spans multiple journals. We then brief the editor(s) and if there are irregularities in peer review or acceptance processes, we work with them to understand what may have happened. We scrutinize the data from each submission, looking for any patterns that are suggestive of third-party interference, a common source or telltale paper mill tactics. We also examine the entire journal for explanations behind this unusual activity and look for indications of genuine errors before assuming wrongdoing.
Next, we contact the authors about our findings and give them a chance to respond. If authors can’t be verified or remain unreachable, we proceed without their involvement. (In many cases it becomes very clear when a third party has tampered with our process, but there are times when it’s less clear-cut.
We make the decision to retract with the editor based on COPE’s retraction guidance. For example, if there is clear evidence of an experimental error, data fabrication, or a compromised peer review process (e.g., a peer review ring), we will retract. Finally, when notices are ready we allow authors the opportunity to note whether they agree or disagree with our decision, for fairness and transparency.
Throughout every step of the process, we ask ourselves: Is a completely new review warranted? Is this a valid concern that impacts the publication? Is this an honest error, or is it a pattern of behavior?
While it might appear that we’re dragging our feet before making a final decision, sifting through all the information to answer these questions can take a long time. But we strive to be as transparent as possible along the way. For example, when we do find sufficient indications of error or unethical practices, we are not afraid to issue Expressions of Concern to let the public know that an investigation is warranted. When our investigation is complete, we issue retractions or corrections as soon as possible when warranted or, in other cases, publicly state that we no longer see issues with the papers.
Still, with this level of attention given to each case, we can’t approach all cases as quickly as we would like, and we prioritize areas that affect people, policy, and the future of research.
Why do we do it?
Sometimes the process of a bulk retraction is initiated when a specialist or an industry “sleuth” notices a dubious pattern, such as image manipulation, and comes to us with a list of problematic articles with evidence to indicate why we should take action. But this type of ‘package-wrapping-and-leaving on the doorstep’ is rare.
Instead, most of the time, a bulk retraction occurs when we notice something off with just one paper – a request to change the author list after acceptance, a peer reviewer contacts us to let us know that a paper was published despite their seeing a need for substantial revisions, etc. In those cases, we start with the dubious paper and keep pulling the string for as long as necessary. We could easily stop at the first article and move on and save hours of staff time without anyone noticing (at least in the short term). But doing so would mean that we are evading one of our central responsibilities as an academic publisher.
As an independent company, we don’t have to bend to the views of shareholders. This gives us the freedom to prioritize academic integrity, make improvements, and shake things up if needed. Sometimes, this means making tough and even unpopular decisions to upload scholarly standards. For instance, my team and I have had to end publishing partnerships with highly regarded scholars when we discovered their unscrupulous behavior.
This independence extends to our editors. We uphold editorial independence enthusiastically and unapologetically. While we offer help, training, and advice, it’s the editors who have the final say over article selection and peer review, as long as they are academically rigorous and work in accordance with best practice, including guidelines from the Committee on Publication Ethics (COPE).
Post-publication, editors maintain authority over decisions to publish expressions of concern or other similar notes as well as retractions, again, as long as they are in accordance with COPE. We often help guide them through this process, and in the vast majority of ethics cases, are confident in the decisions they make. From time to time, when the editor is not willing to take action or is implicated in the unethical behavior, we believe it is our duty to step in. And we do. Finding the balance between accommodating editor wishes and gatekeeping is not easy, but we prioritize maintaining the integrity of the scholarly record as we navigate that balance.
I often compare our role to the news media; when research is published in our journals, it goes “on the record.” Once on the record, it changes the direction of future research for years to come and impacts policy and patient protocols, so “bad” science can lead to bad research, bad policy, and bad protocol. It can also foster increased mistrust in science and so, by continuously correcting the scholarly record, we also help to maintain trust in honest research.
In my experience, retractions also signal trust to our community of researchers and editors – they can feel confident that their rigorous research, which they have spent weeks, months, or more carefully conducting, writing, and/or reviewing, is in good company. As no publisher is immune to rising misconduct or paper mills, those that aren’t increasing their number of retractions to reflect that might not be vigilant enough. A clear message that we hear time and time again and was a major theme at this year’s STM conference is that we should be cleaning up the record and retracting more articles, more quickly.
Making Integrity Integral
But let’s keep in mind, the vast majority of scholarly research is conducted, reviewed, accepted, and published in good faith. And when retractions do occur, especially one-off retractions, they are often the result of error or oversight rather than something more nefarious – a teachable moment and a learning experience for researcher and editor alike.
I see the rise in bulk retractions as a hopeful sign for the future of scholarship. It means that we aren’t content to sweep bad published research under the rug. And, I can say with confidence, that with each new bulk retraction, we are refining, and sometimes redefining, our processes and practices to improve the system before tomorrow’s research is submitted to our journals.
While our processes and rigor are our own, we’re not unique; our colleagues at other reputable academic publishers are also acting to protect the integrity of the research enterprise. I’ve found that working with other publishers and bodies in the industry, such as the STM Integrity Hub, to regularly share new concerns and best practices is the best way to stop unethical research behavior before it gets accepted.
My interest in research integrity comes from a deeply personal place of needing to be part of an ethical research ecosystem that does not infringe on the rights of others. With my work, I hope to instill in others a similar way of working so that students graduating today, 15 years from now, and beyond, publish the research that will improve the world.
Adya Misra is Research Integrity & Inclusion Manager at Sage Publications.
In February, David Allison came across a study with a familiar problem.
The authors of the study purported to show an educational program helped women lose weight, but they had not directly compared the treatment and control groups. Instead, they’d used a statistically invalid method to compare changes within the groups.
Allison, the dean at Indiana University’s School of Public Health in Bloomington, along with researcher Luis-Enrique Becerra-Garcia and other colleagues, in July submitted a critique of the study to the journal that had published it. Four days later, Nauman Khalid, the journal’s editor in chief, wrote to the study’s lead author.
“I got excellent feedback from Dr. Becerra-Garcia,” Khalid wrote. “According to their analysis, the statistical tool that you used in your research is wrong and not well-validated.”
Now, after a lack of adequate response from the authors, the study will be retracted, according to Khalid.
Becerra-Garcia called Khalid’s action “prompt and decisive.”
Although the study has not yet been retracted, the response from the journal is a departure from the back-and-forth between authors and publishers that can stall retractions. Another paper Allison’s group critiqued was recently retracted – more than three years after they first raised concerns about the work.
The researchers examined the effects of an educational program based on Brazil’s new dietary guidelines, and found the program led to significantly decreased body mass and increased quality of life for the women who followed it compared to women in a control group.
The commentary by Allison’s group critiquing the study appeared in the same journal in September.
Allison has been working for years to correct statistical errors in the nutrition literature. He told Retraction Watch that studies that use these kinds of invalid statistical methods “are more likely to mislead readers to think that effectiveness of some intervention has been shown by accepted statistical procedures with known and stated error rates when it has not been. This may lead to the unwitting adoption or investment of time, money, effort, and risk in inert interventions.”
Becerra-Garcia told Retraction Watch:
Regarding statistical methods, it is pertinent to note that in a randomized controlled trial (RCT) design like the one used in Giacomello et al., 2023, to test for the effectiveness of an intervention, it is crucial to directly compare differences between groups in a significance test rather than relying solely on inspecting changes within each group. This entails testing whether the control and the intervention group differ in their respective change in outcomes from before to after treatment. This study only analyzed within-group changes (before vs. after) without statistically comparing between groups, which is a statistical error known as the “Differences in Nominal Significance” (DINS) error.
Becerra-Garcia added that he and his colleagues have detected the same error in other studies, and the mistake can sometimes lead to incorrect conclusions, as it did with this study.
On July 25, well before the critique was published, Khalid emailed Daniel Fernandes Martins, the senior author of the study and an adjunct professor at the University of Southern Santa Catarina in Brazil, informing him of the critique.
“Per their commentary, they recommend correcting or retracting your original article,” he wrote.
He also wrote that Martins, who did not respond to an email from Retraction Watch, would have 15 days to take either action, according to the journal’s policy. However, the study remained online for many months.
After the critique of the study was published in September, Becerra-Garcia asked Khalid about the status of the study, writing that he and his coauthors “believe it is essential to address these issues to maintain the integrity of the scientific literature.”
In reply, Khalid said the authors of the study had requested the study be withdrawn without comment. To do so would be against the journal’s ethical standards, he said, and instead, the study would soon be retracted.
Khalid told Retraction Watch:
The main reason for retraction includes significant issues in statistical analysis of data and wrong presentation of conclusion. The initial research data was cross-verified by Prof. Allison’s group, and the re-evaluated data was shared with original authors via published commentary. Moreover, the original authors wanted to withdraw their manuscript, which I didn’t agree.
He added that “the authors need to be more vigilant in statistical analysis with human subjects since the minute change in conclusion has significant impact on what we see in this research.”
A spokesperson for Sage, the journal’s publisher, said that they were “actively investigating this [case] in accordance with COPE guidelines.” The retraction guidelines set out by the Committee on Publication Ethics (COPE) state that a study should be retracted if there is “clear evidence that the findings are unreliable,” including “as a result of major error (eg, miscalculation or experimental error).”
A cancer journal has retracted a 2016 paper by a group in China after deciding – more than five years after publication – it couldn’t stand behind the work. The article, “The preoperative platelet–lymphocyte ratio versus neutrophil–lymphocyte ratio: which is better as a prognostic factor in oral squamous cell carcinoma?”, appeared in Therapeutic Advances in … Continue reading Retraction leads to review change at SAGE journal
Finding papers produced by paper mills has become a major headache for many of the world’s largest publishers over the past year, and they’re largely playing catch-up since sleuths began identifying them a few years ago. But there may be a new way: Earlier this month, Adam Day, a data scientist at SAGE Publishing, posted … Continue reading How to find evidence of paper mills using peer review comments
A SAGE journal has retracted 122 papers because of “clear indicators that the submission and/or peer review process for these papers was manipulated.” Those indicators, according to The International Journal of Electrical Engineering & Education: include but are not limited to submission patterns consistent with the use of paper mills, collusion between authors and reviewers … Continue reading Journal retracts 122 papers at once
A psychiatry journal has retracted two papers on Covid-19 and mental health, and a third on racism, after concluding that an author on the articles rigged the peer-review process. The papers, which appeared in the International Journal of Social Psychiatry (IJSP), were co-authored by Debanjan Banerjee, then geriactric psychiatrist at the National Institute of Mental … Continue reading Journal retracts three papers — including two on COVID-19 — because ‘trainee editor’ committed misconduct
The publisher SAGE is in the process of retracting more than 30 papers across three of its journals after determining that they were churned out by paper mills — prompting the company to take a closer look at its policies and procedures. The suspect papers were initially flagged by Elisabeth Bik and others as part … Continue reading Publisher retracting more than 30 articles from paper mills
