Following criticism from scientists around the world, a virology journal has retracted a paper describing the first test in humans of an Iran-made vaccine against COVID-19.
Iran licensed the home-grown Noora vaccine for emergency use in 2022 and has reportedly administered millions of doses to its citizens. The country’s health authorities say the shot is 94% effective.
The now-retracted paper, published in 2022 in the Journal of Medical Virology, was the only report on the clinical development of the vaccine to have appeared in an international journal. The article has been cited 10 times, according to Clarivate’s Web of Science.
As we reported in October, Forthal questioned the efficacy of the vaccine and expressed surprise “a manuscript containing so many serious flaws would have been accepted for publication following peer review, and given these issues, a retraction may be in order.”
The journal’s editor-in-chief, Shou-Jiang Gao, said at the time the paper had undergone two rounds of “rigorously [sic] review by experts of the field” before it was published. The authors had responded to Forthal’s critique, Gao told us, and their response had “already undergone 3 rounds of review, each with 2 reviewers”:
The last decision was made on October 24, 2023 and deadline for submitting the revision is November 23, 2023. So, we are waiting for the authors to submit the last revision before accepting and publishing it. This should fully address the issues that are raised.
But the response was never published. Meanwhile, in January, Gideon Meyerowitz-Katz, an epidemiologist in Australia, flagged additional problems in the paper on PubPeer, including several “impossible” and “contradictory” numbers.
On March 2, the journal announced the paper had been retracted, stating:
The retraction has been agreed due to concerns raised by third parties regarding issues with the data presented in the article. Several inconsistencies concerning the information provided about the analyzed subjects were additionally identified. Furthermore, the authors failed to disclose the presence of potential conflicts of interest that may have affected the interpretation of the results presented. Accordingly, the editors consider the conclusions of this manuscript to be invalid. The authors have been informed of the decision to retract but did not agree with it.
Corresponding author Hassan Abolghasemi of Baqiyatallah University of Medical Sciences, in Tehran, told us by email:
Retraction of our article was a political decision not a scientific decision because there was a pressure on journal based on [apartheid] scientific issue. Our response to the comment never accepted by [PubPeer] and journal to be published.
Two days after the retraction, another group of researchers published a study in mice comparing the effect of four COVID-19 vaccines used in Iran.
“Our results indicate significant immunogenicity and neutralization efficacy induced by PastoCovac Plus and Sinopharm, but not by Noora and SpikoGen,” the team wrote in the article, which cited the now-retracted paper. “This suggests the need for additional comparative assessment of the potency and efficacy of these four vaccines in vaccinated subjects.”
A journal is retracting a paper on the purported harms of vaccines against COVID-19 written in part by authors who have had similar work retracted before.
Graham Parker, Director of Publishing and Customer Success at Cureus, told Retraction Watch:
I can confirm we will be retracting it by the end of the week, as we have provided the authors with a deadline to reply and indicate whether they agree or disagree with the retraction.
The senior author on the work was Peter McCullough, a cardiologist at the Institute of Pure and Applied Knowledge who lost his board certification after the American Board of Internal Medicine found he had “provided false or inaccurate medical information to the public.”
Indeed, McCullough had already lost one paper, in Current Problems in Cardiology, from Elsevier, when he and his colleagues submitted their latest opus to Cureus. And SSRN, which hosts preprints for The Lancet, another Elsevier journal, had removed work by him and colleagues claiming large numbers of deaths from COVID-19 vaccines.
A few days after the paper appeared, we asked John Adler Jr., the editor in chief of Cureus, if the track record of the authors concerned him. His response seemed to admit to the risk, but he also defended the journal’s vetting of the paper:
Yes I am aware that many of these authors are skeptical zealots when it comes to the dangers of vaccines. Our editorial response was extra vigilance during the peer review process with 8 different reviewers weighing in on publication or not, including a few with strong statistics knowledge. Therefore, a credible peer review process was followed and the chips fell where they may. That is all I can say. If you or other readers were to note fatal flaws in this article now that it is published, i.e. failure to accurately report financial COIs [conflicts of interest], totally erroneous statistical analysis, fake data etc. we will of course re-evaluate at any time.
Adler then took a jab at other journals:
The decision process Cureus made, contrasts sharply with Elsevier’s seeming editorial decision to just censor the article using ad hominem concerns.
In a Feb. 9, 2024 letter to the journal and the publisher, John P. Moore, a microbiologist at Weill Cornell Medicine in New York City, and Gregg Gonsalves, an epidemiologist at Yale School of Public Health, in New Haven, Conn., expressed their “serious concerns” about the article. Among their objections:
The authors utterly lack relevant professional qualifications that would enable them to assess the scientific publications they draw on and/or attempt to criticize. The authors self-describe their affiliations under the rubric of “Independent Research”, or list private foundations, or in one case report an academic discipline unrelated to biology. In short, the authors cannot draw on years of training in biological science, but appear to be self-taught via the “University of Google”.
They continue:
The point here is that the Cureus review merely regurgitates claims about mRNA vaccines that have circulated on the internet and been debunked over and over again, including by fact-checking organizations (e.g., Factcheck.org, and the USA Today and Politico factcheck teams).
They conclude:
By bringing this highly problematic review to your attention, we hope that you will conduct a thorough review of how it was accepted for publication in Cureus under the Springer Nature imprimatur. How appropriate was the peer review process? How did the editor act? Is the acceptance of this review symptomatic of a wider problem at the journal? Finally, if you share our views that this review is so flawed as to be dangerous to public health, you may well decide that it should be retracted.
Springer Nature had apparently been looking into the case already, and ended up agreeing with Moore, Gonsalves and other critics of the article.
Steve Kirsch, a co-author of the paper, announced the retraction on his Substack over the weekend:
The paper I co-authored with 6 other authors will be retracted by the journal because the publisher won’t allow any paper that is counter-narrative to be published.
According to Kirsch’s post, Springer Nature’s inquiry found:
a significant number of concerns with your article that in our view can’t be remedied with a correction. The concerns include, but are not limited to:
We find that the article is misrepresenting all-cause mortality data
We find that the article appears to be misrepresenting VAERs data
The article states that the Pfizer COVID-19 vaccine saved two lives and caused 27 deaths per 100,000 vaccinations, and the Moderna vaccine saved 3.9 lives and caused 10.8 deaths per 100,000 vaccinations, though there does not appear to be convincing evidence for this claim.
Incorrect claim: Vaccines are gene therapy products.
The article states that vaccines are contaminated with high levels of DNA. Upon review we found that the cited references are not sufficient to support these claims.
The article states that SV40 promoter can cause cancer because SV40 virus can cause cancer in some organisms and inconclusively in humans. However, we find that this is misrepresenting the cited study (Li, S., MacLaughlin, F., Fewell, J. et al. Muscle-specific enhancement of gene expression by incorporation of SV40 enhancer in the expression plasmid. Gene Ther 8, 494–497 (2001). https://doi.org/10.1038/sj.gt.3301419
The article states that mRNA COVID-19 vaccines did not undergo adequate safety and efficacy testing, which the journal considers to be incorrect
The article incorrectly states that spike proteins produced by COVID-19 vaccination linger in the body and cause adverse effects.
Waving the white flag, a bowed but unbroken Kirsch wrote:
It doesn’t do any good to show them these reasons are all bogus. The laundry list of items is simply a placeholder to make it look like the journal is following the science.
Nothing we can say on appeal will make any difference.
The decision was made to retract the paper and facts don’t matter. It’s about supporting the narrative. When they write “in our view can’t be remedied with a correction” it means “don’t even bother arguing with us, your paper is retracted.”
For his part, Moore said:
The journal and publisher responded courteously and professionally to our letter, and I was pleased by the final outcome. They did what needed to be done.
This week Nick Arning, Helen Fryer and I released two related preprints describing a new method called Doublethink, and its application to identifying risk factors for COVID-19 hospitalization in UK Biobank:
Doublethink enables joint Bayesian and frequentist hypothesis testing when there is model uncertainty by interconverting Bayesian posterior odds and classical (frequentist) p-values. It has broad implications because (i) it reveals connections between the Bayesian approach to model averaging and the classical approach to multiple testing, and (ii) it brings the benefits of Bayesian model averaging to classical statistics.
Doublethink addresses two fundamental problems in hypothesis testing:
In classical tests, the statistical evidence that one variable directly affects an outcome generally depends on which other variables are assumed to directly affect it.
In Bayesian tests, the statistical evidence that one variable directly affects an outcome depends on the prior assumptions.
These issues are addressed by computing p-values from Bayesian model-averaged posterior odds, which (1) account for model uncertainty and (2) are theoretically invariant to prior assumptions, assuming large sample sizes.
Doublethink simultaneously controls the frequentist family-wise error rate (FWER) and the Bayesian false discovery rate (FDR). It builds on Johnson's Bayesian tests based on likelihood ratio statistics, and Karamata's theory of regular variation.
Identifying direct risk factors in UK Biobank with Doublethink
We applied Doublethink to identify direct risk factors for COVID-19 hospitalization in UK Biobank. This is a well-studied problem but we took an 'exposome-wide' approach in which we evaluated whether 1,900 variables measured in the UK Biobank each affected the outcome. This is still an under-utilized approach in epidemiology, which usually focuses on candidate risk factors.
Exposome-wide approaches have potential benefits over candidate risk factor approaches, including:
The ability to discover unexpected results.
Stringent control for multiple testing.
Avoidance of bias in choosing candidate risk factors or deciding to publish.
However, we only studied the direct effects of variables on the outcome. This means we cannot make statements about the total (direct and indirect) effects of a variable, e.g. smoking, on the outcome, which are needed in applications like assessing potential interventions.
We identified individual variables and groups of variables that were 'exposome-wide significant' at 9% FDR and 0.05% FWER, after accounting for the direct effects of all other variables.
Comparing our results to over 100 published studies of COVID-19 in UK Biobank, we
Recapitulated several commonly reported direct risk factors, e.g. age, sex, and obesity.
Excluded others, e.g. diabetes, cardiovascular disease, and hypertension, which might be mediated through other variables that measure general comorbidity.
Identified some infrequently reported direct risk factors, both individually, e.g. lung infection, and as groups, e.g. constipation/urinary tract infection, which might reflect underlying kidney disease.
The ability to test groups of variables, which increases sensitivity, was one of the benefits of Doublethink's model-averaging approach. It is particularly helpful in large biobanks that measure thousands of variables, because correlation between variables is pervasive, and can dilute the significance of individual variables that measure similar phenomena, like the numerous types of deprivation index. It serves as a flexible alternative to pre-analysis variable filtering algorithms, while controlling the risk of false positives by pre-defining significance thresholds for all possible tests.
To read more, please check out the preprints here and here.
Amidst the COVID-19 calamity, one can argue that science is one of the few aspects of the human response that has worked relatively well. However, despite the many advances in preventing and treating COVID-19, there have also been missteps as the world has scrambled to respond to a deadly new pathogen. It has been humbling for the U.S. to lead large high-income countries in per capita deaths from COVID-19 even with its wealth and scientific expertise. We are all too aware of the needless illnesses and deaths that have resulted from misguided political leadership, inadequate preparation, delayed responses, fragile supply chains, health disparities, and vaccine hesitancy. But we will not dwell on these issues here. Rather, we would like to review the COVID-19 pandemic through the prism of the 3R’s of research integrity: rigor, reproducibility, and responsibility. These form the fundamental pillars of the foundation of science. It is appropriate that we devote more attention to the foibles than to the successes so that we can learn from the mistakes and missed opportunities. What could have been done better? What needs to improve?
Has COVID research been rigorous and reproducible? Certainly, much of the work that brought us vaccines and new therapies has been. But unfortunately, other instances have fallen far short of rigor. Perhaps the most prominent example early in the pandemic related to the use of hydroxychloroquine for the treatment of COVID-19. Chloroquine and hydroxychloroquine, drugs used for the treatment of malaria and rheumatic diseases, were suggested as possible therapies for SARS in 2003, but were not formally studied in a clinical setting. Shortly after the COVID pandemic began to take off in China, rumors of a possible benefit of these agents began to appear in social media. On February 19, 2020, a letter was published in an obscure Asian journal, which mentioned a press briefing reporting that chloroquine improved clinical outcomes compared to control treatment in more than 100 patients with COVID-19 in China. No data were provided however, and the paper initially attracted little notice. However, on March 4, an article from a prominent research group in France led by Dr. Didier Raoult appeared in the International Journal of Antimicrobial Agents, reviewing the in vitro antiviral activity of chloroquine against coronaviruses and referring to the reportedly favorable experience in China.
On March 13, two cryptocurrency investors, Greg Rigano and James Todaro, posted a non-peer-reviewed paper online, touting chloroquine as “an effective treatment for COVID-19.” The authors claimed some prestigious institutional affiliations, which were subsequently disavowed, but nevertheless their paper was tweeted by Elon Musk to over 40 million followers, and one of the authors was interviewed by Tucker Carlson of Fox News, who publicized the claims to millions of viewers. On March 16, a preprint from Dr. Raoult and his group reported the use of hydroxychloroquine with or without azithromycin in 26 patients, who were compared with 16 controls receiving standard care. Viral clearance as measured by PCR was reported to be more rapid in recipients of hydroxychloroquine and much more rapid in those who also received azithromycin. The paper was accepted for publication soon afterward.
Red flags about this paper were raised almost immediately, but not before hydroxychloroquine was being touted by President Donald Trump as a “game changer” that would be fast-tracked by the FDA for approval. On March 28, the FDA issued Emergency Use Authorization for hydroxychloroquine and chloroquine in patients with severe COVID-19. Among the concerns regarding the French study were the small sample size, the lack of randomization (which resulted in poorly matched study and control groups), the use of different diagnostic assays, and the failure to account for six patients who had been initially enrolled in the study. It was also noted that the Editor-in-Chief of the journal publishing the article was one of the authors, and the review process took less than 24 hours. The same research group subsequently published a paper reporting that hydroxychloroquine and azithromycin were 92% effective in more than 1,000 patients with early COVID-19. However, the follow-up study did not include a control group. Dr. Raoult was quoted as saying that randomized controlled trials are unnecessary and unethical in deadly infectious diseases and appeal only to statisticians “who have never seen a patient.” Meanwhile, he initiated a lawsuit against Dr. Elisabeth Bik, one of the scientists who had criticized his original paper on hydroxychloroquine.
We now know from numerous subsequent clinical studies involving thousands of patients that hydroxychloroquine, with or without azithromycin, is not beneficial for patients at any stage of COVID-19, nor for the prevention of infection, and may even be associated with a higher risk of death. With very few exceptions, the results are highly consistent. The FDA Emergency Use Authorization was withdrawn on June 15, 2020. Yet hydroxychloroquine has been given to countless patients on the basis of non-rigorous science and continues to be given in some settings to this day.
Arturo Casadevall
As if to prove the adage that history repeats itself—the first time as tragedy, the second as farce, no sooner had enthusiasm for hydroxychloroquine begun to wane, when a new unproven treatment began to gain rapidly in popularity—ivermectin. Ivermectin is a macrocyclic lactone used to treat parasitic infections in humans and other animals. Early in the pandemic, Australian scientists reported that ivermectin could inhibit the replication of SARS-CoV-2 in vitro. It was quickly pointed out that the concentration of ivermectin required to inhibit viral replication greatly exceeded concentrations achievable from normal human dosing of the drug, but unfortunately this warning was unheeded.
Soon there were reports that ivermectin could prevent SARS-CoV-2 infection or reduce disease progression in patients with mild-to-moderate COVID-19. A systematic review found that most clinical studies of ivermectin failed to meet predefined eligibility criteria and suffered from imprecision and a high risk of bias. The authors of the review concluded that it is uncertain whether ivermectin is beneficial in COVID-19. But this message was drowned out by ringing endorsements from groups like “America’s Frontline Doctors,” a group of physicians closely aligned with right-wing political organizations, which had originally advocated the use of hydroxychloroquine to treat COVID. A related group calling themselves the Front Line COVID-19 Critical Care Alliance, or FLCCC, led by a critical care physician in Wisconsin named Pierre Kory, lent their voices to these efforts. Dr. Kory was quoted as saying, “My dream is that every household has ivermectin in their cupboard. And you take it upon development of the first symptom of anything approximating a viral symptom. . . Even if it’s not COVID, it’s safe to take it and it’s probably effective against that virus.”
During that latter part of the summer of 2020, an explosive rise in ivermectin prescriptions was observed in the US. High doses of ivermectin can produce a variety of adverse effects including gastrointestinal symptoms, seizures, respiratory failure, and coma. Poison centers found themselves swamped with calls about ivermectin overdoses. The FDA even issued a warning not to take ivermectin for COVID, noting that some people were even taking ivermectin preparations intended for the deworming of horses. Off-label ivermectin use continued to be popular on social media even though careful reviewers found serious flaws in the purported evidence for its benefits in COVID. An independent analysisfailed to find a single clinical trial demonstrating a benefit from ivermectin in COVID that did not contain “either obvious signs of fabrication or errors so critical that they invalidated the study,” including the same patient data being used to represent multiple subjects, non-random patient selection, numbers unlikely to occur naturally, incorrectly calculated percentages, and an inability of local health organizations to corroborate that the studies took place.”
Were these aberrations? Extreme cases perhaps, but systematic analyses suggests that deficiencies in the rigor of COVID-related research have been commonplace. A study of 686 COVID-19 clinical research articles found shorter time to publication and lower methodological quality than for other articles in the same journals. This suggests a lowering of the usual standards for publication in the time of plague.
Another issue that cannot be ignored is the important role of social media in disseminating and amplifying misinformation. A commentary has discussed an urgent need for improved understanding and stewardship of global collective behavior, cautioning that it has become easy to connect and share information through social media, but “in contexts where decisions depend upon accurate information, such processes can undermine collective intelligence and promote dangerous behavior.” A lack of trust in government institutions is undermining public health efforts. A country-level analysis demonstrates an inverse correlation between confidence in institutions and COVID burden. As the epidemiologist Jay Kaufman observed, “Science alone can’t heal a sick society. . . Science is a social process. . . To restore faith in science, there must be faith in social institutions.” Our social divisions are literally killing us.
According to Retraction Watch, more than 350 articles relating to COVID-19 have been retracted at the time of this writing. This is a modest number in view of the nearly 400,000 articles published on COVID, but it is nevertheless instructive to look at a few of the most high-profile retractions. Two of the retracted articles were published in the most prestigious and selective clinical journals in the world, the Lancet and the New England Journal of Medicine.
The Lancet paper, published in May 2020, purported to describe a multinational registry analysis of 96,032 patients hospitalized with COVID and found that hydroxychloroquine and chloroquine were associated with decreased in-hospital survival. The New England Journal of Medicine paper, published the following month, purported to analyze 8,910 patients hospitalized with COVID-19 and found no association between the use of angiotensin converting enzyme inhibitors or angiotensin receptor blockers and in-hospital mortality. Note that both of these findings have been subsequently confirmed by other legitimate studies, so what is in dispute is not the bottom-line findings of these articles. The problem is that the data were made up out of thin air. The retracted articles claimed to draw from an enormous registry including patients from 671 hospitals on six continents. One would expect such a monumental effort to involve hundreds if not thousands of contributors. However, suspicions quickly arose when no one could identify even a single hospital that had contributed to this registry, and major discrepancies were noted between the number of reported cases for some regions and data from independent sources.
Although both papers were retracted within weeks of publication, significant damage was done. Clinical trials had been suspended and international guidelines had been revised on the basis of the published findings. Editors and reviewers were apparently seduced by the reputation of the first author (a respected Harvard Professor) and the power of big data. The pressures of the pandemic may have allowed the papers to slip through the peer review process with less than the usual scrutiny. Dr. Sapan Desai, the founder of Surgisphere, the company that claimed to have assembled the registry, turned out to be a surgeon with multiple prior malpractice claims and little experience in data analytics. An investigation of his company identified just a handful of employees, several of whom had no background in science or data analysis, and the company was occupying rented office space. Dr. Amit Patel, a co-author on the studies, turned out to be Dr. Desai’s brother-in-law. The first author Dr. Mandeep Mehra admitted to not having seen any primary data. Although Dr. Desai has not admitted any wrongdoing, the Surgisphere papers are now believed to have been a hoax based on fabricated data. The editor-in-chief of the Lancet concluded that the study was a “monumental fraud.” Nevertheless, the papers have continued to be cited despite their retraction; each has been cited more than 700 times, and many of the citing articles appear to be unaware that the papers have been retracted.
Further evidence of peer review failure can be seen in two retracted articles by a psychologist named Harald Walach. Walach published articles claiming that COVID vaccines were responsible for two deaths for every three infections prevented and that face mask use results in hypercapnia in children. Although the papers were strongly criticized for their nonsensical findings and quickly retracted by the journals Vaccines and JAMA Pediatrics, they were widely circulated and continue to be cited by anti-vaccine and anti-face mask advocates.
Serious questions regarding scientific responsibility have arisen during the pandemic. Although SARS-CoV-2 was initially assumed to have arisen as a zoonotic pathogen making a species jump from bats to humans, possibly via an intermediate host, subsequent attention was focused on the possibility of transmission from a lab leak involving scientists at the Wuhan Institute of Virology, where coronavirus research is performed. Speculations were made about whether so-called “Gain-of-Function” research to identify viral adaptations that facilitate human infection might have inadvertently led to the emergence of SARS-CoV-2. At this time, the weight of scientific evidence favors a natural origin. Nevertheless, the lack of transparency on the part of the Chinese government and the politicization of the issue have raised uncomfortable questions about whether scientists are adequately considering and mitigating the public health risks of research with dangerous pathogens. As Dr. Tedros Ghebreyesus, the Director-General of the WHO has noted, “Lab accidents happen,” and it is incumbent upon scientists to be cognizant of these hazards and to take every possible precaution to minimize the risk to the public.In his classic description of the Black Death in London, A Journal of the Plague Year, Daniel Defoe wrote “It mattered not from whence it came.” But the origin of the pandemic does matter. Whether by zoonotic spillover or lab accident, we need to understand how the pandemic began in order to take appropriate measures to prevent this from happening again.
Although we have focused on some examples in which science fell short of its ideals during the COVID pandemic, overall, the scientific community acquitted itself well. We have already mentioned the remarkable success of vaccine development efforts. Scientists threw themselves wholeheartedly into COVID research, putting their other interests on hold. Collaborative and team science allowed the rapid execution of multinational trials, and data sharing, open access and preprint servers facilitated the timely dissemination of information. Critical knowledge regarding pre-symptomatic spread and superspreading, non-pharmaceutical interventions, diagnostic testing, distinction between protective and detrimental immune responses, ventilatory management, thromboembolic complications, and viral variants was quickly and widely shared, allowing substantial improvements in clinical outcomes as the pandemic proceeded.
Nevertheless, both the strengths and weaknesses of the contemporary scientific enterprise have been on ample display. Scientists produced a massive number of publications, but wading through this literature to find the truth has not been easy. As Robert Peter Gale wryly observed, the pandemic would be quickly ended if SARS-CoV-2 could be conquered by publications, guidelines, and interminable meetings alone. Due to the urgency of the pandemic, scientific standards were sometimes compromised. Limitations of randomized clinical trials as an exclusive source of knowledge to guide clinical decision-making were exposed. Important questions were raised about the potential dangers of scientific research on pathogenic microbes, while the origins of the pandemic have remained uncertain.
Vigorous debate is often regarded as a sign of a healthy scientific enterprise. However, open bickering among scientists about public health policies instead contributed to increasing societal polarization and undermining of public trust in science during the pandemic. Peter Sandman, an expert on risk communication, identified many mistakes made by officials in their messaging to the public, including a failure to communicate uncertainty and acknowledge error. Resistance to lockdowns and mandates were frequently met by backlash and hostility to public health officials. Unfortunately, the loss of credibility and public resistance to expert guidance have proven persistent, even as pandemic waves have receded.
Social media has posed a formidable challenge by creating alternative sources of information and disinformation that compete with science in influencing policy-making and public opinion. This represents a serious threat that society has scarcely begun to address. With science as just one among many potential sources of information, the imperative of making science as rigorous, as reproducible, and as responsible as possible becomes clear. Otherwise, why should anyone believe scientists rather than other sources claiming to represent the truth? For science to retain its influence in society, efforts to make it more rigorous, reproducible, and responsible must be a ceaseless undertaking. Only scientists can carry the torch and continue to perform the high-quality research that will lead to better prevention and treatment, not only of COVID, but of future threats as well.
As Albert Camus observed in The Plague, “What’s natural is the microbe. All the rest—health, integrity, purity—is a product of the human will, of a vigilance that must never falter.” We can and must learn from our experience because it is a certainty that COVID will not be the last pandemic.
Ferric C. Fang is a Professor of Laboratory Medicine and Pathology, Microbiology, Medicine, and Global Health at the University of Washington School of Medicine, Director of the Clinical Microbiology Laboratory at Harborview Medical Center, and a former Editor-in-Chief of Infection and Immunity. He is a member of the board of directors of The Center For Scientific Integrity, Retraction Watch’s parent nonprofit organization.
Arturo Casadevall is a Bloomberg Distinguished Professor and Chair of Molecular Microbiology and Immunology at the Johns Hopkins Bloomberg School of Public Health, a Professor of Medicine at the Johns Hopkins School of Medicine, and the Editor-in-Chief of mBio.
A controversial paper on the safety and immunogenicity of an Iran-made COVID-19 vaccine is being investigated by the U.S.-based publisher Wiley, Retraction Watch has learned.
The paper describes the vaccine’s first test in humans, marking the only time results from the clinical development of the homegrown shot have been reported in international journals.
But in a commentary published in May 2023, in the same journal, Donald Forthal, chief of the University of California, Irvine’s Division of Infectious Diseases, expressed surprise that “a manuscript containing so many serious flaws would have been accepted for publication following peer review, and given these issues, a retraction may be in order.”
Among several problems, Forthal wrote, was that more than 80% of the placebo group had neutralizing antibodies against SARS-CoV-2, “which is very surprising and unexpected, and raises the possibility that occult COVID‐19 infections could have confounded the results across all cohorts.”
Forthal, who studies the immune response to viral infections, also raised doubts about the serum samples from convalescent patients, which were used as a benchmark to gauge the vaccine’s ability to elicit protective antibodies:
Who are the convalescent serum donors? If they are largely asymptomatic or mildly symptomatic patients, or if they are months out from infection, the titers will be low, thereby providing a favorably biased comparator for the vaccine‐induced titers.
He added:
The results overall also raise concern about the clinical efficacy of the Noora vaccine at preventing COVID‐19 infection in a population, as 60% (9/15) of recipients of the 80 mcg dose series developed virus‐neutralizing antibody titers of less than 1/16th the level observed in most of the convalescent serum samples tested. By contrast, the Novavax recombinant spike protein vaccine generated mean virus‐neutralizing antibody titers greater than fourfold that of symptomatic COVID‐19 outpatients.
Forthal also flagged “a systematic problem with the references, as many of the citations are incorrect and not relevant to the issue discussed.” And he said a glaring conflict of interest had not been stated: Several authors were employees of Baqiyatallah University of Medical Sciences, in Iran, “which funded the study and has developed the vaccine for commercial purposes.”
The deficiencies in this publication raise serious questions about the effectiveness of the Noora vaccine in preventing COVID-19 infection. More than four months after publication of the Letter to the Editor, there has been no published response from the authors of the original paper.
Neither the corresponding authors nor the university responded to requests for comments.
A Wiley spokesperson said the publisher was “aware of the concerns” and that an “investigation is currently ongoing.”
Shou-Jiang Gao, the journal’s editor-in-chief, told us:
Upon receiving the comments [from Forthal], the authors have submitted a “response” and it is currently under a third round of revision.
The original article had undergone rigorously [sic] review by experts of the field before it was published. In fact, it underwent 2 rounds of reviews, each with 2 reviewers. The “response” that the authors have submitted has already undergone 3 rounds of review, each with 2 reviewers. The last decision was made on October 24, 2023 and deadline for submitting the revision is November 23, 2023. So, we are waiting for the authors to submit the last revision before accepting and publishing it. This should fully address the issues that are raised.
Dr. Donald Forthal has reached out to us many, many times and we have updated him the status [sic]. We believe this should go through a fair evaluation and peer review process, which takes some time. We ask you and him to be patient when this runs through the due review course.
The Noora vaccine is among several COVID-19 vaccines available in Iran. Only the results of the phase 1 trial have been published in an international journal, but the vaccine reportedly has also undergone phase 2 and 3 testing and was found to be 94% effective, according to Iranian health authorities.
As the COVID-19 Public Health Emergency expired on May 11, 2023, the COVID-19 Exposure Notifications System (ENS) in the majority of states also shut down. For the past three years, APHL has played a critical role in the US ENS.
The COVID-19 ENS made it possible for users to receive smartphone alerts when they — or more accurately, their phone — had been in the vicinity of someone who soon after reported a positive COVID-19 test through the ENS.
APHL, in collaboration with Google, Apple and Microsoft, and with program funding and guidance from partners at the Centers for Disease Control and Prevention, has provided support for the ENS by hosting national servers since August 2020.
“The availability of these servers eliminated a significant burden for state public health authorities and enabled exposure notifications to occur across state lines, despite each state having a unique solution,” explained Emma Sudduth, APHL consultant and program manager for the national server operations.
At one point, 28 states were using the ENS and relying on the servers hosted by APHL. Over time, as pandemic restrictions loosened and vaccine uptake increased, some states’ public health authorities discontinued their exposure notification solutions. Most of the remaining states shut down ENS on May 11 as Apple, Google and APHL discontinued their support for key components of the system.
The ENS saved countless lives across the country, as people were alerted to exposures, allowing them to get tested quickly and take precautions to reduce the ongoing spread of COVID-19. Research in Washington state in June 2021 showed that the state’s exposure notifications tool, known as WA Notify, had saved an estimated 30–120 lives and likely prevented about 6,000 COVID-19 cases during the first four months of its use.
By 2023, WA Notify had approximately 235,000 participants share a positive test result, which generated more than 2.5 million anonymous exposure notifications.
“We’re tremendously proud of what WA Notify was able to accomplish in a relatively short amount of time, and eager to find ways to utilize this life-saving technology again in the future,” said Bryant Thomas Karras, MD, chief medical informatics officer at Washington State Department of Health. “Much of WA Notify’s success can be attributed to innovative collaboration among public, private and academic partners,” which included APHL, Apple, Google, Microsoft and the University of Washington.
Unprecedented private-public health partnership
It became clear early on in the pandemic that most states did not have the time, funding or capacity to create their own ENS. Even if they did, it would have resulted in several different types of exposure notifications tools across the country.
To solve this problem, Google and Apple partnered to create a turnkey solution called Exposure Notifications Express (ENX). ENX made it easier for public health agencies to launch a state ENS by eliminating the need to build their own tool. These private technology companies looked to APHL, as a trusted public health partner, to host the servers.
“APHL has a long history of supporting states in data exchange and the hosting of solutions,” said Scott Becker, APHL CEO. “Our participation in the ENS gave the US public health community a capable and accountable partner for hosting key components of this groundbreaking technology.”
APHL stepped up to provide the necessary national key server, which holds the ENS data for the entire country. With APHL taking responsibility for storing and securing the data, states did not have to host and maintain their own servers. The Bluetooth-enabled notifications and the digital language known as “exposure notification keys” protected privacy.
Furthermore, a national centralized server allowed exposure notifications to work between users who had tools published by different states, ensuring notification regardless of state boundaries.
“The national infrastructure supported by APHL was essential to the deployment and maintenance of exposure notification systems,” Karras said. “By securely hosting the [national servers] APHL removed the burden on individual public health authorities to build and host their own servers, and importantly, enabled communication, i.e., interoperability, between exposure notification tools deployed in the U.S. Interoperability ensured that individuals using the systems could seamlessly notify others when traveling across states.”
Lessons for the future
Throughout the nearly three years of ENS, lessons were learned and applied to improve the system over time. This resulted in more states providing an exposure notification solution and more users in those states adopting it, which led to higher levels of notifications.
The knowledge and experience gained with the use of the ENS means that, in the future, public health won’t have to start at baseline to build a system from scratch. In addition, the ability to learn from different states about their experiences with ENS could be leveraged for future public health solutions.
The private-public health partnership — with Google and Apple working together with APHL, along with the public health community and state health departments — was the first of its kind and shows the value of such measures. The ENS saved lives, prevented cases and slowed disease spread. The public health community will continue to explore how such innovations can be used in the future to improve public health.
“Washington State Department of Health recognizes the great value of public-private-academic partnerships and collaboration with other states to develop interoperable systems,” Karras said. “This collaboration allowed us to accomplish something that would not have been possible without working together. We are committed to strengthening these partnerships that have made exposure notification systems so successful.”
At APHL, Sudduth said, “The collaboration between private technology companies and public health in this endeavor made a far stronger solution. Learning from this experience and growing these established relationships will strengthen public health.”
While wastewater testing is typically used in environmental health applications, wastewater testing for infectious diseases was a prominent discussion topic at the inaugural APHL ID Lab Con. This surveillance approach is relatively new in the United States and rose to prominence during the COVID-19 pandemic in large part due to CDC’s National Wastewater Surveillance System. Wastewater surveillance was a reoccurring theme throughout the conference, and two major ideas emerged: it’s useful and it should be here to stay.
What is Wastewater Surveillance?
Wastewater surveillance is a strategy for early detection of disease trends in a community and serves as a complement to clinical data. Since most of the US population is connected to municipal sewage treatment systems, wastewater surveillance creates a snapshot of how diseases and pathogens move through a community. Wastewater surveillance can gather data regardless of whether community members are symptomatic or if they seek medical attention when they are feeling sick. This can be because of their own choices, opportunities or access to healthcare, or are using COVID at home testing kits. It also provides relatively quick, reliable data that can help inform public health action. Wastewater surveillance has been used for decades for polio surveillance in other countries and gained popularity in the US during the COVID-19 pandemic. But it isn’t limited to just SARS-CoV-2, the virus that causes COVID. Many laboratories are expanding to other pathogens of concern and implementing wastewater surveillance as a staple tool in public health.
There were three different sessions at ID Lab Con that focused on wastewater surveillance and I learned new information from all of them. Below are summaries of the presentations made during each session. There is a lot here, but I found each presentation to be so interesting that I couldn’t hold back.
Wastewater Surveillance: Building Systems that Lead to Actionable Data
During this plenary session, panelists discussed their experiences collaborating with the US Food and Drug Administration (FDA), technical challenges faced when translating data to epidemiological practice, using wastewater surveillance for novel targets, and using it to inform public health actions.
Lauren Turner, PhD, of the Virginia Department of Consolidated Laboratory Services (DCLS) presented on their collaborative project with FDA that piloted wastewater surveillance efforts for SARS-CoV-2 variants in Virginia from 2021-2022. They implemented wastewater surveillance in targeted food production areas within the state such as tree nut, fruit and animal meat processing facilities due to the impact of the COVID-19 pandemic on worker illness rates and food manufacturing productivity. They were able to leverage national sequencing protocols established by the FDA Center for Food Safety and Applied Nutrition (CFSAN) to efficiently complete this project. Virginia developed the wastewater surveillance assay for SARS-CoV-2 quantification, then collaborated with FDA to determine a sequencing protocol and created a bioinformatics pipeline to collaborate with bioinformaticians to better understand the data. Overall, the data provided insight into SARS-CoV-2 viral load proportions in the sample in general as well as the representation of different variants. After the successful completion of the pilot project, DCLS will continue performing wastewater surveillance for SARS-COV-2 and will expand to other pathogens of concern. They have created their own two-step pipeline, affectionately named SPLINTER after the Teenage Mutant Ninja Turtles character, which analyzes their sequencing results using FDA’s sequencing protocols. While DCLS noted the challenges posed by a new technology, such as complex mixed sample analysis, they are thankful for the FDA pilot project and excited to see where wastewater surveillance takes them next.
Dagmara Antkiewicz, PhD, began her presentation, “Technical Challenges to Enhanced Integration of Wastewater-based Pathogen Data in Epidemiological Practice” by providing a brief background on wastewater surveillance work at the Wisconsin State Laboratory of Hygiene (WSLH). As early adopters of wastewater surveillance, WSLH began using this innovative technology in the fall of 2020, less than one year into the COVID-19 pandemic. Their program covers 50% of the state’s population with geographical coverage ranging from large cities to small municipalities. WSLH has also been performing weekly whole genome sequencing on a subset of the wastewater surveillance samples for over a year and has successfully on-boarded wastewater influenza and respiratory syncytial virus (RSV) assays. Genomic surveillance of wastewater is a relatively new approach as wastewater surveillance was typically used to quantify SARS-CoV-2 during the pandemic. Although uncommon right now, sequencing may be the new frontier for wastewater surveillance. However impressive WSLH’s wastewater program may be, Antkiewicz focused this presentation on the challenges of this testing. Noting that wastewater surveillance is here to stay, she posed the question, “How do we move forward with it?”
The biggest challenge Antkiewicz mentioned was the lack of a standard method. While APHL and the US Centers for Disease Control and Prevention (CDC) have come out with some guidance, every laboratory is running different tests based on what works best for them. Another noted challenge stemmed from the relative infancy of this testing in the US. As an epidemiological tool, there is still much to be uncovered—laboratory testing sensitivity, normalization and turnaround time all appear to be sufficient, but long-term data will tell if current approaches are adequate. Going forward, WSLH plans to continue using wastewater surveillance for SARS-CoV-2, influenza and RSV, while evolving the system to provide new sources of data to help understand how diseases move through a community.
With all the hype around the new show The Last of Us, it’s no surprise a fungus made it to the stage at ID Lab Con.Alessandro Rossi, PhD, Utah Public Health Laboratory, shared their work using wastewater surveillance to detect Candida auris.C. auris is a fungus that causes severe infections that are often resistant to medication and difficult to identify. This fungus is becoming more common and can spread in hospitals and nursing homes. Although at the start of this project, there was no documented evidence of person-to-person transmission of C. auris in Utah, Rossi and his colleagues thought it would be useful to determine whether wastewater surveillance could detect it in the environment. Leveraging infrastructure implemented during the COVID-19 pandemic, they transitioned to looking for C. auris. With just a few tweaks to a clinical culture method they were able to recover the fungus from wastewater sent from neighboring Nevada in May 2022. This result allowed them to establish an epidemiological link between wastewater isolates and healthcare facilities within the sampled sewershed. In collaboration with the Southern Nevada Water Authority, they also used community-level wastewater surveillance via PCR to follow the transfer of a C. auris– infected patient from Nevada to a city in Utah where the pathogen was not previously detected. The patient was admitted in early November and in only a few weeks, the laboratory was detecting the fungus in the wastewater. No other cases were identified, but C. auris was detected until mid-February. Efforts that began from curiosity of whether C. auris could be recovered from wastewater turned into an organism-specific proof-of-concept study on the feasibility of wastewater based epidemiology.
With SARS-CoV-2 detection under their belt, many laboratories have begun to branch out to see if wastewater surveillance technologies can be applied to other pathogens. One major component of wastewater surveillance is the use of sequencing to determine the presence of specific pathogens in the community that are not commonly looked for in other surveillance measures. In his presentation, Eric Vaughn and the DC Public Health Laboratory detailed their sequencing journey. To use laboratory bench time most efficiently, they determined the best kit for their workflow: a rapid sequencing kit. Thanks to this choice, they were able to find genes representative of sexually transmitted infections, and a host of E. coli, tuberculosis and Salmonella that wouldn’t have known otherwise. As they continued sequencing, they would move on to emerging infections just as mpox cases began to pop up around the country. By using a simple kit, they were able to make a significant discovery—identifying how much of a wastewater sample’s viral load was made up of the mpox virus. The very same library prep kit would also allow them to see seasonal viruses such as influenza A and B. Next, the DC laboratory set their sights on Salmonella. In DC, there is no mandatory reporting, meaning the laboratory is not guaranteed to catch every Salmonella case that goes to a DC hospital. However, using wastewater surveillance, they can determine a baseline of Salmonella within the city over time. By determining the prevalence of different Salmonella serotypes in the area, they can create a baseline that allows the laboratory to notice if something new or unusual shows up in the wastewater. This will enable the laboratory to communicate a potential Salmonella outbreak in the city to epidemiologists without the need for clinical data. Looking to the future, the DC Public Health Laboratory will continue to identify increased prevalence of pathogens, especially around long-term care facilities and schools.
Pushing Back the Frontiers if Science: A Review of This Year’s Literature
During this plenary session, panelists discussed the current literature on various aspects of public health. Topics ranged from respiratory viruses, mycology, next generation sequencing and, of course, wastewater surveillance, presented by Michael Pentella, PhD, D(ABMM), director of the State Hygienic Laboratory at the University of Iowa.
While wastewater surveillance is a relatively new technology in the US, there is no shortage of scientific literature on the topic—in fact, Pentella found over 1,200 papers when preparing his presentation. The first paper he covered was a systematic review that proves wastewater surveillance is a useful tool in the public health toolbox. The next articles were split into pre-analytical, analytical and post-analytical, with pre-analytical focusing on sampling approaches and the analytical part focusing on methods. But the post-analytical paper focused on ethics—an emerging conversation in wastewater surveillance that discusses the need for wastewater results to be considered as human health data. If wastewater data are treated similarly to clinical data, it will protect privacy and advance this science more efficiently as ethical concerns would likely hinder progress. While still a new technology to the US, available literature shows that wastewater surveillance has established itself as a powerful tool to identify, locate and manage outbreaks, but ethical considerations will need to underlie scientific decisions moving forward.
Responding to Emerging Infectious Disease
During this session, panelists shared their expertise on responding to various infectious diseases that have plagued the country these past few years. Kirsten St. George, PhD, Wadsworth Center, discussed how they leveraged their wastewater surveillance infrastructure developed during the pandemic to respond to a polio outbreak.
St. George began her presentation with a brief background on polio and its vaccinations. As polio had been declared eradicated in the US decades ago, vaccination efforts ceased nearly 20 years ago. Most notably, only 30% of infected individuals show symptoms, allowing this dangerous virus to silently travel though communities. In June 2022, when an immunocompetent adult presented at the hospital with fever, fatigue and stiff joints, nothing seemed to be exceptionally out of the ordinary. Samples were collected from the patient and sent to the Wadsworth Center where they were confirmed to be positive for enterovirus, which again, was nothing out of the ordinary. Everything was business as usual until the laboratory manager came to St. George with shocking news: they had received the routine VP1 sequencing results and the sample was positive for poliovirus. Her immediate response was “run them again.” Unfortunately, the sample had been evaluated three separate times with the same results. After confirmation from CDC, it was official: polio had been resurrected in New York. After some investigation, it was determined the patient had been exposed in the US. Wadsworth Center needed to track the virus to find out how long it had been present, so they turned to a newer addition in their public health arsenal: wastewater surveillance. To do this, they used legacy samples from fourteen counties near the initial case and found poliovirus had already been circulating before the initial case was identified. The laboratory continued to use wastewater data to guide epidemiologists and inform public health interventions such as vaccination efforts in low vaccination-rate regions. New York will continue to monitor the wastewater for poliovirus in the fourteen counties and areas with high travel rates since poliovirus is still endemic in other parts of the world. The laboratory has also developed a plan to scale up surveillance should another case arise. Due to the significant percentage of asymptomatic polio, wastewater surveillance and clinical data provided public health officials with the information they needed to keep this outbreak under control. It also provided the rationale to comprehensively plan for future responses.
While wastewater surveillance certainly has its challenges as a new technology in the US, attendees at APHL ID Lab Con learned it can play a key role in public health practice. With its ability to detect disease trends within a community in an unbiased way, identify the presence of diseases without clinical case data, and distinguish additional pathogens of concern, laboratories have been able to transition their pandemic-developed wastewater surveillance infrastructure into a versatile tool for a variety of post-pandemic public health needs.
Getting the right person to deliver the right message at the right time saves lives, but only if the audience can make sense of the message.
People need information they can find, understand, and use to make the best decisions for their health every day. The same is true before and during an emergency when there’s an increase in the amount of information and speed at which it comes out.
Health literacy is all about finding, understanding, and using information and making information findable, understandable, and usable. Health literacy is important to effectively prepare for and safely respond to an emergency like a natural disaster.
Two Parts to Health Literacy
The definition of health literacy was updated in August 2020 to acknowledge health literacy as the shared responsibility of individuals and organizations.
Organizational health literacy is the degree to which organizations equitably help people find, understand, and use information and services to inform health-related decisions and actions for themselves and others.
Personal health literacy is the degree to which people have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others.
Health Literacy in an Emergency
Taking care of our health is part of everyday life, not just when we visit a doctor, emergency department, or hospital.
Health literacy can help you prepare your health for an emergency and stay healthy during an emergency. For example, health literacy can affect your ability to
Many different factors can affect a person’s health literacy. Often people face multiple challenges that can make it difficult or even impossible for them to find, understand, and use information to make decisions. These challenges can include cultural differences, physical or mental disabilities, and unfamiliarity with emergency response terms.(1)
The term “social distancing” is one that confused people, who—up until the COVID-19 pandemic—had little or no experience with disease outbreaks. CDC responded by taking a plain language approach. Instead of asking people to “social distance” themselves from others, CDC said to “stay 6 feet away from others.”
Plain language is not “dumbing down” information or changing the meaning of a message. It’s about creating communication people can understand the first time they read it or hear it.
Bring Down Barriers to Health Literacy
Health literacy is the shared responsibility of the whole community. Businesses, schools, community leaders, government agencies, health insurers, healthcare providers, the media, and many other organizations and individuals all have a part to play in improving health literacy. Some of the ways we can do that include the following:
Work with health educators and other preparedness partners to familiarize people with health information and services and build their health literacy skills over time.
Consult with trusted messengers, including community, cultural, and faith leaders, to better understand your audience (e.g., cultural and linguistic norms, environment, and history) and to recruit members of your intended audience who can help you develop your messages or test them.
Work with trusted messengers to share your messages.
Use certified translators and interpreters who can adapt to your intended audience’s language preferences, communication expectations, and health literacy skills.
Practice clear communication strategies and techniques (e.g., follow plain language guidelines and define new and unfamiliar terms and acronyms).
Translate messages into multiple languages, including American Sign Language. Publish messages in alternate formats like braille, large print, and simplified text.(2, 3)
Improving health literacy requires many sectors and organizations to work together to make health information, resources, and services accessible to everyone.
Be About It
Everyone is responsible for improving health literacy. Here are some ways health, including crisis and emergency risk communicators, can “be about” improving health literacy.
Use these talking points to make the case for building a health-literate organization.
Take an online health literacy course to improve your practical skills as a health communicator. Training in health literacy, plain language, and culture and communication is essential for anyone working in health information and services.
Seek out and share resources that can help you better understand, relate to, and communicate with others.
Visit the Non-CDC Training webpage for more training materials on health literacy, plain language, cultural competency, consumer-patient skill building, and shared decision-making.
Thanks in advance for your questions and comments on this Public Health Matters post. Please note that CDC does not give personal medical advice. If you are concerned you have a disease or condition, talk to your doctor.
Have a question for CDC? CDC-INFO (http://www.cdc.gov/cdc-info/index.html) offers live agents by phone and email to help you find the latest, reliable, and science-based health information on more than 750 health topics.
I talked about how data science has been used during the COVID-19 pandemic, spanning vaccine design, clinical trials, surveillance and policy advice, and highlighting the identification of risk factors for disease.
If you like this talk, you might be interested in the following courses available this academic year:
A journal editor is defending his decision to publish a new paper showing that ivermectin can prevent Covid-19, despite more than a dozen retractions of such papers from the literature. The article, “Regular Use of Ivermectin as Prophylaxis for COVID-19 Led Up to a 92% Reduction in COVID-19 Mortality Rate in a Dose-Response Manner: Results … Continue reading Journal says ivermectin study met standard for ‘credible science’
) against COVID-19 in adults: A randomized, double-blind, placebo-controlled, Phase 1 trial.” 
