A paper on green innovation that drew sharp rebuke for using questionable and undisclosed methods to replace missing data will be retracted, its publisher told Retraction Watch.

Previous work by one of the authors, a professor of economics in Sweden, is also facing scrutiny, according to another publisher. 

As we reported earlier this month, Almas Heshmati of Jönköping University mended a dataset full of gaps by liberally applying Excel’s autofill function and copying data between countries – operations other experts described as “horrendous” and “beyond concern.”

Heshmati and his coauthor, Mike Tsionas, a professor of economics at Lancaster University in the UK who died recently, made no mention of missing data or how they dealt with them in their 2023 article, “Green innovations and patents in OECD countries.” Instead, the paper gave the impression of a complete dataset. One economist argued in a guest post on our site that there was “no justification” for such lack of disclosure.

Elsevier, in whose Journal of Cleaner Production the study appeared, moved quickly on the new information. A spokesperson for the publisher told us yesterday: “We have investigated the paper and can confirm that it will be retracted.”

We first contacted Heshmati after a PhD student tipped us off about his dubious research practices. The student had obtained Heshmati’s dataset, which, along with email correspondence between the professor and the student, revealed how Heshmati had approached the numerous missing observations in the data. 

When we presented Heshmati with the criticism leveled against him, the researcher stood by his methods, but said they should have been “acknowledged and explained.” He had missed doing so “unintentionally in the writing stage of the paper,” he said.

It’s a mistake Heshmati has made twice, it turns out. In 2020, he and two colleagues published a paper in Empirical Economics, a Springer Nature title, that bore strong resemblance to the 2023 article and relied on the same patched-up dataset. The article mentioned neither the data gaps nor the Excel operations.

The 2020 paper, “Green innovations and patenting renewable energy technologies,” has been cited 23 times, according to Clarivate’s Web of Science.

A spokesperson for Springer Nature told us:

Thank you for bringing these concerns to our attention. We are now looking into the matter carefully following an established procedure and in line with best-practice COPE guidance. We would be happy to provide an update when we have completed our investigation.

When we reached out to Heshmati about the 2020 paper, he told us he used the same data as in the later article, but with some “differences in the way the variables are defined and transformed.” As such, he said, the description of the data was similar between the two papers:

The first paper is cited in the second paper. Journal of Cleaner Production has strict control of similarity rate and decline [sic] papers with high rate of similarity regardless of their authors.

Heshmati also said he was “disappointed” by Retraction Watch’s “approach and clearly implied false accusations”:

As you well know, I was open to supporting [the PhD student who contacted Heshmati], after exchanging some e-mail [I] sent him a zoom link and had a long zoom meeting with him explaining the reason of imputation of missing observations and their benefits and harms. I shared the data with him where all imputed points are highlighted in yellow color to allow future replacement. I have been all the way open, honest, and willing to share information without fear of negative publicity. I still believe that I did right and would use the technique again on same data but with deleting years and countries with high frequency of missing values and produce sensitivity analysis of the result with full and restricted sample.

Esfandiar Maasoumi, a professor of economics at Emory University in Atlanta and first author of the 2020 paper, said he believed Heshmati to be “honest and reliable,” but denied involvement in and knowledge of the data imputation. He told Retraction Watch:

I take data accuracy and transparency very seriously. I did not have any hand in data and code/implementation parts of the paper, and relied on my coauthors for these components. I regard my coauthor, Dr. Heshmati, as very honest and reliable and well informed. I would reject any implication that he has intentionally done anything improper.

Maasoumi also characterized Retraction Watch’s coverage of Heshmati’s research as “sensational” and “rather uninformed” and said we jumped to conclusions about intentions:

The criticism of lack of reporting should be placed on both authors and journals that do not allow or accommodate full descriptions of work in scarce journal pages. I noted that Dr. Heshmati had willingly, and openly engaged with [the PhD student who approached Heshmati about the missing data] and completely cooperated with him/her. This is clear indication of his good and honest intentions. Personally, my experience with other authors for the last 50 years is that it is next to impossible to get data, code, and meaningful correspondence with authors of papers, published or otherwise.

He elaborated, quoting experts we spoke with:

Any reader of your work here, and generally, would note that the central, key message, is ” dishonesty”, or ” cheating” as you put it and imply it very strongly. Given that those implications are categorically false, will you write a retraction, a correction, and contact the same sources immediately?

Heshmati did hand over his data to the PhD student, if grudgingly.

“I have the excel data file but do not distribute it as I may update and use it again in research,” Heshmati wrote in an email seen by Retraction Watch. “Now I have teaching and a busy schedule, but if your intension [sic] is to learn I may show you the file.”

The student vowed not to share the spreadsheet with others – a promise he later broke after realizing just how appalling Heshmati’s methods were. So much of the data had been filled in by Heshmati it felt to the student like “fabrication” warranting a retraction.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, subscribe to our free daily digest or paid weekly update, follow us on Twitter, like us on Facebook, or add us to your RSS reader. If you find a retraction that’s not in The Retraction Watch Database, you can let us know here. For comments or feedback, email us at team@retractionwatch.com.

Last year, a new study on green innovations and patents in 27 countries left one reader slack-jawed. The findings were no surprise. What was baffling was how the authors, two professors of economics in Europe, had pulled off the research in the first place. 

The reader, a PhD student in economics, was working with the same data described in the paper. He knew they were riddled with holes – sometimes big ones: For several countries, observations for some of the variables the study tracked were completely absent. The authors made no mention of how they dealt with this problem. On the contrary, they wrote they had “balanced panel data,” which in economic parlance means a dataset with no gaps.

“I was dumbstruck for a week,” said the student, who requested anonymity for fear of harming his career. (His identity is known to Retraction Watch.)

The student wrote a polite email to the paper’s first author, Almas Heshmati, a professor of economics at Jönköping University in Sweden, asking how he dealt with the missing data. 

In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data. He had marked anywhere from two to four observations before or after the missing values and dragged the selected cells down or up, depending on the case. The program then filled in the blanks. If the new numbers turned negative, Heshmati replaced them with the last positive value Excel had spit out. 

The student was shocked. Replacing missing observations with substitute values – an operation known in statistics as imputation – is a common but controversial technique in economics that allows certain types of analyses to be carried out on incomplete data. Researchers have established methods for the practice; each comes with its own drawbacks that affect how the results are interpreted. As far as the student knew, Excel’s autofill function was not among these methods, especially not when applied in a haphazard way without clear justification.

But it got worse. Heshmati’s data, which the student convinced him to share, showed that in several instances where there were no observations to use for the autofill operation, the professor had taken the values from an adjacent country in the spreadsheet. New Zealand’s data had been copied from the Netherlands, for example, and the United States’ data from the United Kingdom. 

This way, Heshmati had filled in thousands of empty cells in the dataset – well over one in 10 – including missing values for the study’s outcome variables. A table listing descriptive statistics for the study’s 25 variables referred to “783 observations” of each variable, but did not mention that many of these “observations” were in fact imputations.

“This fellow, he imputed everything,” the student said. “He is a professor, he should know that if you do so much imputation then your data will be entirely fabricated.”

Other experts echoed the student’s concerns when told of the Excel operations underlying the paper.

“That sounds rather horrendous,” said Andrew Harvey, a professor of econometrics at the University of Cambridge, in England. “If you fill in lots of data points in this way it will invalidate a lot of the statistics and associated tests. There are ways of dealing with these problems correctly but they do require some effort.

“Interpolating data is bad practice but lots of people do it and it’s not dishonest so long as it’s mentioned,” Harvey added. “The other point about copying data from one country to another sounds much worse.”

Søren Johansen, an econometrician and professor emeritus at the University of Copenhagen, in Denmark, characterized what Heshmati did as “cheating.” 

“The reason it’s cheating isn’t that he’s done it, but that he hasn’t written it down,” Johansen said. “It’s pretty egregious.” 

The paper, “Green innovations and patents in OECD countries,” was published in the Journal of Cleaner Production, a highly ranked title from Elsevier. It has been cited just once, according to Clarivate’s Web of Science.

Neither the publisher nor the journal’s editors, whom the student said he alerted to his concerns, have responded to our requests for comment.

Heshmati’s coauthor, Mike Tsionas, a professor of economics at Lancaster University in the UK, died recently. In a eulogy posted on LinkedIn in January, the International Finance and Banking Society hailed Tsionas as “a true luminary in the field of econometrics.” 

In a series of emails to Retraction Watch, Heshmati, who, according to the paper, was responsible for data curation, first said Tsionas had been aware of how Heshmati dealt with the missing data.

“If we do not use imputation, such data is almost useless,” Heshmati said. He added that the description of the data in the paper as “balanced” referred to “the final data” – that is, the mended dataset.

Referring to the imputation, Heshmati wrote in a subsequent email:

Of course, the procedure must be acknowledged and explained. I have missed to explain the imputation procedure in the data section unintentionally in the writing stage of the paper. I am fully responsible for imputations and missing to acknowledge it.

He added that when he was approached by the PhD student: 

I offered him a zoom meeting to explain to him the procedure and even gave him the data. If I had other intensions [sic] and did not believe in my imputation approach, I would not share the data with him. If I had to start over again, I would have managed the data in the same way as the alternative would mean dropping several countries and years.

Gary Smith, a professor of economics at Pomona College in Claremont, California, said the copying of data between countries was “beyond concerning.” He reviewed Heshmati’s spreadsheet for Retraction Watch and found five cases where more than two dozen data points had been copied from one country to another. 

Marco Hafner, a senior economist at the RAND Corporation, a nonprofit think tank, said “using the autofill function may not be the best of ideas in the first place as I can imagine it is not directly evident to what conditions missing values have been determined/imputed.”

Hafner, who is research leader at RAND Europe, added that “under reasonable assumptions and if it’s really necessary for analytical reasons, one could fill in data gaps for one country with data from another country.” But, he said, the impact of those assumptions would need to be reported in a sensitivity analysis – something Heshmati said he had not done. 

“At the bare minimum,” Hafner said, the paper should have stated the assumptions underlying the imputation and how it was done – something that, he added, would have reduced the chances of the work getting published should the reviewers find the methods inappropriate.

Like Retraction Watch? You can make a tax-deductible contribution to support our work, subscribe to our free daily digest or paid weekly update, follow us on Twitter, like us on Facebook, or add us to your RSS reader. If you find a retraction that’s not in The Retraction Watch Database, you can let us know here. For comments or feedback, email us at team@retractionwatch.com.