Scientists say “sloppy science” more serious than fraud

An article on Nature: INDEX reports on a recent survey of scientists: Cutting corners a bigger problem than research fraud. The subtitle says it all: Scientists are more concerned about the impact of sloppy science than outright scientific fraud.

The survey was published on BioMed Central.
Bouter, L.M., Tijdink, J., Axelsen, N., Martinson, B.C., and ter Riet, G. (2016) Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Research Integrity and Peer Review, 1(1), 17. [doi: 10.1186/s41073-016-0024-5]

Abstract
Background


Codes of conduct mainly focus on research misconduct that takes the form of fabrication, falsification, and plagiarism. However, at the aggregate level, lesser forms of research misbehavior may be more important due to their much higher prevalence. Little is known about what the most frequent research misbehaviors are and what their impact is if they occur.

Methods

A survey was conducted among 1353 attendees of international research integrity conferences. They were asked to score 60 research misbehaviors according to their views on and perceptions of the frequency of occurrence, preventability, impact on truth (validity), and impact on trust between scientists on 5-point scales. We expressed the aggregate level impact as the product of frequency scores and truth, trust and preventability scores, respectively. We ranked misbehaviors based on mean scores. Additionally, relevant demographic and professional background information was collected from participants.

Results

Response was 17% of those who were sent the invitational email and 33% of those who opened it. The rankings suggest that selective reporting, selective citing, and flaws in quality assurance and mentoring are viewed as the major problems of modern research. The “deadly sins” of fabrication and falsification ranked highest on the impact on truth but low to moderate on aggregate level impact on truth, due to their low estimated frequency. Plagiarism is thought to be common but to have little impact on truth although it ranked high on aggregate level impact on trust.

Conclusions

We designed a comprehensive list of 60 major and minor research misbehaviors. Our respondents were much more concerned over sloppy science than about scientific fraud (FFP). In the fostering of responsible conduct of research, we recommend to develop interventions that actively discourage the high ranking misbehaviors from our study.
This is important because science journals and leading scientific societies tend to focus on fraud and retractions as the leading causes of mistrust in science. This survey suggests—correctly in my opinion—that the real problems are far more serious.

Here are the top five problems according to the survey results.
  • Selectively cite to enhance your own findings or convictions
  • Insufficiently supervise or mentor junior coworkers
  • Not publish a valid “negative” study
  • Demand or accept an authorship for which one does not qualify
  • Selectively cite to please editors, reviewers, or colleagues
If these are the real problems then that puts the leading journals in an awkward position. So far, they have put the blame on scientists and pretended that they are not part of the problem. But if those five problems are the real reasons why scientists and the general public are losing faith in science then the journals themselves are partly to blame (e.g. ENCODE, Arsenic-Life). Science journals need to improve peer review and they need to punish authors and reviewers who allow publication of sloppy science.


Oldest Homo sapiens a “nothingburger”? Plus US health care policy approaches The End

0000-0002-8715-2896 Oldest Homo sapiens a “nothingburger”? Plus top journos blast secrecy on health care law   Posted June 16, 2017 by Tabitha M. Powledge in Uncategorized post-info AddThis Sharing Buttons above OLDEST HOMO SAPIENS? It’s

“The Perils of Public Outreach”

Julia Shaw is a forensic psychologist. She is currently a senior lecturer in criminology at the London South Bank University (London, UK). Shaw is concerned that we are creating a culture where public outreach is being unfairly attacked. Read her Scientific American post at: The Perils of Public Outreach.

Shaw's point is rather interesting. She believes that scientists who participate in public outreach are being unfairly criticized. Let's look closely at her argument.
What scientists write in academic publications is generally intended for a scientific community, full of nuance and precise language. Instead, what scientists say and write in public forums is intended for lay audiences, almost invariably losing nuance but gaining impact and social relevance. This makes statements made in public forums particularly ripe for attack.
I agree that what scientists say in public forums will often stimulate debate. But isn't that the point? Of course it's true that public statements are ripe for attack because that's often why you make public statements. I don't see this as a serious problem. If you can't stand the heat then stay out of the kitchen. I hope Shaw won't mind if I treat some of her public statements as ripe for attack. :-)

But there's another point that troubles me. Shaw claims that scientific publications are full of nuance and precise language. Presumably this makes them more accurate. Public outreach, on the other hand, is less precise and that makes it an easy target for attack.

I strongly disagree with her claim that the scientific literature is more accurate.
We find ourselves in a complicated space. On the one hand scientists are professionals, and want to keep the profession reputable. We want to make sure that ideas conveyed by scientists are accurate. On the other hand, if we censor the simplified or creative speech of scientists in public arenas, we have the potential to stifle important communication and debate.
There's no great danger that we are censoring public debate by attacking public statements. Shaw might be confused about the difference between rigorous healthy debate and censorship.

It's true that scientists are professionals but their attempt to keep the profession "reputable" often involves criticizing anyone who engages in public outreach especially if they are critical of the scientific literature. I don't think this is what Shaw means.

In my experience, the most serious attempts at censorship are not aimed at the style (e.g. lack of nuance) of those writing for a wide audience but instead are aimed at the forum they have chosen. For example, much of the criticism of scientific papers can be found on blogs. The authors of the papers often claim that the criticism is unfair— it should have been published in scientific journals and not in a public forum. That's what we saw when ENCODE was criticised, for example.

The point in this case is to claim that the scientific literature is "professional" and accurate but blogs aren't. Therefore the "attack" is unfair because it wasn't peer reviewed. That's a very real attempt at censorship.
Academic articles often take years to craft into something that colleagues and editors agree is worthy of being published. An interview for the media can take minutes, and is often an improvised attempt to convey a very complex issue on the spot.

If you put these two types of communications next to each other, “real” science is likely to slaughter popular science. It’s like taking a pen to a gunfight. And, it can be difficult for scientists who read each other’s comments in popular outlets to remember that an informal discussion of science should not, and cannot, stand on the same ground as a formal one.
This is complicated. I strongly disagree with the implication that scientific papers are well-crafted and accurate, especially when it comes to conclusions and interpretation of data. We have tons of examples of papers in reputable journals that make outrageous and inaccurate claims. Peer review isn't working.

When bloggers or journalists take on a recent paper it's very likely that the scientific paper will be the loser. The "real" science will not win the fight in spite of what Shaw might think.

I'm not sure if she is specifically referring to media interviews in this article. In my experience, when the authors of scientific papers are interviewed by the press they often reveal what we only suspected when we read the paper. We learn the real motives and conclusions from the press interview and not from the obtuse language in the paper. I think the informal communication is far more revealing, and more truthful, than the formal one.

Is Shaw worried about people who criticize what scientists say in press releases? Is she defending press releases because they are supposed to be less accurate and precise than the actual paper?
Scientists must be careful to not police other scientists in a way that makes critical discussion impossible, and that dissuades dissemination.

Only then can we ensure that we can have real impact and continue to make science accessible.
I'm confused. Are we supposed to stop criticizing scientific papers that make ridiculous claims? Is the scientific literature the ultimate example of precision, nuance, and accuracy compared to public forums? Are we supposed to avoid criticism of public statements by scientists because we need to cut them a bit of slack when they speak in public? That doesn't make sense.

I'm confused about what this scientist (Julia Shaw) is saying in a public forum. I hope she won't think I'm trying to censor her. :-)


What’s in Your Genome? Chapter 4: Pervasive Transcription

I'm working (slowly) on a book called What's in Your Genome?: 90% of your genome is junk! The first chapter is an introduction to genomes and DNA [What's in Your Genome? Chapter 1: Introducing Genomes ]. Chapter 2 is an overview of the human genome. It's a summary of known functional sequences and known junk DNA [What's in Your Genome? Chapter 2: The Big Picture]. Chapter 3 defines "genes" and describes protein-coding genes and alternative splicing [What's in Your Genome? Chapter 3: What Is a Gene?].

Chapter 4 is all about pervasive transcription and genes for functional noncoding RNAs.
Chapter 4: Pervasive Transcription
  • How much of the genome is transcribed?
  • How do we know about pervasive transcription?
  • Different kinds of noncoding RNAs
  •         Box 4-1: Long noncoding RNAs (lncRNAs)
  • Understanding transcription
  •         Box 4-2: Revisiting the Central Dogma
  • What the scientific papers don’t tell you
  •         Box 4-3: John Mattick proves his hypothesis?
  • On the origin of new genes
  • The biggest blow to junk?
  •         Box 4-4: How do you tell if it’s functional?
  • Biochemistry is messy
  • Evolution as a tinkerer
  •         Box 4-5: Dealing with junk RNA
  • Change your worldview


What’s in Your Genome? Chapter 3: What Is a Gene?

I'm working (slowly) on a book called What's in Your Genome?: 90% of your genome is junk! The first chapter is an introduction to genomes and DNA [What's in Your Genome? Chapter 1: Introducing Genomes ]. Chapter 2 is an overview of the human genome. It's a summary of known functional sequences and known junk DNA [What's in Your Genome? Chapter 2: The Big Picture]. Here's the TOC entry for Chapter 3: What Is a Gene?. The goal is to define "gene" and determine how many protein-coding genes are in the human genome. (Noncoding genes are described in the next chapter.)

Chapter 3: What Is a Gene?
  • Defining a gene
  •         Box 3-1: Philosophers and genes
  • Counting Genes
  • Misleading statements about the number of genes
  • Introns and the evolution of split genes
  • Introns are mostly junk
  • Alternative splicing
  •         Box 3-2: Competing databases
  • Alternative splicing and disease
  •         Box 3-3: The false logic of the argument from         complexity
  • Gene families and the birth & death of genes
  •         Box 3-4: Real orphans in the human genome
  • Different kinds of pseudogenes
  •         Box 3-5: Conserved pseudogenes and Ken Miller’s         argument against intelligent design
  • Are they really pseudogenes?
  • How accurate is the genome sequence?
  • The Central Dogma of Molecular Biology
  • ENCODE proposes a “new” definition of “gene”
  • What is noncoding DNA?
  • Dark matter

What’s in Your Genome? Chapter 2: The Big Picture

I'm working (slowly) on a book called What's in Your Genome?: 90% of your genome is junk! I thought I'd post the TOC for each chapter as I finish the first drafts. Here's chapter 2.

Chapter 2: The Big Picture
  • How much of the genome has been sequenced?
  • Whose genome was sequenced?
  • How many genes?
  • Pseudogenes
  • Regulatory sequences
  • Origins of replication
  • Centromeres
  • Telomeres
  • Scaffold Attachment regions (SARs)
  • Transposons
  • Viruses
  • Mitochondrial DNA (NumtS)
  • How much of our genome is functional?


What’s in Your Genome? Chapter 1: Introducing Genomes

I'm working (slowly) on a book called What's in Your Genome?: 90% of your genome is junk! I thought I'd post the TOC for each chapter as I finish the first drafts. Here's chapter 1.

Chapter 1: Introducing Genomes
  • The genome war
  • What is DNA?
  • Chromatin
  • How big is your genome?
  • Active genes?
  • What do you need to know?


The Edge question 2017

Every year John Brockman asks his stable of friends an interesting question. Brockman is a literary agent and most of the people who respond are clients of his. (I want to be one.) The question and responses are posted on his website Edge. This year's question is, "What scientific term or concept ought to be more widely known?"

This year, the introduction is more interesting than the responses. Here's part of what Brokman wrote,
Many people, even many scientists, have traditionally had a narrow view of science as controlled, replicated experiments performed in the laboratory—and as consisting quintessentially of physics, chemistry, and molecular biology. The essence of science is conveyed by its Latin etymology: scientia, meaning knowledge. The scientific method is simply that body of practices best suited for obtaining reliable knowledge. The practices vary among fields: the controlled laboratory experiment is possible in molecular biology, physics, and chemistry, but it is either impossible, immoral, or illegal in many other fields customarily considered sciences, including all of the historical sciences: astronomy, epidemiology, evolutionary biology, most of the earth sciences, and paleontology. If the scientific method can be defined as those practices best suited for obtaining knowledge in a particular field, then science itself is simply the body of knowledge obtained by those practices.

Science—that is, reliable methods for obtaining knowledge—is an essential part of psychology and the social sciences, especially economics, geography, history, and political science. Not just the broad observation-based and statistical methods of the historical sciences but also detailed techniques of the conventional sciences (such as genetics and molecular biology and animal behavior) are proving essential for tackling problems in the social sciences. Science is nothing more nor less than the most reliable way of gaining knowledge about anything, whether it be the human spirit, the role of great figures in history, or the structure of DNA.
Many people prefer other definitions of "science" but that's okay. As long as you make the effort to define your terms you can have productive discussion about epistemology and whether something counts as science or not.

Here's some of the answers ....

The Genetic Book of the Dead, Richard Dawkins (adaptation)
Reciprocal Altruism Margaret Levi (more adaptation)
Common Sense Jared Diamond (interesting)
Evolve Victoria Wyatt (an adaptationist perspective)
Affordances Daniel Dennett (???)
Neoteny Brian Christian (starts well, ends badly)
The Neural Code John Horgan (no end in sight for neurobiology)
Scientific Realism Rebecca Newberger Goldstein (interesting)
Life History Alison Gopnik (humans adapt)
Negativity Bias Michael Shermer (I have a negative bias)
Morphogenetic Field Robert Provine (interesting and confusing)
Trolley Problem Daniel Rockmore (I agree, it should be widely known & understood)
Determinism Jerry Coyne (I agree, but good luck convincing the public)
Type I and Type II Errors Phil Rosenzweig (excellent idea!)
Biosociation James Geary (hmmmm ....)
Uncertainty Lawrence Krauss (I agree but the public isn't ready for this)
Epigenetics Leo M. Chalupa (blah!)
DNA George Church (not for everyone)
Non Ergodic Stuart A. Kauffman (as clear as most of Kauffman's writings)
Natural Selection Jonathan B. Losos ("tinkerer" would be a better choice of words)
Gaia Hypothesis Hans Ulrich Obrist (I think public needs to know less about Gaia)
The Scientist Stuart Firestein (confusing)
Sexual Selection Rory Sutherland (sex and adaptation)
Mismatch Conditions Daniel Lieberman (adaptation gone wrong)
Isoplation Mismatch David C. Queller (adaptation makes new species)
The Transcriptome Andrés Roemer (confusing)
Somatic Evolution Itai Yanai (depends on your definition of evolution)
Homeostasis Martin Lercher (I disagree, homeostasis is over-rated)
Phylogeny Richard Prum (I agree with the concept)
Replicator Power Susan Blackmore (adaptation leads to design)
Phenotyic Plasticity Nicolas Baumard (adaptation for plasticity)
Maladaptation Aubrey de Grey (adaptive trade-offs look like maladaptation)
Exaptation W. Tecumseh Fitch (preparing for adaptation)
The Scientific Method Nigel Goldenfeld (ask good questions)

Nobody asked me to contribute an answer to Brockman's question but if they had I would have suggested "Random Genetic Drift." I think everyone needs to know about alternatives to adaptation and the real null hypothesis of evolutionary biology. It would have provided a nice contrast to some of the other answers.


Nature opposes misinformation (pot, kettle, black)

The lead editorial in last week's issue of Nature (Dec. 8, 2016) urges us to Take the time and effort to correct misinformation. The author (Phil Williamson) is a scientist whose major research interest is climate change and the issue he's addressing is climate change denial. That's a clear example of misinformation but there are other, more subtle, examples that also need attention. I like what he says in the opening paragraphs,

Most researchers who have tried to engage online with ill-informed journalists or pseudoscientists will be familiar with Brandolini’s law (also known as the Bullshit Asymmetry Principle): the amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it. Is it really worth taking the time and effort to challenge, correct and clarify articles that claim to be about science but in most cases seem to represent a political ideology?

I think it is. Challenging falsehoods and misrepresentation may not seem to have any immediate effect, but someone, somewhere, will hear or read our response. The target is not the peddler of nonsense, but those readers who have an open mind on scientific problems. A lie may be able to travel around the world before the truth has its shoes on, but an unchallenged untruth will never stop.
I've had a bit of experience trying to engage journalists who appear to be ill-informed. I've had little success in convincing them that their reporting leaves a lot to be desired.

I agree with Phil Williamson that challenging falsehoods and misrepresentation is absolutely necessary even if it has no immediate effect. Recently I posted a piece on the misrepresentations of the ENCODE results in 2007 and pointed a finger at Nature and their editors [The ENCODE publicity campaign of 2007]. They are responsible because they did not ensure that the main paper (Birney et al., 2007) was subjected to appropriate peer review. They are responsible because they promoted misrepresentations in their News article and they are responsible because they published a rather silly News & Views article that did little to correct the misrepresentations.

That was nine years ago. Nature never admitted they were partly to blame for misrepresenting the function of the human genome.

Pot, kettle, black

Is there a more effective way of challenging misrepresentation and getting corrections—or at least acknowledgement of error? Perhaps there is. Two Nature editorials from the week before addresses this very issue. The first one says Academia must resist political confirmation bias. I wish they had changed the online title to just read "Resist confirmation bias." That would have been more appropriate. Not all biases are political.

Here's an example of confirmation bias. Imagine you are convinced that humans are much more complex than all other species. You were disappointed to learn that we have about the same number of genes are many other species and even fewer than some plants. Imagine you are ignorant of modern evolutionary theory—you think that natural selection is all-powerful. You are very skeptical of junk DNA because it conflicts with your view of a complex genome that has been honed by selection.

Imagine you do some experiments provide evidence of pervasive transcription. Most of the transcripts are confined to one type of cell and they are present at very low levels. Maybe it's just junk RNA that has no function. You do a test to see whether the DNA that's complementary to those transcripts is conserved or not, knowing that sequence conservation is an important proxy for function. The test shows that the DNA is not conserved. What do you conclude?

The normal response is to conclude that you're probably dealing with junk RNA that has no biological function. You don't have proof but it seems very likely that this is the proper explanation of your result. But there's a problem. You are already convinced, for non-scientific reasons, that most of our genome should be functional. You are already convinced, for non-scientific reasons, that there's some sort of "missing complexity" in the human genome that needs to be explained.

This leads you to re-interpret the result and view it as some sort of confirmation of your biases. You construct an elaborate hypothetical scenario that gets around the lack of conservation problem and allows for massive amounts of biologically functional transcripts that aren't conserved but may be selected in the future. There must be something wrong with using conservation to identify function and rule out junk. You publish a paper saying,
Our data support these hypotheses, and we have generalized this idea over many different functional elements. The presence of conserved function encoded by conserved orthologous bases is a commonplace assumption in comparative genomics; our findings indicate that there could be a sizable set of functionally conserved but non-orthologous elements in the human genome, and these seem unconstrained across mammals.
This is what Birney et al. said in their 2007 paper. That's confirmation bias at work. This misresentation of their results and misrepresentation of their conclusions was pointed out many times in 2007 and afterward.

The second editorial in the Dec. 1 issue is: Post-publication criticism is crucial, but should be constructive. The subtitle carries the main message: "In an era of online discussion, debate must remain nuanced and courteous." Here's how Nature editors think criticism should be handled,
Anyone who finds flaws should seek corrections with diplomacy and humility. A gloating sense of ‘gotcha’ does not help to provide constructive criticism; some ill-considered phrases have caused lasting damage. But many scientists use their blogs for credible, restrained, nuanced criticism, often engaging the authors whose works are criticized.

Sharing and discussion of scientific work has changed drastically in a world of blogs, online repositories and Twitter. The fact remains, however, that self-correction is at the heart of science. Critics — curated or not — should be courteous, but criticism itself must be embraced.
That's a good description of how the criticism of Nature and the ENCODE authors was handled in 2007. Here's how I responded in 2007 [What is a gene, post-ENCODE?].
This is not news. We've known about this kind of data for 15 years and it's one of the reasons why many scientists over-estimated the number of humans genes in the decade leading up to the publication of the human genome sequence. The importance of the ENCODE project is that a significant fraction of the human genome has been analyzed in detail (1%) and that the group made some serious attempts to find out whether the transcripts really represent functional RNAs.

My initial impression is that they have failed to demonstrate that the rare transcripts of junk DNA are anything other than artifacts or accidents. It's still an open question as far as I'm concerned.
And here's how Mike White politely responded back in 2007 [Our Genomes, ENCODE, and Intelligent Design ],
Despite what you may read, there is still a lot of junk DNA. The ENOCDE project does not "sound the death-knell for junk DNA." Our genomes are filled with fossils of genetic parasites, inactive genes, and other low-complexity, very repetitive sequence, and it's extremely clear that most of this stuff no functional role. Much of this sequence may be transcribed, but remember that the ENCODE evidence for most of this transcription is indirect - their direct measurements only detected transcripts for ~14% of the regions they studied. Even if much of it is transcribed, this mainly suggests that it is not worth expending energy to actively repress this transcription, since there are so many other controls in place to deal with unwanted transcripts in the cell.
Here's third polite response from John Timmer [ENCODE finds the human genome to be an active place],
There seems to be three possible interpretations for all these extra transcripts. One is that, even though we haven't detected a biological function, and evolution doesn't conserve them, they are actually specifically functional. This would be the "there is no junk DNA" take on matters. The opposite extreme would be an "it's all junk" view of it. From this perspective, the starting and stopping of transcription is just an inherently noisy process and doesn't do humans enough harm to create a selective pressure to improve it.

Somewhere between the two would be the view that few of these extra transcripts are useful in themselves, but it's useful having them present on the collective level. Reasons could include anything ranging from excess RNA performing some sort of structural function through to the random transcripts being a rich source of new genes.

Personally, I fall into the "it's all junk" end of the spectrum. If almost all of these sequences are not conserved by evolution, and we haven't found a function for any of them yet, it's hard to see how the "none of it's junk" view can be maintained. There's also an absence of support for the intervening view, again because of a lack of evidence for actual utility. The genomes of closely related species have revealed very few genes added from non-coding DNA, and all of the structural RNA we've found has very specific sequence requirements. The all-junk view, in contrast, is consistent with current data. We've wondered for decades how transcription factors can act specifically and at long distances despite their relatively weak specificity for DNA. This data answers that question simply: they don't.
Did any of these polite considerate criticisms have an effect? Nope. Nada. Nothing.

We know this because five years later, in 2012, ENCODE published their complete analysis of functional regions of the human genome and they doubled down on the definition of "function." In the second stage of the publicity campaign, Nature and their editors went out of their way to ignore all previous criticism and actively promote the idea that most of our genome is functional and junk DNA is dead.

So I call "bullshit" when I read Nature editorials calling for constructive and polite criticism. By 2012, most bloggers realized that the only way to get attention when you are criticizing a big powerful group of scientists and their journalist friends is to be a little less polite. Here's how John Timmer responded to the second round of misrepresentation in 2012 [A bad precedent, repeated],
This brings us to the ENCODE project, which was set up to provide a comprehensive look at how the human genome behaves inside cells. Back in 2007, the consortium published its first results after having surveyed one percent of the human genome, and the results foreshadowed this past week's events. The first work largely looked at what parts of the genome were made into RNA, a key carrier of genetic information. But the ENCODE press materials performed a sleight-of-hand, indicating that anything made into RNA must have a noticeable impact on the organism: "the genome contains very little unused sequences; genes are just one of many types of DNA sequences that have a functional impact."

There was a small problem with this: we already knew it probably wasn't true. Transposons and dead viruses both produce RNAs that have no known function, and may be harmful in some contexts. So do copies of genes that are mutated into uselessness. If that weren't enough, just a few weeks later, researchers reported that genes that are otherwise shut down often produce short pieces of RNA that are then immediately digested.

So even as the paper was released, we already knew the ENCODE definition of "functional impact" was, at best, broad to the point of being meaningless. At worst, it was actively misleading.

But because these releases are such an important part of framing the discussion that follows in the popular press, the resulting coverage reflected ENCODE's spin on its results. If it was functional, it couldn't be junk. The concept of junk DNA was declared dead far and wide, all based on a set of findings that were perfectly consistent with it.

Four years later, ENCODE apparently decided to kill it again.

... ENCODE remains a phenomenally successful effort, one that will continue to pay dividends by accelerating basic science research for decades to come. And the issue of what constitutes junk DNA is likely to remain controversial—I expect we'll continue to find more individual pieces of it that perform useful functions, but the majority will remain evolutionary baggage that doesn't do enough harm for us to eliminate it. Since neither issue is likely to go away, it would probably be worth our time to consider how we might prevent a mess like this from happening again.

The ENCODE team itself bears a particular responsibility here. The scientists themselves should have known what the most critical part of the story was—the definition of "functional" and all the nuance and caveats involved in that—and made sure the press officers understood it. Those press officers knew they would play a key role in shaping the resulting coverage, and should have made sure they got this right. The team has now failed to do this twice.
But even that kind of response didn't get much attention. Here's how Dan Graur succeeded in Graur et al. (2013).
Here, we detail the many logical and methodological transgressions involved in assigning functionality to almost every nucleotide in the human genome. The ENCODE results were predicted by one of its authors to necessitate the rewriting of textbooks. We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.

... So, what have we learned from the efforts of 442 researchers consuming 288 million dollars? According to Eric Lander, a Human Genome Project luminary, ENCODE is the “Google Maps of the human genome” (Kolata 2012). We beg to differ, ENCODE is considerably worse than even Apple Maps. Evolutionary conservation may be frustratingly silent on the nature of the functions it highlights, but progress in understanding the functional significance of DNA sequences can only be achieved by not ignoring evolutionary principles.

High-throughput genomics and the centralization of science funding have enabled Big Science to generate “high-impact false positives” by the truckload (The PLoS Medicine Editors 2005; Platt et al. 2010; Anonymous 2012a, 2012b; MacArthur 2012; Moyer 2012). Those involved in Big Science will do well to remember the depressingly true popular maxim: “If it is too good to be true, it is too good to be true.”

We conclude that the ENCODE Consortium has, so far, failed to provide a compelling reason to abandon the prevailing understanding among evolutionary biologists according to which most of the human genome is devoid of function. The ENCODE results were predicted by one of its lead authors to necessitate the rewriting of textbooks (Pennisi 2012a, 2012b). We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.
That's what gets attention. It's not polite and it's not constructive.1 But it's proper criticism and it's fair. ENCODE and Nature didn't listen in 2007 so they needed to be hit over the head in a way that would force them to respond.

They did respond. In 2014 ENCODE basically retracted many of their claims, albeit, with lots of qualifiers (Kellis et al., 2014). Nature and their editors, on the other hand, have never admitted to their mistakes in 2007 and in 2012. The journal did, however, mention the ENCODE retraction in an editorial published on May 7, 2014 [ENCODE debate revived online]. Some editor wrote,
Kellis says that ENCODE isn't backing away from anything. The 80% claim, he says, was misunderstood and misreported. Roughly that proportion of the genome might be biochemically active, he explains, but some of that activity is undoubtedly meaningless, leaving unanswered the question of how much of it is really 'functional'. Kellis also argues that focusing on the portion of the genome that is shaped by natural selection can be misleading. For example, he says, genes that cause Alzheimer's disease or other late-in-life disorders may be largely immune to evolutionary pressure, but they are still definitely functional.
Do you see the irony? The ENCODE Consortium is now claiming their results were "misunderstood and misreported" in the media. This isn't true, the papers themselves are responsible for the misrepresentation [The truth about ENCODE]. But even if this accusation were true, the main culprit is Nature which blithely reports this criticism without realizing it's directed at them!

Nature editors ignore criticism and promote misrepresentation but they write editorials condemning others who do the same thing.

Hypocrites.

The Tone Wars

There's been a lot of push-back concerning frank and open criticism that's less than polite. It's led to the "Tone Wars." What it boils down to is this. Polite criticism is okay because we can ignore it but frank and harsh criticism in not the way scientists are supposed to behave because it's harder to ignore and it makes us feel guilty. Another part of the "Tone Wars" is to advocate ignoring blogs and only pay attention to criticism that's published in the scientific literature where it can be peer-reviewed. This complaint often comes from authors who have greatly benefited from PR campaigns in the popular press outside of the scientific literature.

Let's quote again from the Nature editorial on May 7, 2014—the one responding to the ENCODE retraction of their original claims.
In the social-media age, scientific disagreements can quickly become public — and vitriolic. A report from the ENCODE (Encyclopedia of DNA Elements) Project consortium proposes a framework for quantifying the functional parts of the human genome. It follows a controversial 2012 Nature paper by the same group that concluded that 80% of the genome is biochemically functional (Nature 489, 57–74; 2012). Dan Graur, who studies molecular evolutionary bioinformatics at the University of Houston in Texas and is a vocal ENCODE critic, weighed in on this latest report. ENCODE's “stupid claims” from 2012 have finally come to back to “bite them in the proverbial junk”, Graur wrote on his blog. The targets noticed. “Some people seek attention through hyperbole and mockery,” says the report's first author Manolis Kellis, a computer scientist at the Massachusetts Institute of Technology in Cambridge. “We should stay focused on the issues.”

This is just the latest skirmish in an ongoing battle. In a scathing 2013 article, Graur and co-authors argued that the ENCODE researchers had essentially ignored evolutionary evidence that suggests that only 2–15% of the genome was under pressure from natural selection (D. Graur et al. Genome Biol. Evol. 5, 578–590; 2013).
Of all the criticisms over the previous seven years (2007 - 2014), which one gets attention? Is it the polite, constructive criticisms published on so many blogs? Nope.

Hypocrites.


Image Credit: The image of the pot and kettle mocking each other is from a list of English Idioms at: The Pot Calling The Kettle Black.

1. An example of "constructive" criticism would be to call for a halt in funding the major ENCODE labs and for an apology from Nature.

Birney, E., Stamatoyannopoulos, J. A., Dutta, A., Guigó, R., Gingeras, T. R., Margulies, E. H., Weng, Z., Snyder, M., Dermitzakis, E. T., et al. (2007) Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project. Nature, 447:799-816. [doi: 10.1038/nature05874]

Graur, D., Zheng, Y., Price, N., Azevedo, R.B., Zufall, R.A., and Elhaik, E. (2013) On the immortality of television sets: “function” in the human genome according to the evolution-free gospel of ENCODE. Genome Biology and Evolution, 5:578-590. [doi: 10.1093/gbe/evt028]

Kellis, M., Wold, B., Snyder, M.P., Bernstein, B.E., Kundaje, A., Marinov, G.K., Ward, L.D., Birney, E., Crawford, G.E., and Dekker, J. (2014) Defining functional DNA elements in the human genome. Proceedings of the National Academy of Sciences, 111:6131-6138. [doi: 10.1073/pnas.1318948111]