Looking for falsified images in Alzheimer’s study

Charles Piller, for Science, highlights the work of Matthew Schrag, who uses image analysis to look for falsified data, recently scrutinizing a link between a protein and Alzheimer’s:

“So much in our field is not reproducible, so it’s a huge advantage to understand when data streams might not be reliable,” Schrag says. “Some of that’s going to happen reproducing data on the bench. But if it can happen in simpler, faster ways—such as image analysis—it should.” Eventually Schrag ran across the seminal Nature paper, the basis for many others. It, too, seemed to contain multiple doctored images.

Science asked two independent image analysts—Bik and Jana Christopher—to review Schrag’s findings about that paper and others by Lesné. They say some supposed manipulation might be digital artifacts that can occur inadvertently during image processing, a possibility Schrag concedes. But Bik found his conclusions compelling and sound. Christopher concurred about the many duplicated images and some markings suggesting cut-and-pasted Western blots flagged by Schrag. She also identified additional dubious blots and backgrounds he had missed.

Tags: , ,

Operative concepts

Gave a talk “The Good Species” yesterday (26 April 2022) to the HPS crowd at UniMelb. The discussion went a way I didn’t expect: classification…

New edited species book

So, what have I been doing for the Covid Lockdown. Many things. This is one of them. The CRC Press link is here, but I’ll…

Scientists with bad data

Tim Harford warns against bad data in science:

Some frauds seem comical. In the 1970s, a researcher named William Summerlin claimed to have found a way to prevent skin grafts from being rejected by the recipient. He demonstrated his results by showing a white mouse with a dark patch of fur, apparently a graft from a black mouse. It transpired that the dark patch had been coloured with a felt-tip pen. Yet academic fraud is no joke.

Tags: , ,

Is science the only way of knowing?

Most of us learned that science provides good answers to all sort of questions ranging from whether a certain drug is useful in treating COVID-19 to whether humans evolved from primitive apes. A more interesting question is whether there are any limitations to science or whether there are any other effective ways of knowing. The question is related to the charge of "scientism," which is often used as a pejorative term to describe those of us who think that science is the only way of knowing.

I've discussed these issue many times of this blog so I won't rehash all the arguments. Suffice to say that there are two definitions of science; the broad definition and the narrow one. The narrow definition says that science is merely the activity carried out by geologists, chemists, physicists, and biologists. Using this definition it would be silly to say that science is the only way of knowing. The broad definition can be roughly described as: science is a way of knowing that relies on evidence, logic (rationality), and healthy skepticism.

The broad definition is the one preferred by many philosophers and it goes something like this ...

Unfortunately neither "science" nor any other established term in the English language covers all the disciplines that are parts of this community of knowledge disciplines. For lack of a better term, I will call them "science(s) in the broad sense." (The German word "Wissenschaft," the closest translation of "science" into that language, has this wider meaning; that is, it includes all the academic specialties, including the humanities. So does the Latin "scientia.") Science in a broad sense seeks knowledge about nature (natural science), about ourselves (psychology and medicine), about our societies (social science and history), about our physical constructions (technological science), and about our thought construction (linguistics, literary studies, mathematics, and philosophy). (Philosophy, of course, is a science in this broad sense of the word.)

Sven Ove Hanson "Defining Pseudoscience and Science" in Philosophy of Pseudescience: Reconsidering the Demarcation Problem.

Clearly, scientific education ought to mean the implanting of a rational, sceptical, experimental habit of mind. It ought to mean acquiring a method – a method that can be used on any problem that one meets – and not simply piling up a lot of facts.

George Orwell

Using the broad definition, one can make a strong case that science is the only proven way of gaining knowledge. All other contenders are either trivial (mathematics), wrong (religion) or misguided (philosophy). So far, nobody that I know has been able to make a convincing case for any non-scientific way of knowing. Thus, I adopt as my working hypothesis the view that science is the only way of knowing.

Last year, Jerry Coyne revived the debate by posting an article about our favorite philosopher Maarten Boudry.1 Boudry also adopts the broad definition of science and agrees that there are no other ways of knowing [Scientism schmientism! Why there are no other ways of knowing apart from science (broadly construed)]. As I mentioned above, the debate is related to the charge of "scientism," which is often levelled against people like Boudry and Coyne (and me).

The debate over science as a way of knowing hasn't been settled. There are still lots of philosphers fighting a rearguard action to save philosophy and the humanities from the science invasion. Boudry and Massimo Pigliucci have put together a series of papers on the debate and it's a must-read for anyone who participates in this war. One of the defenders of philosophy in this book is Stephen Law, who is active on Facebook so you can engage in the debate there.

Stephen claims that there are two kinds of questions to which science cannot supply answers: moral questions and philosophical questions. Neither of those make any sense to me. Moral questions are essentially questions about the best way for societies to behave and the answers to those questions clearly depend on evidence and on observations about existing societies. As for philosophical questions, Law describes them like this,

On my view, philosophical questions are, for the most part, conceptual rather than scientific or empirical, and the methods of philosophy are, broadly speaking, conceptual rather than scientific or empirical.

Stephen Law recognizes the distinction between "questions" and "knowledge" and, while he defends philosophy as "valuable exercise," he admits that pure reason alone can't reveal reality.

So perhaps, there's at least this much right about scientism: armchair philosophical reflection alone can't reveal anything about reality outside of our own minds. However, as I say, that doesn't mean such methods are without value.

If you've read this far, then good for you! Read the ongoing debate between Jerry Coyne and Adam Gopnik [Are The Methods Used By Science The Only Ways Of Knowing?]. Now watch this lecture given by Jerry Coyne in India a few years ago to see if you can refute the idea that science is the only way of knowing.

1. That's Boudry on the right in a photo taken back in 2010 when he was just a graduate student attending a conference at the University of Toronto. He's with Stefaan Blanke. I also visited Maarten in Gent, Belgium a few years later.

Is science a social construct?

Richard Dawkins has written an essay for The Spectator in which he says,

"[Science is not] a social construct. It’s simply true. Or at least truth is real and science is the best way we have of finding it. ‘Alternative ways of knowing’ may be consoling, they may be sincere, they may be quaint, they may have a poetic or mythic beauty, but the one thing they are not is true. As well as being real, moreover, science has a crystalline, poetic beauty of its own.

The essay is not particularly provocative but it did provoke Jerry Coyne who pointed out that, "The profession of science" can be contrued as a social construct. In this sense Jerry is agreeing with his former supervisor, Richard Lewontin1 who wrote,

"Science is a social institution about which there is a great deal of misunderstanding, even among those who are part of it. We think that science is an institution, a set of methods, a set of people, a great body of knowledge that we call scientific, is somehow apart from the forces that rule our everyday lives and tha goven the structure of our society... The problems that science deals with, the ideas that it uses in investigating those problems, even the so-called scientific results that come out of scientific investigation, are all deeply influenced by predispositions that derive from the society in which we live. Scientists do not begin life as scientists after all, but as social beings immersed in a family, a state, a productive structure, and they view nature through a lens that has been molded by their social structure."

Coincidently, I just happened to be reading Science Fictions an excellent book by Stuart Ritchie who also believes that science is a social construct but he has a slighly different take on the matter.

"Science has cured diseases, mapped the brain, forcasted the climate, and split the atom; it's the best method we have of figuring out how the universe works and of bending it to our will. It is, in other words, our best way of moving towards the truth. Of course, we might never get there—a glance at history shows us hubristic it is to claim any facts as absolute or unchanging. For ratcheting our way towards better knowledge about the world, though, the methods of science is as good as it gets.

But we can't make progress withthose methods alone. It's not enough to make a solitary observation in your lab; you must also convince other scientists that you've discovered something real. This is where the social part comes. Philosophers have long discussed how important it is for scientists to show their fellow researchers how they came to their conclusions.

Dawkins, Coyne, Lewontin, and Ritchie are all right in different ways. Dawkins is talking about science as a way of knowing, although he restricts his definition of science to the natural sciences. The others are referring to the practice of science, or as Jerry Coyne puts it, the profession. It's true that the methods of science are the best way we have to get at the truth and it's true that the way of knowing is not a social construct in any meanigful sense.

Jerry Coyne is right to point out that the methods are employed by human scientists (he's also restricting the practice of science to scientists) and humans are fallible. In that sense, the enterprise of (natural) science is a social construct. Lewontin warns us that scientists have biases and prejudices and that may affect how they do science.

Ritchie makes a diffferent point by emphasizing that (natural) science is a collective endeavor and that "truth" often requires a consensus. That's the sense in which science is social. This is supposed to make science more robust, according to Ritchie, because real knowledge only emerges after carefull and skeptical scrutiny by other scientists. His book is mostly about how that process isn't working and why science is in big trouble. He's right about that.

I think it's important to distinguish between science as a way of knowing and the behavior and practice of scientists. The second one is affected by society and its flaws are well-known but the value of science as way of knowing can't be so easily dismissed.

1. The book is actually a series of lectures (The Massey Lectures) that Lewontin gave in Toronto (Ontario, Canada) in 1990. I attended those lectures.

Ill of the dead

I have found it necessary, in the course of this volume, to speak of the departed; for the misgovernment of the Royal Society has not…

On the importance of controls

When doing an exeriment, it's important to keep the number of variables to a minimum and it's important to have scientific controls. There are two types of controls. A negative control covers the possibility that you will get a signal by chance; for example, if you are testing an enzyme to see whether it degrades sugar then the negative control will be a tube with no enzyme. Some of the sugar may degrade spontaneoulsy and you need to know this. A positive control is when you deliberately add something that you know will give a positive result; for example, if you are doing a test to see if your sample contains protein then you want to add an extra sample that contains a known amount of protein to make sure all your reagents are working.

Lots of controls are more complicated than the examples I gave but the principle is important. It's true that some experiments don't appear to need the appropriate controls but that may be an illusion. The controls might still be necessary in order to properly interpret the results but they're not done because they are very difficult. This is often true of genomics experiments.

Consider the ENCODE experiments where a great effort was made to map RNA transcripts, transcription factor binding sites, and open chromatin domains. In order to interpet these results correctly, you need both positive and negative controls but the most important was the negative control. Here's how Sean Eddy describes the required control (Eddy 2013):

To clarify what noise means, I propose the Random Genome Project. Suppose we put a few million bases of entirely random synthetic DNA into a human cell, and do an ENCODE project on it. Will it be reproducibly transcribed into mRNA-like transcripts, reproducibly bound by DNA-binding proteins, and reproducibly wrapped around histones marked by specific chromatin modifications? I think yes.

... Even as a thought experiment, the Random Genome Project states a null hypothesis that has been largely absent from these discussions in genomics. It emphasizes that it is reasonable to expect reproducible biochemical activities ... in random unselected DNA.

This may be a case where creating the control isn't easy but we are reaching the stage where it may become necessary because stamp-collecting will only get you so far. Ford Doolittle has come up with a similar type of control to interpret the functional elements (FE) described by ENCODE (Doolittle, 2013):

Suppose that there had been (and probably, some day, there will be) ENCODE projects aimed at enumerating, by transcriptional and chromatin mapping, factor footprinting, and so forth, all of the FEs in the genomes of Takifugu and a lungfish, some small and large genomed amphibians (including several species of Plethodon), plants, and various protists. There are, I think, two possible general outcomes of this thought experiment, neither of which would give us clear license to abandon junk. The first outcome would be that FEs (estimated to be in the millions in our genome) turn out to be more or less constant in number, regardless of C-value—at least among similarly complex organisms. ... The second likely general outcome of my thought experiment would be that FEs as defined by ENCODE increase in number with C-value, regardless of apparent organismal complexity.

I've been thinking a lot lately about transcripts and alternative splicing. Massive numbers of RNAs are being identified in all kinds of tissues and all kinds of species now that the techniques have become routine. When multiple transcript variants from the same gene are identified they are usually interpreted as genuine examples of alternative splicing. The field needs controls. The negative control is similar to the one proposed by Sean Eddy but it's important to have a positive control, which in this case would be a well-characterized set of genes with real alternative splicing where the function of the splice variants has been demonstrated. If your RNA-Seq experiment fails to detect the known alternatively spliced genes then something is wrong with the experiment.

It's not easy to identify this set of genes; that's why I admire the effort made by a graduate student (soon to be Ph.D.) at the University of British Columbia, Shams Bhuiyan, who tried very hard to comb the literature to come up with some gold standards to serve as positive controls (Bhuiyan, 2018). His efforts were not very successful because there aren't very many of these genuine examples. This is a problem for the field of alternative splicing but most workers ignore it.

This brings me to a recent paper that caught my eye:

Uebbing, S., Gockley, J., Reilly, S.K., Kocher, A.A., Geller, E., Gandotra, N., Scharfe, C., Cotney, J. and Noonan, J.P. (2019) Massively parallel discovery of human-specific substitutions that alter neurodevelopmental enhancer activity. Proc. Natl. Acad. Sci. (USA) 118: e2007049118. [doi: 10.1073/pnas.2007049118]

Genetic changes that altered the function of gene regulatory elements have been implicated in the evolution of human traits such as the expansion of the cerebral cortex. However, identifying the particular changes that modified regulatory activity during human evolution remain challenging. Here we used massively parallel enhancer assays in neural stem cells to quantify the functional impact of >32,000 human-specific substitutions in >4,300 human accelerated regions (HARs) and human gain enhancers (HGEs), which include enhancers with novel activities in humans. We found that >30% of active HARs and HGEs exhibited differential activity between human and chimpanzee. We isolated the effects of human-specific substitutions from background genetic variation to identify the effects of genetic changes most relevant to human evolution. We found that substitutions interacted in both additive and nonadditive ways to modify enhancer function. Substitutions within HARs, which are highly constrained compared to HGEs, showed smaller effects on enhancer activity, suggesting that the impact of human-specific substitutions is buffered in enhancers with constrained ancestral functions. Our findings yield insight into how human-specific genetic changes altered enhancer function and provide a rich set of candidates for studies of regulatory evolution in humans.

This is a very complicated set of experiments using techniques that I'm not familiar with. I suspect that there are only a few hundred scientists in the entire world that can read this paper and understand exactly what was done and whether the experiments were performed correctly. I imagine that there are even fewer who can evaluate the results in the proper context.

The objective is to identify mutations in the human genome that are responsible for making us different from our ancestors, notably the common ancestor we share with chimps. The authors assume, correctly, that these differences are likely to reside in regulatory sequences. They focused on regions of the genome that have been previously identified as the sites of chromatin modifications and/or transcription factor binding sites. They then narrowed down the search by choosing only those sites that showed either accelerated changes in the human lineage (1,363 HARs) or increased enhancer activities in humans (3,027 HGEs).

All of these sites, plus their chimp counterparts, were linked to reporter genes and the constructs were assayed for their ability to drive transcription of the reporter gene in cultures of human neural stem cells. Those cells were chosen because the authors expect a lot of human-specific changes in brain cells as opposed to other tissues. (That's not a reasonable assumption and, furthermore, it looks like brain cells have a lot more spurious transcription than other cells (except for testes).)

They found that only 12% of their HARs were active in this assay and only 34% of HGEs were active. That's interesting but it doesn't tell us a lot; for example, it doesn't tell us whether any of these sites are biologically significant because we don't have the results of Sean Eddy's Random Genome Project to tell us how many of ENCODE's sites are significant. We know that some small fraction of random DNA sequences have enhancer activity and we know that this fraction increases when you select for stretches of DNA that are known to bind transcription factors. What that means is that many of these sites are not real regulatory sequences but we don't know which ones are real and which ones are spurious.

Next, they focused on those sites that showed differential expression of the reporter genes when you compared the chimp and human versions. About 3% of all HARs and 12% of all HGEs fell into this category. Then they looked at the specific nucleotide differences to see if they were responsible for the differential expression and they found some examples, but most of them were modest changes (less than 2-fold). Here's the conclusion:

We identified 424 HARs and HGEs with human-specific changes in enhancer activity in human neural stem cells, as well as individual sequence changes that contribute to those regulatory innovations. These findings now enable detailed experimental analyses of candidate loci underlying the evolution of the human cortex, including in humanized cellular models and humanized mice. Comprehensive studies of the HARs and HGEs we have uncovered here, both individually and in combination, will provide novel and fundamental insights into uniquely human features of the brain.

This is a typical ENCODE-type conclusion. It leaves all the hard work to others. But here's the rub. How many labs are willing to take one of those 424 candidates and devote money, graduate students, and post-docs, to finding out whether they are really regulatory sites? I bet there are very few because, like the rest of us, they are so skeptical of the result that they are unwilling to risk their careers on it.

The experiments conducted by Uebbing et al. lack proper controls. There are times when simple data collection experiments are justified and there are times when additional genomics survey experiments are useful but as we enter 2021 we need to recognize that those times are behind us. The time has come to sort the wheat from the chaff and that means calling a halt to publishing experiments that can't be meaningfully interpreted.

Image Credit: The control flowchart is from ErrantScience.com.

Bhuiyan, S.A., Ly, S., Phan, M., Huntington, B., Hogan, E., Liu, C.C., Liu, J. and Pavlidis, P. (2018) Systematic evaluation of isoform function in literature reports of alternative splicing. BMC Genomics 19: 637. [doi: 10.1186/s12864-018-5013-2]

Doolittle, W.F. (2013) Is junk DNA bunk? A critique of ENCODE. Proc. Natl. Acad. Sci. (USA) 110: 5294-5300. [doi: 10.1073/pnas.1221376110]

Eddy, S.R. (2013) The ENCODE project: missteps overshadowing a success. Current Biology 23: R259-R261. [10.1016/j.cub.2013.03.023]

3 Things We’ve Learned From NASA’s Mars InSight

Clouds drift over the dome-covered seismometer, known as SEIS, belonging to NASA's InSight lander, on Mars

Scientists are finding new mysteries since the geophysics mission landed two years ago.

Spinoff Highlights NASA Technology Paying Dividends in the US Economy

On May 28, 2020, UCLA's Dr. Tisha Wang (far left) and colleagues pose after testing a compressed-air version of the VITAL prototype

NASA's technology is at the forefront of space exploration, but it can also be applied here on Earth - from improving cellular networks to saving lives in the pandemic.