PLOS Update

In 2009, we launched PLOS Currents as an experimental platform for rapid communication of non-standard publications. A few communities embraced the experiment enthusiastically from the start, and the contributions of researchers who volunteered as editors

One Small Step for Preprints, One Giant Step Forward for Open Scientific Communications

Thanks to our recent partnership with bioRxiv, PLOS authors will have the choice of posting their submitted manuscript on the bioRxiv preprint server on May 1st. Preprints enable authors to accelerate the dissemination of their

Pursuing Preprints, Preventing Pandemics: How pre-peer-review posting can accelerate outbreak science | PLOS Research News

0000-0002-8715-2896 Preprints – scientific papers posted online prior to peer review – offer opportunities to share and discuss research rapidly and openly, and Michael Johansson, infectious disease expert and founder of the nonprofit Outbreak Science,

Retrospection and Direction: A Q&A with Peter Walter

0000-0002-8715-28960000-0001-7318-5892 “In our panel it is the stated and adhered to policy that we will not consider where a paper is published. Rather, in our evaluations we assess its real impact in a field. Change

Social Media, Preprints, and a Dinosaur Tooth

0000-0002-8715-2896Map showing the maximum extent of the Western Interior Seaway that split North America during the Late Cretaceous. The location of the new tooth from Mississippi is indicated by a red dot. Researchers have debated

Open Data Projects Win Wellcome Trust, NIH and HHMI Open Science Prize

0000-0002-8715-28960000-0001-7318-5892 “Scientists can do much more with their own data if things are shared publicly and shared publicly quickly in order to have potential for real world impact.” -Trevor Bedford, lead of the Open Science

I’m Excited! A Post Pre-Print-Posting-Powwow Post

I just got back from attending a meeting organized by a new group called ASAPbio whose mission is to promote the use of pre-prints in biology.

I should start by saying that I am a big believer in this mission. I have been working for two decades to convince biomedical researchers that the Internet can be more than a place to download PDFs from paywalled journal websites, and universal posting of pre-prints – or “immediate publication” as I think it should be known – is a crucial step towards the more effective use of the Internet in science communication. We should have done this 20 years ago, when the modern Internet was born, but better late than never.

There were reasons to be skeptical about this meeting. Change needs to happen on the ground not in conference halls – I have been to too many publishing meetings that involved a lot of great talks about the problems with publishing and how to fix them, but which didn’t amount to much because these calls weren’t translated into action. Second, the elite scientists, funders and publishers who formed the bulk of the invite-only ASAPbio attendees have generally been the least responsive to calls to reform biomedical publishing (I understand why this was the target group – while young, Internet-savvy scientists tend to be much more supportive in principle, they are reluctant to act because of fears about how it will affect their careers, and are looking towards the establishment to take the first steps). Finally, my new partner-in-crime Leslie Vosshall and I spent a lot of time and energy trying to rally support for pre-prints online leading up to the meeting, and it wasn’t like people were knocking down the doors to sign on to the cause.

However, I wouldn’t have kept at this for almost half my life it I wasn’t an eternal optimist, and I went into the meeting hoping, if not believing, that this time might be different. And I have to say I was pleasantly surprised. By the end of the meeting’s 24 hours it seemed like nearly everyone in attendance was sold on the idea that biomedical researchers should all post pre-prints of their work, and had already turned their attention to questions about how to do it. And there was a surprisingly little resistance to the idea that post-publication review of papers initially posted as pre-prints could, at least in principle, fulfill the functions that pre-publication review currently carries out. That’s not to say there weren’t concerns and even some objections – there were, as I will discuss below. But these were all dealt with to varying degrees, and there seemed to be a general attitude these concerns can be addressed, and did not constitute reasons not to proceed.

Honestly, I don’t think any new ideas emerged from the meeting. Everything that was discussed has been discussed and written about extensively before. But the purpose of the meeting was not to break new ground. Rather I think the organizers were trying to do three things (I’m projecting a bit here since I wasn’t one of the organizers):

  • To transfer knowledge from the small group of us who have been in the trenches of this movement to prominent members of the research community who are open to these ideas, but who hadn’t really ever given them much thought or attention
  • To make sure potential pitfalls and challenges of pre-prints were discussed. Although the meeting was dominated by members of the establishment, there were several young-PIs and postdocs, representatives of different fields and a few international participants, who raised a number of important issue and generally kept the meeting from becoming a self-congratulatory elite-fest.
  • To inspire everyone to act in tangible ways to promote pre-print use.

And I think the meeting was highly effective all three regards. For those of you who weren’t there and didn’t follow online or on video, here’s a rough summary of what happened (there are archived videos here).

The opening night was dominated by a keynote talk from Paul Ginsparg, who in 1991 started an online pre-print server for physics that is now the locus for the initial publishing of essentially all new work in physics, mathematics and some areas of computer science. Paul is a personal hero of mine – for what he did with arXiv and for just being a no bullshit advocate for sanity in science publishing – so I was bummed that he couldn’t make it person because of weather-related travel issues. But his appearance as a giant head on a giant screen by video-conference was a fitting representation for his giant place in pre-print history. His talk was very effective in squashing any of the typical gloom-and-doom about the end of quality science that often happens when pre-prints are discussed. A little bit of biology exceptionalism came up in the Q&A (“Yeah, it works for physics, but biology is different…”) but I thought Paul put most of those ideas to rest, especially the idea that all physics is done by giant groups working underground surrounded by large metal tubes.

The second day had two sessions, each structured around a series of a dozen or so five minute talks, followed by breakout sessions and then discussion. The morning focused on why people don’t use pre-prints – concerns about establishing priority, being able to publish in journals, getting jobs and funding – and how to address these concerns, while the afternoon sessions were about how to use pre-prints in evaluating papers and scientists and in finding and organizing published scientific information.

I can’t summarize everything that was discussed, but I have a lot of  thoughts on the meeting and where to go from here in no particular order:

I was surprised at how uncontroversial pre-prints were

Having watched the battles over Harold Varmus’ proposal to have biologists embrace pre-prints in 1999, and having taken infinite flak over the last 20 years for promoting a model of science communication based on immediate publication and post-publication peer review, I expected the idea that biologists should make their work initially available as pre-prints to be controversial. But it wasn’t. Essentially everyone at the meeting embraced the basic concept of pre-prints from the beginning, and we spent most of the meeting discussing details about how a pre-print system in biology can and should work, and how to build momentum for pre-print use.

I honestly don’t know how this happened. Pre-prints are close to invisible in biology (we didn’t really have a viable pre-print server until a year or so ago) and other recent efforts to promote pre-print usage in biology have been poorly received. There is lots of evidence from social media that most members of the community fall somewhere in the skeptical to hostile range when discussing pre-prints. Some of it is selection bias – people hostile to pre-prints weren’t likely to agree to come to a meeting on pre-prints that they (mostly) had to pay their own way to attend.

But I think it’s bigger than that. I think the publishing zeitgeist may have finally shifted. I’ve felt this way before, so I’m not sure I’m a good witness. But I think people are really ready for it this time. The signs were certainly there: after all Ron Vale, who organized ASAPbio, is no publishing radical – his publishing record is everything I’ve been trying to fight against for the last 20 years. But now he’s a convert, at least on pre-prints, and others are following suit. I don’t know whether it’s because all our work has finally paid off, or if it’s just time. The Internet has become so ingrained in our lives, maybe people finally realized how ridiculous it is that people all over the world could watch the ASAPbio meeting streaming live on their computers, but they have to wait months and months and months to be able to read about our latest science.

In the end I don’t really care why things seem to have changed. Even as I redouble my efforts to make sure this moment doesn’t elude us, I’m going to celebrate – this has been a long time coming.

Glamour journals remain a huge problem

One of the most shocking moments of this meeting came in a discussion right before the close about how to move forward to make pre-prints work. Marc Kirschner, a prominent cell biologist, made the suggestion that people at the meeting publish pre-prints of their papers at the time of submission so long as it is OK with the journal they plan to submit it to. I don’t think Kirschner was trying to set down some kind of abstract principle. Rather I think he was speaking to the reality that no matter how effectively we sell pre-prints, in the short run most scientists are still going to strive to put their work in the highest profile journals they can get them into; and we can make progress with pre-prints if we point out that a lot of journals people choose to publish in for other reasons allow them to post pre-prints and they should avail themselves of this opportunity.

This was the one time at the meeting where I lost my cool (a publishing meeting where I lose my cool only once is a first). It’s not that it surprises me that journals have this kind of hold on people. But I was still flabbergasted that after a meeting whose entire point was that it would be really good for science if people posted pre-prints, someone could suggest that we should give journals – not scientists – the power to decide whether pre-print posting is okay. And I couldn’t believe that people in the audience didn’t rise up in outrage at the most glaring and obvious example of how dysfunctional and toxic – one might even say dystopian – our relationship to journals is.

This is why I maintain my position – echoed by Vitek Tracz at the meeting, and endorsed by a handful of others – that science communication is never going to function optimally until we rid ourselves of the publish or reject paradigm employed by virtually all journals, and  until we and stop defining our success as scientists based on whether or not we could winkle our way into one of the uber-exclusive slots in glamorous journals. If anything is going to stop the move towards pre-prints, it’s going to be our proclivity for “glamor humping” (as blogger DrugMonkey has aptly dubbed this phenomenon). And if anything has the power to undermine the benefits of pre-prints, it’s if we allow this mentality to dominate in the post-journal world.

People have weird views of priority

One of the few new things I learned at this meeting is how obsessed a large number of people are with technical definitions of priority. We spent 30 minutes talking about whether pre-prints should count in establishing priority for discoveries. First of all, I can’t believe there’s any question about this – of course they should! But more importantly who thinks that questions of priority actually get decided by carefully scrutinizing who published what, when and on what date? It’s a lovely scholarly ideal to imagine that there’s some kind of court of science justice where hearings are held on every new idea or discovery, and a panel of judges looks at everything that’s been published or said about the idea is presented, and they then rule on who really was the first to publish, or present, the idea/discovery in a sufficiently complete form to get credit for it.

But I got news for all the people counting submission dates on the head of a pin – outside of patent cases, where such courts really do exist, at least in theory, that ain’t the way it works. True priority is constantly losing out in the real world, where who you are, where you work, where you publish and how you sell yourself are often far more important than submission or publication dates in determining who gets credit (and its trappings) for scientific advances.

Cell Press has a horrible, but kind of sane, policy on pre-prints

One of the things that I think a lot of people coming to the meeting didn’t realize is that many journals are perfectly fine with people posting pre-prints of articles that are being considered by the journal. Some, like eLife, PLOSPeerJ and Genetics actively encourage it. Others, like EMBOPNASScience and all Nature journals unambiguously allow pre-print posting. On the flip side, journals from the American Chemical Society and some other publishers will not accept papers if they were posted as pre-prints. And then there’s Cell.

Cell‘s policy is, on the surface, hard to parse:

If you have questions about whether posting a manuscript or data that you plan to submit to this journal on an openly available preprint server or poster repository would affect consideration, we encourage you to contact an editor so that we may provide more specific guidance. In many cases, posting will be possible.

Fortunately, Emilie Marcus, CEO of Cell Press and Editor-in-Chief of Cell, was at the meeting to explain it to us. Her response was, and I’m paraphrasing but I think I’m capturing it correctly, is that they are happy to publish papers initially posted as pre-prints so long as the information in the paper had not already been noticed by people in the field. In other words, it’s ok to post pre-prints so long as nobody noticed the pre-print. That is, they are rather unambiguously not endorsing the point of pre-prints, which is to get your work out to the community more quickly and effectively.

This is a pretty cynical policy. Cell clearly wants to get credit for being down with pre-prints without actually sanctioning them. But I actually found Marcus’s explanation of the policy to make sense, in a way. She views Cell as a publisher, and, as such, its role is to make information public. If that information has already been successfully conveyed by other means, then the role of publisher is no longer required.

This is obviously a quaint view – Cell is technically a publisher, but it’s more important role is as a selector of research that it deems to be interesting and important. So I think it’s more appropriate to look at this as a business decision. In refusing to help make pre-prints a reality, Elsevier and Cell Press are acting as if they believe pre-prints are a threat to their bottom line. And they’re right. Because if pre-prints become universal, who in their right mind is going to subscribe to Cell?

Maybe the other journals that endorse pre-prints are banking on the symbiosis between pre-prints and journals that exists in physics being extended to biomedicine. In questions after his talk Ginsparg said that ~80% of papers published in the arXiv are ultimately published in a peer-reviewed journal. And these journals are almost exclusively subscription based. So why don’t libraries cancel these subscriptions? The optimistic answer (for those who like journals) is that libraries want to support the services journals provide and are willing to pay for them even if they’re not providing access to the literature. This may be true. But the money in physics publishing is a drop in the bucket compared to biomedicine, and I just can’t see libraries continuing to spend millions of dollars per year on subscriptions to journals that provide paywalled access to content that is freely available elsewhere. I could be wrong, of course, but it seems like Elsevier, who for all their flaws clearly know how to make money, in this case agrees with me.

I don’t know what effect the Cell policy will have in the short run. I’d like to think people who are supportive of pre-prints will think twice before sending a paper to Cell in the future because of this policy (of course I’d like it if they never considered Cell in the first place, but who am I kidding). But I suspect this is going to be a drag on the growth of pre-prints — how big a drag, I don’t know, but it’s something we’re probably going to have to work around.

There are a lot of challenges in building a fair and effective pre-print system

The position of young scientists on pre-prints is interesting. On the one hand, they have never scienced without the Internet, and are accustomed to being able to get access to information easily and quickly. On the other hand, they are afraid that the kinds of changes we are pushing will make their lives more difficult, and will make many of the pathologies in the current system worse, especially those biased against them, worse. Even those who have no reservations about the pre-prints and/or post-publication review, don’t feel like they’re in a position to lead the charge.

This is one of the biggest challenges we have moving forward. I have no doubt that science communication systems build around immediate publication and post-publication review can be better for both science and scientists. But that doesn’t mean they automatically will be better. Indeed, I share many of other’s concerns about turning science into an even bigger popularity contest than it already is; about making it easier for powerful scientists to reinforce their positions and thwart their less powerful competitors; about increasing the potency of biases the myriad biases that poison training, hiring, promotion and funding; about making the process of receiving feedback on your work even less pleasant and uncollegial than it already is; and about increasing the incentives for scientists to prioritize glamour over doing rigorous, high-quality and durable work.

I will write more elsewhere about these issues and how I think we should try to address them. But it is of paramount importance that everybody who is trying to promote the move to pre-prints and beyond, and who is building systems to do this, be mindful of all these risks and do everything in their power to make sure the new systems work for everyone in science. We have to remember that for every bigshot who opposes pre-prints because they want to preserve their ability to publish in Cell, there are hundreds of scientists who just want to preserve their ability to do science. If this latter group doesn’t believe that pre-print posting is good for them, we will not only fail to convince them to join us on this path, but we run the serious risk of making science worse than it already is. And that would be a disaster.

Will attendees of the meeting practice what they preached

Much of the focus of the meeting organizers was on getting people who attended the meeting to sign on to a series of documents expressing various types of commitment to promoting pre-prints in biomedicine (you can see these on the ASAPbio site). These documents are fairly strong, and I will sign them. But I’m sick of pledges. I’ve been down this path too many times before. People come to meetings, they sign a document saying they do all sorts of great stuff, and then they forget about it.

The only thing that matters to me is making sure that the people who attended the meeting and who seemed really energized about making pre-prints work start to put this enthusiasm into practice immediately. I look forward to quick, concrete action from funders. But the immediate goal of the scientists at the meeting or who support its goals must be to start posting pre-prints. This is especially true of prominent, senior scientists. There were four Nobelists at the meeting, many members of national academies, and other A-list scientists. It’s a small number of people in the grand scheme of things, but if these scientists demonstrate that they are really committed to making pre-prints by starting to post pre-prints in the next week (I suspect that most people at this level have a paper under review at all time). I am confident that their commitment is genuine – indeed some have already posted pre-prints from their labs since the meeting ended yesterday.

Obviously we don’t want pre-prints to be the domain of the scientific 1%. But we have to start somewhere, and if people who have nothing to lose won’t lead the way, then it will never happen. But it seems like they actually are leading the way. There’s tons more hard work to do, but let’s not miss this opportunity. The rainbow unicorn is watching.

ArcLive Rainbow Unicorn

 

How Did That Make It Through Peer Review?

Image credit: AJC ajcann.wordpress.comThe title’s question is one I’ve heard asked many times over the years. It has been uttered by senior colleagues, grad students, amateurs, and just about everyone else, too. The query is usually raised in

Do preprints count for anything? Not according to Elife & G3 & some authors ..



Well, just got pointed to this paper: Metagenomic chromosome conformation capture (meta3C) unveils the diversity of chromosome organization in microorganisms | eLife by Martial Marbouty, Axel Cournac, Jean-François Flot, Hervé Marie-Nelly, Julien Mozziconacci, Romain Koszul.  Seems potentially really interesting.

It is similar in concept and in many aspects to a paper we published in PeerJ earlier in the year (see Beitel et al., 2014 Beitel CW, Froenicke L, Lang JM, Korf IF, Michelmore RW, Eisen JA, Darling AE. (2014) Strain- and plasmid-level deconvolution of a synthetic metagenome by sequencing proximity ligation products. PeerJ 2:e415 http://dx.doi.org/10.7717/peerj.415.

Yet despite the similarities to our paper and to another paper that was formally published around the time of ours, this new paper does not mention these other pieces of work any where in the introduction as having any type of "prior work" relevance.  Instead, they wait until late in their discussion:
Taking advantage of chromatin conformation capture data to address genomic questions is a dynamic field: while this paper was under review, two studies were released that also aimed at exploiting the physical contacts between DNA molecules to deconvolve genomes from controlled mixes of microorganisms (Beitel et al., 2014; Burton et al., 2014).
Clearly, what they are trying to do here is to claim that since they paper was submitted before these other two (including ours) was published, that they should get some sort of "priority" for their work.  Let's look at that in more detail.  Their paper was received May 9, 2014.  Our paper was published online May 27 and the other related paper by Burton et al. was published online May 22.  In general, if a paper on what your paper is about comes out just after you submit your paper, while your paper is still in review, the common, normal thing to be asked to do is to rewrite your paper to deal with the fact that you were, in essence, scooped.  But that does not really appear to be the case here.  They are treating this in a way as "oh look, some new papers came out at the last minute and we have commented on them."  The last minute would be in this case, 6 months before this new paper was accepted.  Seems like a long time to treat this as "ooh - a new paper came up that we will add a few comments about".

But - one could quibble about the ethics and policies of dealing with papers that were published after one submitted one's own paper.  From my experience, I have always had to do major rewrites to deal with such papers.  But maybe E-Life has different policies.  Who knows.  But that is where things get really annoying here.  This is because it was May 27 when our FINAL paper came out online at PeerJ. However, the preprint of the paper was published on February 27, more than two months before their paper was even submitted.  So does this mean that the authors of this new paper do not believe that preprints exist?  It is pretty clear on the web site for our paper that there is a preprint that was published earlier.  Given what they were working on - something directly related to what our preprint/paper was about, one would assume they would have seen it with a few simple Google searches.  Or a reviewer might have pointed them to it.  Maybe not.  I do not know.  But either way, our preprint was published long before their paper was submitted and therefore I believe they should have discussed it in more detail.

Is this a sign that some people believe preprints are nothing more than rumors?  I hope not.  Preprints are a great way to share research prior to the delays that can happen in peer review.  And in my opinion, preprints should count as prior research and be cited as such.  I note - the Burton group in their paper in G3 also did not reference our preprint in what I consider to be a reasonable manner.  They add some comments in their acknowledgements
While this manuscript was in preparation, a preprint describing a related method appeared in PeerJ PrePrints (Beitel et al. 2014a). Note added in proof: this preprint was subsequently published (Beitel et al. 2014b). 
Given that our preprint was published before their paper was submitted too, I believe that they also should have made more reference to it in their paper.   But again, I can only guess that both the Burton and the Marbouty group just do not see preprints as being respectable scientific objects.  That is a bad precedent to set and I think the wrong one too.  And it is a shame.  A preprint is a publication.  No - it is not peer reviewed.  But that does not mean it should not be considered part of the scientific literature in some way.  I note - this new paper from the Marbouty group seems really interesting.  Not sure I want to dig into it any deeper if they are going to play games with the timing of submission vs. published "papers" as part of how they are positioning themselves to be viewed as doing something novel.