{"id":1445,"date":"2018-08-24T04:32:30","date_gmt":"2018-08-24T04:32:30","guid":{"rendered":"http:\/\/163.180.4.222\/lab\/?p=1445"},"modified":"2019-10-15T19:43:11","modified_gmt":"2019-10-15T10:43:11","slug":"no-more-excuses-for-non-reproducible-methods","status":"publish","type":"post","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=1445","title":{"rendered":"No more excuses for non-reproducible methods"},"content":{"rendered":"

 <\/p>\n

 <\/p>\n

(\uc6d0\ubb38<\/a>)<\/p>\n

 <\/p>\n

 <\/p>\n

Online technologies make it easy to share precise experimental protocols \u2014 and doing so is essential to modern science, says Lenny Teytelman.<\/h5>\n

 <\/p>\n

 <\/p>\n

\n

Here\u2019s a one-two punch to spark camaraderie among scientists. First, ask: \u201cHow long did it take to get your PhD?\u201d Then follow up with: \u201cHow long would it have taken if all your experiments had worked the first or second time?\u201d<\/p>\n

Part of the probable time difference is due to inexperience, but not all of it. News last month brought a powerful reminder that access to detailed methods can be essential for getting experiments to work. In 2013, the US$1.6-million Reproducibility Project: Cancer Biology set out to repeat key experiments from 50 high-profile cancer papers, and so assess the extent to which published results can be replicated.\u00a0Instead<\/a>, the project has decided to stop at 18 papers. One big reason for this was the difficulty of working out what exactly was done in the original experiments. Protocols \u2014 precise step-by-step recipes for repeating experiments \u2014 are missing from published research more often than not, and even the original researchers can have trouble pinpointing particulars years later.<\/p>\n

I sympathize with those who struggle to chase down these details, but I am glad that their efforts have generated such a buzz. It will accelerate a shift towards better reporting.<\/p>\n

Now should be springtime for methods sharing. Mobile-friendly, web-based technologies are maturing just as the need to improve reproducibility has gained widespread attention. A new era of more-efficient, more-confident science is ours to lose.<\/p>\n

My own obsession with sharing methods started with an all-too-common frustration. I spent the first year and a half of my postdoc working out a protocol for single-cell microscopy. Assiduous tinkering showed that subtle changes to sample preparation were crucial. Instead of 1 microlitre of a chemical, I needed 5. Instead of a 15-minute incubation, I needed an hour. Alas, the general technique had already been published, so I got no credit for the work. Anyone else using the published recipe would have been either getting misleading results, or sharing my frustration at having had to waste time discovering the necessary adjustments for themselves \u2014 hence my enthusiasm for a central place to update protocols and share tips.<\/p>\n

In 2012, two colleagues and I decided to launch exactly this kind of resource. It turns out that we were far from the first to recognize this need. When my PhD co-adviser realized I was serious about leaving my postdoc to pursue such a project, he connected me with a researcher who had tried something similar a dozen years earlier. That project, BioProtocol.com, raised a million dollars of venture capital to build a protocol repository. This was in the era of flip phones with green-and-black screens, and before online tools and ideals of sharing had surged. The company shut down during the dotcom bust, but the entrepreneurs retained a wealth of experience and insight, which they generously shared. Another venture, Protocol Online, was launched around the same time to organize disparate life-science protocols in a central database. Later efforts include OpenWetWare (a wikipedia-like site for sharing step-by-step protocols) and Protocol Exchange (a preprint server for protocols, hosted by the publisher of\u00a0Nature<\/i>).<\/p>\n

We launched\u00a0protocols.io<\/a>\u00a0(a company in which I own equity) in 2014. This open-access repository of science methods lets researchers create and modify protocols, update versions and share them, either with select collaborators or with everyone. The protocols are dynamic and interactive, rather than static PDFs, so researchers can take notes and track time on smartphones as they perform experiments. When they publish papers, they can get a persistent identifier (in the form of a digital object identifier, or DOI) for their protocol and add it to the methods section. More than 10,000 protocols have now been uploaded, attracting 100,000 views every month. We and the authors frequently receive e-mails from researchers thanking us for saving them time.<\/p>\n

One reason these efforts have taken off is that increased attention to problems with reproducibility has spurred initiatives with publishers, vendors, funders, and individual laboratories. Some 200 journals now instruct authors to link to protocols.io or a similar repository; reviewers have also started to push for this. And funders including the US-based Gordon and Betty Moore Foundation, Chan Zuckerberg Initiative and childhood-cancer charity Alex\u2019s Lemonade Stand Foundation either state explicitly in their guidelines, or plan to state, that methodological resources are to be shared in a public repository.<\/p>\n

But the gap between meticulous methods and adequate description remains. To fill it, efforts must start at the bench, well before results are ready to be written up. Lab members and lab heads should be on the lookout for tools that facilitate tracking, and be willing to give them a try. And decisions about how to document and share methods should be made when researchers are designing their experiments, not when they are writing their manuscripts.<\/p>\n

Of course, fully described, shared protocols will not fix everything in science. Knowing how much to document is a judgement call; some conditions will necessarily vary between labs. And even a detailed method will not produce reliable results if an experiment is vulnerable to artefacts, or if assays have not been thoroughly validated.<\/p>\n

Still, skimpy protocols stall science, and the scientific community must mobilize itself to do more. Writing \u201cwe used a slightly modified version of the protocol from PaperZ\u201d, whether or not the full protocol can be found there, is clearly easier than spelling out all the steps \u2014 but it\u2019s not good enough. One day soon, publishing a paper without a link to a useful protocol in a methods repository will seem as outdated as using a mobile phone that can\u2019t connect to the Internet.<\/p>\n

 <\/p>\n<\/div>\n

Nature<\/span>\u00a0560<\/strong>, 411 (2018)<\/p>\n

doi: 10.1038\/d41586-018-06008-w<\/div>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

    (\uc6d0\ubb38)     Online technologies make it easy to share precise experimental protocols \u2014 and doing so is essential to modern science, says(more…)<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[32,29],"tags":[6,3],"class_list":["post-1445","post","type-post","status-publish","format-standard","hentry","category-essays-on-science","category-lets-do-science","tag-essays-on-science","tag-lets-do-science"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":1480,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=1480","url_meta":{"origin":1445,"position":0},"title":"High-profile journals put to reproducibility test","author":"biochemistry","date":"August 28, 2018","format":false,"excerpt":"\u00a0 \u00a0 (\uc6d0\ubb38) \u00a0 \u00a0 Researchers replicated 62% of social-behaviour findings published in\u00a0Science\u00a0and\u00a0Nature\u00a0\u2014 a result matched almost exactly by a prediction market. \u00a0 \u00a0 A reproducibility effort has put high-profile journals under the spotlight by trying to replicate a slew of social-science results. In the work, published on 27 August\u2026","rel":"","context":"In "Essays on Science"","block_context":{"text":"Essays on Science","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3493,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=3493","url_meta":{"origin":1445,"position":1},"title":"Brazilian biomedical science faces reproducibility test","author":"biochemistry","date":"May 10, 2019","format":false,"excerpt":"\u00a0 \u00a0 Researchers at more than 60 Brazilian labs will assess the replicability of research by their country\u2019s scientists. \u00a0 An ambitious project to test the reproducibility of biomedical experiments by Brazilian scientists is about to get under way. The Brazilian Reproducibility Initiative was launched last year by researchers at\u2026","rel":"","context":"In "Let's Do Biology!"","block_context":{"text":"Let's Do Biology!","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=33"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2237,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=2237","url_meta":{"origin":1445,"position":2},"title":"Industry is more alarmed about reproducibility than academia","author":"biochemistry","date":"December 3, 2018","format":false,"excerpt":"\u00a0 \u00a0 The reproducibility crisis in biomedical science seems to have alarmed industry more than the academic community (see\u00a0C. G. Begley and L. M. Ellis\u00a0Nature\u00a0483, 531\u2013533; 2012). In our view, this is because they have different yardsticks for success in research. Despite the advent of important new therapeutics, the number\u2026","rel":"","context":"In "Essays on Science"","block_context":{"text":"Essays on Science","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3627,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=3627","url_meta":{"origin":1445,"position":3},"title":"Reproducibility trial publishes two conclusions\ufeff for one paper","author":"biochemistry","date":"June 5, 2019","format":false,"excerpt":"\u00a0 \u00a0 The\u00a0British\u00a0Journal of Anaesthesia\u2019s unusual experiment is designed to broaden replicability efforts beyond just methods and results. \u00a0 \u00a0 Anaesthesia has been linked to delirium and death in older patients.Credit: BSIP\/UIG via Getty \u00a0 \u00a0 How deeply an anaesthetist should sedate an elderly person when they have surgery is\u2026","rel":"","context":"In "Essays on Science"","block_context":{"text":"Essays on Science","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":402,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=402","url_meta":{"origin":1445,"position":4},"title":"Write fiction to discover something new in your research","author":"biochemistry","date":"May 30, 2018","format":false,"excerpt":"\u00a0 \u00a0 (\uc6d0\ubb38) \u00a0 \u00a0 Creative writing can help you to approach your science from a completely different perspective \u2014 and boost its impact, says Amanda C. Niehaus. \u00a0 \u00a0 Credit: Getty \u00a0 In the final month of my Australian Research Council fellowship at the University of Queensland in Brisbane,\u2026","rel":"","context":"In "Essays on Science"","block_context":{"text":"Essays on Science","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3166,"url":"https:\/\/biochemistry.khu.ac.kr\/lab\/?p=3166","url_meta":{"origin":1445,"position":5},"title":"Duke University\u2019s huge misconduct fine is a reminder to reward rigour","author":"biochemistry","date":"April 3, 2019","format":false,"excerpt":"\u00a0 \u00a0 US$112.5-million settlement concerning fraudulent data is a casualty of a culture that prizes impact over robustness, says Arturo Casadevall. \u00a0 \u00a0 Last week, Duke University announced it would pay the US government US$112.5 million to settle claims that fraudulent data were used in dozens of research-grant applications. This\u2026","rel":"","context":"In "Essays on Science"","block_context":{"text":"Essays on Science","link":"https:\/\/biochemistry.khu.ac.kr\/lab\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":false,"jetpack_shortlink":"https:\/\/wp.me\/p9Xo1j-nj","_links":{"self":[{"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/posts\/1445","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1445"}],"version-history":[{"count":1,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/posts\/1445\/revisions"}],"predecessor-version":[{"id":4435,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=\/wp\/v2\/posts\/1445\/revisions\/4435"}],"wp:attachment":[{"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1445"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1445"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/biochemistry.khu.ac.kr\/lab\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1445"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}