The problem of reproducibility
Every scientist and student working in the university lab has experienced the phenomenon of 'reproducibility', most often appearing in their lives as 'lack of reproducibility'. It works like this. You find an article from the lab of professor X at the glamorous university H published in the prestigious journal N describing a new technique that is necessary for the execution of your latest great idea (e.g. how to cure cancer). So you need to learn this technique. The way to do this is to reproduce the experiment described in the article in your laboratory with your own hands. You try to do it and… it doesn’t work (9 out of 10 times). Sometimes it takes weeks or months of endless repetitive trials before you can reproduce the published experiment. Sometimes, it never works. The result is wasted time and broken careers for scientists, and lots of wasted money for institutions and science funders.
Every experimental scientist could tell you a few horror stories or anecdotes about how "it did not work in the lab". Recent systematic studies conducted by the California biotech company Amgen confirmed these anecdotes. The Amgen team of about 100 scientists tried to reproduce the findings of 53 'landmark' articles in cancer research published in reputable journals. They managed to reproduce only 6 (about 10%). This shocking result was similar to the studies conducted by the German pharma company Bayer who managed to reproduce the results of 14 out of 67 published studies (about 21%).
Epithets like 'crazy', 'unbelievable', and 'impossible' pop up in the mind of every scientist when they read about these numbers, because science is not science without reproducibility. The reproducibility of published experiments is the foundation of science. No reproducibility – no science. If these numbers are true, or even half-true, it means there is something fundamentally wrong in today’s system of scientific research and education. At the practical level, the US government alone gives nearly $31 billion every year in science funding through the National Institute of Health, which is mainly distributed in research grants to academic scientists. The 20% reproducibility rate means that 80% of this money ($25 billion) is wasted. Even half of this number is too much.
These numbers also present a question about the current status of scientific publishing: what do we actually publish? Why has it become nearly impossible to reproduce many articles authored by academic labs from prestigious universities that have been published in reputable journals? Apparently, the current text-based format of scientific articles is not sufficient to effectively transfer knowledge of experimental methods, which become more and more complex as new technologies are introduced. Whenever possible, scientists prefer to learn experimental methods mainly by watching other scientists instead of just reading. This is because when shown how to perform the experiment, they can clearly see technical details that are difficult for describe and interpretate in text. Some call it 'tacit knowledge'.