0
0
There are many ways to announce a scholarly discovery. Shouting “Eureka!” in the street. Making a podcast episode on Spotify. Buying a space in ”20 Minuten”. Tweeting. However, all these channels, and even talks at academic conferences, generally cannot guarantee that the results are correct. As you probably know, bad science can have severe consequences. Not only does it lay flawed foundations for future works, but it can also adversely impact the society through implications in consumer products and public policies. So how can we trust a research output?
When in doubt, ask an expert
Just like how we avoid fake news, one way is to read from reliable sources. For the academic community, these are the peer-reviewed publi- cations. The most common form is academic journals. The publishing process is similar to that of regular magazines, such as Polykum. Authors write an article and submit to a journal. An editor decides to accept or reject the article, sometimes after requesting the authors for revisions. Once it is accepted, the publisher publishes it in an issue, either in print or online. The difference to most normal magazines is how the editor decides on the merit of the article.
In academic publishing, the editor is generally not the most well-versed in the subject matter of the article. Therefore, she*he looks for experts to review it. Since presumably the authors are also experts, this process is called “peer review”. The editor usually asks the reviewers to comment not only on the correctness of the paper, but also on the presentation and understandability as well as the novelty and impact of the results. Based on these comments, the editor makes her*his decision.
However, the process is neither objective nor bullet-proof. Reviewers may not read the article thoroughly and overlook mistakes. While the methodology can be checked and the analysis can be verified, it is hard to confirm the integrity of the data. Editors may get unsuitable reviewers or ignore their recommendations. Additionally, there is no clear-cut threshold for the novelty of an article, and its impact may be judged best by the test of time. Therefore, the fact that the journals are peer-reviewed is often not enough. People also look at their reputations. Each field has its own set of renowned journals. However, ”Cell”, ”Nature”, and ”Science” are widely considered as prestigious, mainly based on a measure called the ‘impact factor’. The impact factor measures how often a publication in the given journal is cited by other researchers. Still, we need to take this with a grain of salt. Even these apex journals had to retract published articles that were later discovered to be erroneous or fraudulent. At the same time, many phenomenal results are published in lesser-known journals.
Other publishing venues
Besides journals, peer-reviewed publica- tions also include some books and, especially in computer science, some conferences. These special conferences have peer-reviewed selection processes similar to journals. The written reports of the talks there are later published in the proceedings. As the number of slots for talks are limited, some conferences can be very competitive, and publishing in these conferences is comparable to publishing in reputable journals. However, these reports are subject to space constraints and mostly considered as extended abstracts rather than full articles. Furthermore, reviewers for conferences usually have tighter deadlines and hence are only required to read the limited pages that will appear in the proceedings. As such, although counted as peer-reviewed publications, these conference proceedings may not be as rigorous. Authors are therefore encouraged to follow up with full journal versions.
The dark side of publishing
Nowadays, publications are not only a form of knowledge dissemination, but also a performance measure of researchers. However, just like other measures that tie to compensation, this influences how science is conducted. Candidates for research positions and funding are usually evaluated based on the quantity and quality of publications. This creates pressure to produce results that are publishable and “citable”. This may steer researchers towards topics that are more trendy or can guarantee results in a shorter term. The focus on “significant results” makes negative results less favoured or shunned completely. However, these results, if done correctly, could benefit other researchers by revealing problems or documenting methods that don’t work. Some researchers even resort to cheating, using fake data or honourable authorships. Therefore, it is crucial for institutions and funders to have a fair and suitable evaluation process that can recognise research excellence in a more holistic manner and discourage shortcuts.
Open access
Traditionally, readers have to pay fees to the journals to access scientific articles. This does not sound so strange, as we normally need to pay for the magazines we read. However, it has created a barrier to knowledge and hindered research of individuals with limited funding. As such, there has been an increasing discussion in recent years towards open access. Broadly speaking, this movement aims to make scientific knowledge freely accessible to the public. However, the publishing fee still needs to be paid by somebody. In order to move towards open-access, many journals instead charge the authors. This in turn creates other problems. For example, some publishers simply publish articles for payment, without a proper editorial process. On the positive side, with easy electronic publishing nowadays, there are free archiving services, such as arXiv, where authors in many fields can make preprints of their published papers available.
It is worth noting that in any model, the researchers who write and review the articles receive no compensation from the journals. At the same time, they may be charged even an excessive fee, either as readers or authors. This has led to the call for fair open access that supports non-profit journals controlled by the academic community.
Hung Hoang, 32,
is a doctoral student in Theoretical Computer Science. He hasn’t been able to play Dungeons
and Dragons for a long time, because a player keeps having paper deadlines.
