Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Bias against novelty in science
T2 - A cautionary tale for users of bibliometric indicators
AU - Wang, Jian
AU - Veugelers, Reinhilde
AU - Stephan, Paula
PY - 2017/10/31
Y1 - 2017/10/31
N2 - Research which explores unchartered waters has a high potential for major impact but also carries a higher uncertainty of having impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. Viewing scientific research as a combinatorial process, we measure novelty in science by examining whether a published paper makes first-time-ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to all Web of Science research articles published in 2001 across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distant) new combinations, deliver high gains to science: they are more likely to be a top 1% highly cited paper in the long run, to inspire follow-on highly cited research, and to be cited in a broader set of disciplines and in disciplines that are more distant from their “home” field. At the same time, novel research is also more risky, reflected by a higher variance in its citation performance. We also find strong evidence of delayed recognition of novel papers as novel papers are less likely to be top cited when using short time-windows. In addition, we find that novel research is significantly more highly cited in “foreign” fields but not in their “home” field. Finally, novel papers are published in journals with a lower Impact Factor, compared with non-novel papers, ceteris paribus. These findings suggest that science policy, in particular funding decisions which rely on bibliometric indicators based on short-term citation counts and Journal Impact Factors, may be biased against “high risk/high gain” novel research. The findings also caution against a mono-disciplinary approach in peer review to assess the true value of novel research.
AB - Research which explores unchartered waters has a high potential for major impact but also carries a higher uncertainty of having impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. Viewing scientific research as a combinatorial process, we measure novelty in science by examining whether a published paper makes first-time-ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to all Web of Science research articles published in 2001 across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distant) new combinations, deliver high gains to science: they are more likely to be a top 1% highly cited paper in the long run, to inspire follow-on highly cited research, and to be cited in a broader set of disciplines and in disciplines that are more distant from their “home” field. At the same time, novel research is also more risky, reflected by a higher variance in its citation performance. We also find strong evidence of delayed recognition of novel papers as novel papers are less likely to be top cited when using short time-windows. In addition, we find that novel research is significantly more highly cited in “foreign” fields but not in their “home” field. Finally, novel papers are published in journals with a lower Impact Factor, compared with non-novel papers, ceteris paribus. These findings suggest that science policy, in particular funding decisions which rely on bibliometric indicators based on short-term citation counts and Journal Impact Factors, may be biased against “high risk/high gain” novel research. The findings also caution against a mono-disciplinary approach in peer review to assess the true value of novel research.
KW - Bibliometrics
KW - Breakthrough research
KW - Evaluation
KW - Impact
KW - Novelty
U2 - 10.1016/j.respol.2017.06.006
DO - 10.1016/j.respol.2017.06.006
M3 - Journal article
AN - SCOPUS:85025428088
VL - 46
SP - 1416
EP - 1436
JO - Research Policy
JF - Research Policy
SN - 0048-7333
IS - 8
ER -