Skip to main content

The false myth of the rise in Italian self-citations, and the impressively positive effect of bibliometric evaluations on the increase of the impact of Italian research

Submitted by Mike Ciavarella on
research
roars
research
evaluation
bibliometry
italy
anvur

Title: The false myth of the rise in self-citations, and the impressively positive effect of bibliometric evaluations on the increase of the impact of Italian research

Authors: Pietro D'Antuono and Michele Ciavarella

Categories: physics.soc-ph math.ST stat.TH   Comments: 20 pages, 8 figures, 10 tables, in English and inItalian  License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

 https://arxiv.org/abs/1910.02948

\\

 It has recently been claimed by Baccini and coauthors that due to ANVUR’s bibliometric evaluations of individuals, departments, and universities, in Italy there has been a surge in self-citations in the last ten years, thus increasing the “inwardness” of Italian research more than has happened abroad. We have studied the database of Ioannidis et al. published on 12 August 2019 of the one hundred thousand most “highly cited” scientists, including about two thousand Italians, and we found that the problem of self-citations in relation to this scientific elite is not significant in Italy, while perhaps observing a small deviation in the low scores in the rankings. The effect indicated by Baccini et al. consequently, does not seem worrying for the scientific elite (we quantified it in 2% of the total of scientists of the “best” one hundred thousand), and is probably largely concentrated in the further less cited scientists. Evaluation agencies like ANVUR should probably exclude self-citations in future evaluations, for the noise introduced by the young researchers. The overall state of health of the Italian research system and the positive effect of the ANVUR assessments are demonstrated by the number of Italian researchers in the top one hundred thousand, which has increased by comparing the “career” databased of 22 years, with that of the “young” researchers in the “2017” database. Italy, looking at the elite researchers, not only is not the most indulgent in self-citations, but has shown the best improvements, proving that the introduction of ANVUR had a positive effect. Indeed, all countries apart from Italy have suffered a decline, even substantial (–20% on a national Japan scale), of the number of researchers present in the 2017 data sets compared to career data. Italy instead shows a +0.2% on a global basis and an impressive +11.53% on a national basis.

Data available at

https://drive.google.com/open?id=1sJjVusbJZYSjZVY4n9ckFhEA0G6u7iMQ

https://drive.google.com/open?id=1IG9AFcYZniudN4GI3mU54KaR-a3tLATD 

 

 

 

CORRESPONDENCE 
 15 OCTOBER 2019
 

Citation doping not for Italy’s elites

 
 

Pietro D’Antuono &

Michele Ciavarella

 
  •  
  •  
 PDF version

Factors other than citation doping could have contributed to the recent rise in the number of Italians among the 100,000 most highly cited scientists (see Nature http://doi.org/dcgj; 2019).

Of the 100,000 most highly cited scientists in the database compiled by John Ioannidis et al. (PLoS Biol. http://doi.org/gf6ckr; 2019), including some 2,000 Italians, we found that the proportion using self-citation to boost their research impact was probably only 2% (see P. D’Antuono and M. Ciavarella Preprint at https://arxiv.org/abs/1910.02948v1; 2019). The practice seems to be more common among early-career scientists who are otherwise less frequently cited.

ANVUR, the Italian agency for research evaluation (go.nature.com/2kwu5jj), should in our view exclude self-citations from future evaluations, to avoid this ‘noise’.

We consider that the jump in the number of Italians in the 100,000 most highly cited researchers is a symptom of the overall health of the Italian research system. It underscores the positive effect of introducing ANVUR in 2006.

Nature 574, 333 (2019)

Attachment Size
On the effect of self cites on the career ranking of the top 100 PD MC eng-ita3.pdf 4.17 MB
Mike Ciavarella

However, I would like to point out another problem. what I say is not entirely "soft".

1) in a "non-virtuous" university or in a "non-excellence" department, suppose you have an excellent researcher, perhaps among the top 1000 in the world: it may not be promoted or it may not be taken the best in the world from abroad

2) suppose instead in a "virtuous" university, and perhaps in "department of excellence". Top researchers who have won the title of virtuoso or excellence may NOT be rewarded because the commission of related funds is most likely elective and does not include them on an electoral basis.

Michele Ciavarella

Sun, 10/06/2019 - 08:17 Permalink
Mike Ciavarella

I am pleased to inform you that the Editor of Nature has accepted a commentary on this findings for the next issue of Nature.

I have also launched a petition on the "excellent researchers" in Italy at the link 

https://www.change.org/p/ministro-fioramonti-iniziativa-ricercatori-di-eccellenza

The idea is to have a next round of assessment of "excellent researchers" (for example as included in the list of the 100k best scientists published by Ioannidis et al in Plos Biology) and allocate directly some resources for their promotion on the italian university system should be interested.

Tue, 10/08/2019 - 20:22 Permalink