Competence Network for Bibliometrics

This paper discusses drawbacks of the percentile rank method for citation impact normalization which have hitherto been neglected in the bibliometrics literature. The transformation of citation counts to percentile ranks changes ratio scale data into ordinal scale data, for which the notions of the ratio between two values and of the magnitude of a difference between two values are not defined – a substantial loss of information. This distorts citation data particularly severely because the differences between citation counts adjacent in order in publication sets are greater for more highly cited publications and because highly cited publications are more scarce than non-highly cited ones. [...]

We conduct an independent cluster validation study on published clustering solutions of a research testbed corpus, the Astro dataset of publication records from astronomy and astrophysics. We extend the dataset by collecting external validation data serving as proxies for the latent structure of the corpus. Specifically, we collect (1) grant funding information related to the publications, (2) data on topical special issues, (3) on specific journals’ internal topic classifications and (4) usage data from the main online bibliographic database of the discipline. The latter three types of data are newly introduced for the purpose of clustering validation and the rationale for using them for this task is set out.

By individually associating articles to basic or applied research, it is shown that basic articles are cited more frequently than applied ones. Dividing the subject categories of the Web of Science into a basic and an applied part, the mean field-normalization rate is referred to the applied or basic part depending on the research orientation of the paper analysed. By this approach, a distinct difference of the citations for the applied and basic parts of most subject categories is found. However, differences of the citation scores of applied and basic research organisations are found as well, but are less clear. The explanation is that applied and basic research organisations generally publish a mix of basic and applied articles. [...]

The present study is an evaluation of three frequently used institution name disambiguation systems. The Web of Science normalized institution names and Organization Enhanced system and the Scopus Affiliation ID system are tested against a complete, independent institution disambiguation system for a sample of German public sector research organizations. The independent system is used as the gold standard in the evaluations that we perform. We study the coverage of the disambiguation systems and, in particular, the differences in a number of commonly used bibliometric indicators. The key finding is that for the sample institutions, the studied systems provide bibliometric indicator values that have only a limited accuracy. [...]

This article reports the results of a study of the correctness of document type assignments in the commercial citation index database Web of Science (SCIE, SSCI, AHCI collections). The document type assignments for publication records are compared to those given on the official journal websites or in the publication full-texts for a random sample of 791 Web of Science records across the four document type categories articles, letters, reviews and others, according to the definitions of WoS. The proportion of incorrect assignments across document types and its influence on document specific normalized citations scores are analysed. It is found that document type data is correct in 94% of records. [...]

In this article I investigate the shortcomings of exact string match-based author self-citation detection methods. The contributions of this study are twofold. First, I apply a fuzzy string matching algorithm for selfcitation detection and benchmark this approach and other common methods of exclusively author namebased self-citation detection against a manually curated ground truth sample. Near full recall can be achieved with the proposed method while incurring only negligible precision loss. Second, I report some important observations from the results about the extent of latent self-citations and their characteristics and give an example of the effect of improved self-citation detection on the document level self-citation rate of [...].

Here we study how the lengths of the periods from submission to acceptance (review duration) and from acceptance to publication (publication delay) relate to research quality as operationalized by F1000Prime recommendations for a large dataset of publications from the life and health sciences. We find a statistically detectable relationship between shorter peer review duration and recommendations but its effect size is negligibly small.

This study examines how well an idiosyncratic authorship counting rule for co-authored publications recommended by the German Science Foundation (DFG) for medical schools and widely used in performance-based funding systems aligns with the empirical evidence. The DFG rule and two other co-author credit rules are compared with empirical data of percentage contribution statements of authors of co-authored papers in medicine.

The citations to open access preprint versions of papers and citations to papers in journals translated to English are not regularly counted in major proprietary citation index databases or in ordinary bibliometric research assessment even though they arguably reflect a true part of a work’s scientific impact. Here we explore the extent of these phenomena using Web of Science data.

In this talk I lay out which broad classes of bibliometric co-author "counting" methods exist, whether they are in agreement with empirical data and which class is used most in practice. So far unpublished new data on relative co-author contributions to papers will be analyzed for this purpose. The currently dominating method class is found to have a poor validity track record and explanations are sought to understand its paradoxical success.

In this talk, the interaction of algorithmic classification, semantic document vector representations, and discipline specific classification systems of science will be discussed, primarily conceptually, but including some preliminary empirical results of recent research. For science studies, a fine-grained, article-level, comprehensive classification (or taxonomic) system is highly desirable. Existing systems do not suffice because they are either on a journal level (WoS SCs, Scopus ASJC, Science Metrix Journal Classification) or discipline-specific (e.g. MeSH, MSC, PACS, JEL Classification, ...). Algorithmic, data-based solutions have been introduced that can provide such solutions, namely document clustering algorithms. [...]

Applying an ethnographic approach to the research and application of bibliometrics, this paper synthesizes the scope and relevance of scientometric work with actor-network centered approaches from qualitative social sciences. In doing so, I draw on data collected within the Competence Network for Bibliometrics (Kompetenznetzwerk Bibliometrie, KB) in Germany. As a network with 22 high-profile member organizations including Fraunhofer and Max-Planck institutes, KB follows the research output from every field of study, reveals its impact, provides metadata, and makes research visible across disciplinary boundaries. It links partner research institutions and stipulates new disciplinary dynamics in the field of bibliometrics and scientometrics.

A standard procedure in citation analysis is that all papers published in one year are assessed at the same later point in time, implicitly treating all publications as if they were published at the exact same date. This leads to systematic bias in favor of early-months publications and against late-months publications. This contribution analyses the size of this distortion on a large body of publications from all disciplines over citation windows of up to 15 years. It is found that early-month publications enjoy a substantial citation advantage, which arises from citations received in the first three years after publication. [...]