Measuring the Impact of Altmetrics

Illustration: Jesse Lefkowitz

How do you measure the influence of a journal or scientist? Until recently that question was largely settled. For a journal, you could turn to the impact factor (or IF), which determines the relative importance of a journal within its field by looking at how many times its articles get cited in other journals relative to the total number of articles it publishes. PageRank (predating and loosely related to the famous PageRank algorithm used by the Google search engine) is a kind of IF measure that gives greater weight to journals with high impact; a similar measure is the Eigenfactor score created by the evolutionary biologist Carl T. Bergstrom. For an individual scientist, you could calculate his or her h-index (in which h of the scientist’s total number of papers have received at least h citations).

Lately, however, scholars have become increasingly disenchanted with these and similar bibliometric indicators that use such values as total number of articles published or total number of citations. They complain that traditional measures of scientific impact are too slow and too narrow to accurately reflect science in the Internet age.


Enter, then, the new field of article level metrics or, as it is increasingly known, altmetrics. This blend of alternative and metrics refers to tools based on bookmarks, links, blog posts, tweets, and other online measures that presumably indicate ways that readers have been influenced by an article—in short, how much “buzz” the paper is generating online. 


Extremely astute readers may recall an earlier column of mine [see “The Coming Data Deluge,” IEEE Spectrum, February 2011] that took note of researchers using “syndromic surveillance” to predict flu outbreaks based on an analysis of Google searches for flu-related terms. This is part of the emerging field of infodemiology (that is, information-based epidemiology), which is part of a broader field called infoveillance, the monitoring of online health information. If Google searches can show us the influence (no pun intended) of a flu virus on a population, why can’t we use similar online data to judge the influence of a researcher or a scientific article?


Much of the altmetric scholarship has focused on Twitter and what Gunther Eysenbach, a researcher at the University of Toronto, has called tweet metrics. Although the ­prudent neologism collector must be on guard against Twitter-based coinages [see “All A-Twitter,” Spectrum, October 2007] that are just silly (an adjective that can be rightfully applied to the vast majority of them), exceptions sometimes cry out to be made. To wit, I offer you the tweetation, a mash-up of tweet and citation that refers to a Twitter post that links to a scholarly article.


Another of Eysenbach’s creations is the TWn score, which measures the number of tweets within n days of publication. This is the basis of the twimpact factor .


Then there’s the tweeted half life (THL), which is the number of days after publication that it takes for an article to generate 50 percent of the tweetations that occur within a defined TWn period, say 30 days. If the article’s TW30 is 100—that is, it generated 100 tweets in its first 30 days—and it generated 35 tweets on day 0 (the publication date), 10 tweets on day 1, and 8 tweets on day 2, then its THL is 2, because it was on day 2 that it surpassed 50 tweets.


This is all part of what researchers are calling scientometrics 2.0, where data mining techniques are brought to bear on massive social media databases and other online storehouses to search for fresh indicators of scholarly impact. Will they replace traditional measures such as the impact factor? Almost certainly not. The goal is merely to drag the concept of scientific influence into a century characterized by the rapid dissemination of information and near-universal social media. Tweet on.