Skip to Main Content

Research Impact Metrics

Assessing the impact of research and creative works, including citation metrics and citation analysis.

Measuring Impact: Caveats

This toolkit provides information to scholars looking to measure the impact of their research.  Anyone using this guide should be aware that impact is measured differently in different disciplines and research areas. Caveats to note:

  • Impact, productivity, and performance are complex.  Relying on one single number or factor to assess any of these is problematic and very likely to be misleading.

  • Most of the metrics in this toolkit are based on academics’ use of research.  Factors such as community impact or clinical impact are much more difficult to determine and, arguably, just as important.

Author Metrics: H-Index

See our guide on calculating your H-index.

Author-level metrics attempt to measure the impact or reach of an individual’s entire body of work.  The Hirsch-index, or h-index, is one such measure, which is calculated in Scopus, Web of Science, and can be calculated using Google Scholar.  It favors those who have been publishing for a long period of time.

  • Strengths: The h-index looks at an individual’s research contributions (not the average citation numbers of an entire journal over time).
  • Weaknesses:
    • The h-index favors authors who have published for a long period of time.  In an aim to correct for this, compare the h-indices of faculty in the same discipline who have been publishing for approximately the same number of years. This is still problematic, however, as you cannot determine or know future citation numbers or the value of particular research studies in the coming years.
    • “By definition, the h-index cannot exceed the number of publications. Thus it puts small but highly-cited paper sets at a disadvantage (‘small is not beautiful’).” (Glanzel, 2006).  For example, a person who published 2 articles, each cited over 1,000 times, can only have an h-index of 2.
    • “..the h-index is certainly useful for identifying outstanding performance but it seems to fail in assessing fair and good performance” (Ibid.).
  • Considerations:
    • Keep in mind that what is considered a “high” h-index varies widely across disciplines, and even within sub-disciplines.  A high h-index in engineering may be a much larger number than a high h-index in kinesiology. 
    • The h-index is not a precise number.  Depending on a number of factors—such as number of journals indexed in a database, how a database handles duplicate records, if the article has been cited in journals indexed by a database—the h-index for the same author will likely be different in all three sources for the h-index (Scopus, WoS, Google Scholar).
    • Researchers should provide the source of their h-index (Google Scholar, Scopus, etc.). The majority of researchers will have several different h-indices depending on the source used to calculate each h-index, because all sources are incomplete and/or inaccurate to some degree.

W. Glanzel (2006), On the opportunities and limitations of the H-index. Science Focus 1, 10-11.

Journal Metrics: Journal Impact Factor

See our guide on calculating Journal Impact Factor.

The journal impact factor (IF) attempts to measure the average reach or value of all journal articles published in the journal, based on the number of times their journal’s articles are cited.  IFs are found in the Journal Citation Reports database.

  • Strengths:
    • Journal impact factors are recognized in many disciplines and are often used by administrators when considering research impact.
    • Publishing in a journal with a high IF can help new researchers predict their value to the field—their work might not have had much time to be cited, but if accepted to a highly-cited journal, this bodes well for their future impact.
  • Weaknesses
    • Journal impact factors don’t measure an individual’s research impact, nor a particular article’s impact.  They attempt to measure a journal’s overall impact.
    • A new journal will not have a journal impact factor and an IF may be low for the first few years of publication.  This does not necessarily mean that it is a poor or inferior journal.
    • Journal impact factors can be, and sometimes are, manipulated to inflate a journal’s IF.  One example of a well-known manipulation is if a journal publishes a review issue each year.  Doing so creates one citation for each article, but not because an article is being referenced by a new study.  Another example is the inclusion of same-journal citations (or recommendations from editors to add same-journal citations to a lit review) for very little reason other than to boost that journal’s citation numbers.
  • Considerations
    • Impact factors vary widely by discipline.  The journal with the highest IF in biology may have a much higher IF number than the journal with the highest IF in kinesiology.  
    • For the social sciences, not all disciplines’ journals are included in Journal Citation Reports, so an IF for a particular journal may not be available. Journal impact factors are not measured for journals in the humanities because Journal Citation Reports does not include them.

Journal Metrics: SCImago Journal Ranking

SCImago journal rankings look at the number of citations received by a particular journal as well as the "prestige" of the journals in which the citations appear.

This is a free journal-ranking tool, as opposed to the Journal Citation Reports, which is available by subscription only.

The Technical University of Eindhoven in the Netherlands has a comparison of journals by impact factor and by SCImago journal ranking. Sort by IF or SCImago rank by clicking on that header in the table.

Learn more in this article: Matthew E. Falagas, et al. (2008). "Comparison of SCImago Journal Rank Indicator with Journal Impact Factor." The FASEB Journal 22 (8): 2623-2628. doi: 10.1096/fj.08-107938. 

Article Metrics, Alternative Metrics, Altmetrics

This is an attempt to show the value of a particular article by times cited, as well as other factors.

  • This is a growing field in bibliometrics and scientometrics.  Expect to hear more about article-level metrics (ALMs) in the coming years.
  • In addition to times cited, the number of downloads, times bookmarked, and social-media mentions may be notable as there may be a relationship between these numbers and the number of times an article is cited in the academic literature in the future.  This might help new researchers guage or predict their impact. They may also be useful in measuring community impact.

Some publishers (e.g. Elsevier), databases (e.g. ScienceDirect), and journals (e.g. PLOS ONE) provide Altmetrics data alongside with the article information. TRACE, the repository for open access publications at the University of Tennessee, also provides Altmetrics data for articles/documents submitted to it, in the form of number of downloads. 

Click here to see the Altmetrics data of several sample articles.

Monograph Metrics

  • How many libraries have purchased copies of the book? (WorldCat will help you find an estimate of this number.)
  • Has the book been reprinted, reissued, or has an updated edition been published?
  • Did major journals or other publications review the book?
  • Using Google Scholar, how many times has the book been cited?  (This may not be a complete or accurate count.)
  • Can you find any additional citations using Google Books?
  • Did the book win any major awards?