Assessing the Impact of Research: Overview

What is cited reference searching?

Cited references are the articles, books or other materials listed in a bibliography or as works cited in a particular publication. Because citation databases index each reference, it is possible to search these cited references. One can follow a particular cited reference, or cited author, forward in time to find more current articles that have also cited that author or work.

What are impact factors?

 
Impact Factors of journals can be found in Journal Citation Reports (JCR).  For more resources on journal rankings.

What are the top journals in my field?

 
JCR allows you to sort by several criteria, including Impact Factor, Rank, Total Cites, et.  See the journal rankings page for more resources.

Why use cited reference searching with caution?

• Impact factors may play too important role in hiring and tenure decisions
• Citation rates and impact factors vary widely from field to field and shouldn't be taken at face value, but considered relative to the field of research
• Citation rate may be based on a few prolific authors citing each other, including self citations
• Citation searching works better for journal articles than books
• Cross-disciplinary research may produce fewer citations
• Coverage of your particular field in the citation database may be weak
• The research may too recent and not widely known, like emerging fields
• Impact factors refer to the journal as a whole, not to individual papers
• The quality of the journal producing the citation
• Distribution of the citations over time might be more indicative of their importance than the immediacy of the impact factor
• There is a growing tendency of some researchers to go after topics likely to get into high-impact journals, which jeopardizes creativity, can skew the course or even slow the pace of science
• Some journals also cite articles in editorials, reviews news and other non-research articles to increase the number of cites and thus increase the impact factor of the journal
• There is no guarantee that every paper which ought to be cited will be cited. An un-cited author may be ahead of his peers. Mendel and his genetics work went unappreciated for years.
• It is not clear whether the number of times a paper is cited measures its actual quality.
• Some databases that calculate impact factors fail to incorporate publications including textbooks, handbooks and reference books.
• Certain disciplines have low numbers of journals and usage. Therefore, one should only compare journals or researchers within the same discipline.
• Review articles normally are cited more often and therefore can skew results.
• Self-citing may also skew results.
• Some resources used to calculate impact factors have inadequate international coverage.
• Editorial policies can artificially inflate an impact factor.

Adapted from here and here.