Impact Factor : Pros and Cons
By - 12/29/2003
(1 vote)
Javed Ali*, Alka ahuja, Sanjula Baboota and R. K. Khar
Department of Pharmaceutics, Faculty of Pharmacy, Jamia Hamdard, Hamdard Nagar, New Delhi-110062. INDIA .*For correspondance: javedaali@yahoo.com
In recent years the Institute of Scientific Information Journal Citation Reports impact factor has changed from an obscure bibliometric indicator to become the chief quantitative measure of the quality of a journal, its published papers, the scientists who wrote those papers and even the institutes they work within. This article looks at the use and abuse of the impact factor, how it should and should not be used.
Introduction
The impact factor is one of the three standardized measures created by the Institute of Scientific Information (ISI) which can be used to measure the way a journal receives citations to its articles over time (1,2). These characteristics form the basis of the ISI indicators impact factor, immediacy index and cited half-life.
The journal impact factor is a measure of the frequency with which the "average article" in a journal has been cited in a particular year. The impact factor will help you evaluate a journal’s relative importance, especially when you compare it to others in the same field. The impact factor is calculated by dividing the number of current citations to articles published in the two previous years by the total number of articles published in that period. The impact factor is useful in clarifying the significance of absolute (or total) citation frequencies. It eliminates some of the bias of such counts which favour large journals over small ones, or frequently issued journals over less frequently issued ones, and of older journals over newer ones. In the latter case such journals have a larger citable body of literature than smaller or younger journals (3,4).
Calculation of Journal Impact factor
Journal Impact Factor is a calculation of the ratio of number of articles published in a journal in the past two years and the number of times the articles in the journal have been cited. For example:
|
Cites in 2001 to articles published in:
|
2000= | 9556 | Number of articles published in: | 2000= | 379 | |
| 1999= | 12504 | 1999= | 380 | |||
| 2000+99= | 22060 | 2000+99= | 759 | |||
|
Calculation:
|
Cites to recent articles/Number of recent articles = 22060/759 = 29.065 | |||||
Roughly speaking, higher the Impact Factor, more often are the articles from a journal cited.
The journal Immediacy Index is a measure of how quickly the "average article" in a journal is cited. The Immediacy Index will tell you how often articles published in a journal are cited within the same year. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year. The Immediacy Index is useful in comparing how quickly journals are cited. Because it is a per-article average, the Immediacy Index tends to discount the advantage of large journals over small ones. However, frequently issued journals may have an advantage here, because an article published early in the year has a better chance of being cited than one published later in the year. For comparing journals specializing in cutting-edge research, the Immediacy Index can provide a useful perspective (5).
The cited half-life is the number of publication years from the current year which account for 50% of current citations received. A higher or lower cited half-life does not imply any particular value for a journal. For instance, a primary research journal might have a longer cited half-life than a journal that provides rapid communication of current information. Cited Half-Life may be useful to assist in collection management and archiving decisions. Dramatic changes in Cited Half-Life’s over time may indicate a change in a journal’s format. Studying the half-life data of the journals in a comparative study may indicate differences in format and publication history (6)
Variability in the Impact Factors
The value of the impact factor is affected by sociological and statistical factors. Sociological factors include the subject area of the journal, the type of journal and average number of authors per paper Statistical factors include the size of the journal and the citation measurement window.
1. Subjectivity
In general, fundamental and pure subject areas have higher average impact factor than specialized or applied ones. The variation is so significant that the top journal in one field may have an impact factor lower than the bottom journal in another area (7).
In general, fundamental and pure subject areas have higher average impact factor than specialized or applied ones. The variation is so significant that the top journal in one field may have an impact factor lower than the bottom journal in another area (7).
Closely connected to subject area variation is the phenomenon of multiple authorship. The average number of collaborators on a paper varies according to subject area, from social sciences (with about two authors per paper) to life sciences (where there are over four). Comparisons of impact factors should only be made for journals in the same subject area.
2. Article and Journal Type
A short or rapid publication journal will have greater immediacy but a lower cited half-life. As a consequence, a large proportion of the citations it receives will tend to fall within the two-year window of impact factor. In contrast, the full paper journal will have a citation peak around three years after publication, and therefore a lower immediacy than the rapid or short paper journal. It will also have a gentler decline after its peak, and consequently a larger cited half-life. The proportion of citations that fall within the two-year window will be smaller as a result of different curve shape and the impact factor of such a journal will tend to be smaller than its rapid or short paper relative. In case of a review journal, the immediacy index relative to other measures is very low, citations slowly rising to peak many years after publication. The cited half-life is also correspondingly long as the citations decline equally slowly after the peak. So, given that the impact factor measures differing proportions of citations for different journal types or journals with different mixes of article types (8-10).
A short or rapid publication journal will have greater immediacy but a lower cited half-life. As a consequence, a large proportion of the citations it receives will tend to fall within the two-year window of impact factor. In contrast, the full paper journal will have a citation peak around three years after publication, and therefore a lower immediacy than the rapid or short paper journal. It will also have a gentler decline after its peak, and consequently a larger cited half-life. The proportion of citations that fall within the two-year window will be smaller as a result of different curve shape and the impact factor of such a journal will tend to be smaller than its rapid or short paper relative. In case of a review journal, the immediacy index relative to other measures is very low, citations slowly rising to peak many years after publication. The cited half-life is also correspondingly long as the citations decline equally slowly after the peak. So, given that the impact factor measures differing proportions of citations for different journal types or journals with different mixes of article types (8-10).
3. Size Matters
As the impact factor is an average value, it also shows variation due to statistical effects. These relate to the number of items being averaged, that is the size of the journal in terms of articles published per annum or the size of the measurement window (which for the standard or JCR impact factor is two years , in fact, a one year citing window and a two years cited window). When impact factors are compared between years it is important to consider the size of the journal under consideration. Small titles (less than 35 papers per annum) on an average vary in impact factor by more than ± 40% from one year to the next (11).
Applications of Impact factors (12-14)
There have been many innovative applications of journal impact factors. The most common involve market research for publishers and others. But, primarily, JCR (Journal citation Report) provides librarians and researchers with a tool for the management of library journal collections. In market research, the impact factor provides quantitative evidence for editors and publishers for positioning their journals in relation to the competition--especially others in the same subject category, in a vertical rather than a horizontal or interdisciplinary comparison. JCR data may also serve advertisers interested in evaluating the potential of a specific journal.
The most important and recent use of impact is in the process of academic evaluation. The impact factor can be used to provide a gross approximation of the prestige of journals in which individuals have been published. This is best done in conjunction with other considerations such as peer review, productivity, and subject specialty citation rates. As a tool for management of library journal collections, the impact factor supplies the library administrator with information about journals already in the collection and journals under consideration for acquisition. These data must also be combined with cost and circulation data to make rational decisions about purchases of journals.
The impact factor can be useful in all these applications, provided the data is used sensibly. It is important to note that subjective methods can be used in evaluating journals as, for example, by interviews or questionnaires. In general, there is good agreement on the relative value of journals in the appropriate categories. However, the JCR makes possible the realization that many journals do not fit easily into established categories. Often, the only differentiation possible between two or three small journals of average impact is price or subjective judgments such as peer review (15).
Uses of Impact Factor
The Institute for Scientific Information (ISI) does not depend on the impact factor alone in assessing the usefulness of a journal, and neither should anyone else. The impact factor should not be used without careful attention to the many phenomena that influence citation rates, as for example the average number of references cited in the average article. The impact factor should be used with informed peer review. In case of academic evaluation for tenure it is sometimes inappropriate to use the impact of the source journal to estimate the expected frequency of a recently published article. Again, the impact factor should be used with informed peer review. Citation frequencies for individual articles are quite varied.
There are many artifacts that can influence a journal's impact and its ranking in journal lists, not the least of which is the inclusion of review articles or letters.
a) Review Articles
Review articles generally are cited more frequently than typical research articles because they often serve as surrogates for earlier literature, especially in journals that discourage extensive bibliographies. In the JCR system any article containing more than 100 references is coded as a review. Articles in "review" sections of research or clinical journals are also coded as reviews, as are articles whose titles contain the word "review" or "overview."
The Source Data Listing in the JCR not only provides data on the number of reviews in each journal but also provides the average number of references cited in that journal's articles. Naturally, review journals have some of the highest impact factors. Often, the first-ranked journal in the subject category listings will be a review journal (16).
b) Methods Articles
It is widely believed that methods articles attract more citations than other types of articles. However, this is not in fact true. Many journals devoted entirely to methods do not achieve unusual impact. But it is true that among the most cited articles in the literature there are some super classics that give this overall impression. It should be noted that the chronological limitation on the impact calculation eliminates the bias super classics might introduce. Absolute citation frequencies are biased in this way, but, on occasion, a hot paper might affect the current impact of a journal (17).
Variation Between Disciplines
Different specialties exhibit different ranges of peak impact. That is why the JCR provides subject category listings. In this way, journals may be viewed in the context of their specific field. Still, a five-year impact may be more useful to some users and can be calculated by combining the statistical data available from consecutive years of the JCR. It is rare to find that the ranking of a journal will change significantly within its designated category unless the journal's influence has indeed changed.
Different specialties exhibit different ranges of peak impact. That is why the JCR provides subject category listings. In this way, journals may be viewed in the context of their specific field. Still, a five-year impact may be more useful to some users and can be calculated by combining the statistical data available from consecutive years of the JCR. It is rare to find that the ranking of a journal will change significantly within its designated category unless the journal's influence has indeed changed.
Item-by-Item Impact
While ISI does manually code each published source item, it is not feasible to code individually the 12 million references processed each year. Therefore, journal citation counts in JCR do not distinguish between letters, reviews, or original research. So, if a journal publishes a large number of letters, there will usually be a temporary increase in references to those letters. Letters to the Lancet may indeed be cited more often that letters to JAMA or vice versa, but the overall citation count recorded would not take this artifact into account. Detailed computerized article-by-article analyses or audits can be conducted to identify such artifacts.
A user's knowledge of the content and history of the journal studied is very important for appropriate interpretation of impact factors. Situations such as those mentioned above and others such as title change are very important, and often misunderstood, considerations.
A title change affects the impact factor for two years after the change is made. The old and new titles are not unified unless the titles are in the same position alphabetically. In the first year after the title change, the impact is not available for the new title unless the data for old and new can be unified. In the second year, the impact factor is split. The new title may rank lower than expected and the old title may rank higher than expected because only one year of source data is included in its calculation. Title changes for the current year and the previous year are listed in the JCR guide (18-19).
Conclusions
Citation measures, facilitated by the richness of ISI citation databases can provide very useful insights into scholarly research and its communication. Impact factors, as one citation measure, are useful in establishing the influence journals have within the literature of a discipline (20). The impact factor is a very useful tool for evaluation of journals, but it must be used discreetly. Considerations include the amount of review or other types of material published in a journal, variations between disciplines, and item-by-item impact. The journals status in regard to coverage in the ISI databases as well as the occurrence of a title change are also very important.
Citation measures, facilitated by the richness of ISI citation databases can provide very useful insights into scholarly research and its communication. Impact factors, as one citation measure, are useful in establishing the influence journals have within the literature of a discipline (20). The impact factor is a very useful tool for evaluation of journals, but it must be used discreetly. Considerations include the amount of review or other types of material published in a journal, variations between disciplines, and item-by-item impact. The journals status in regard to coverage in the ISI databases as well as the occurrence of a title change are also very important.
References
1. Gross, P. L. and Gross E. M., College libraries and chemical education, Science, 1927; 66: 385-389.
2. Brodman E., Methods of choosing physiology journals, Bull. Med. Libr. Assn., 1944; 32: 479-83.
3. Garfield E., Citation analysis as a tool in journal evaluation, Science, 1972; 178:471-479.
5. Garfield, E. "Which medical journals have the greatest impact?"
Annals of Internal Medicine, 1986; 105(2): 313-20.
Annals of Internal Medicine, 1986; 105(2): 313-20.
6. Garfield, E. "Why are the impacts of the leading medical journals so similar and yet so different?item-by-item audits reveal a diversity of editorial material"
Current Contents, #2, p.3, January 12, 1987
7. Garfield, E. " How can impact factors be improved? "
British Medical Journal, 1996; 313 (7054): 411-413.
Current Contents, #2, p.3, January 12, 1987
7. Garfield, E. " How can impact factors be improved? "
British Medical Journal, 1996; 313 (7054): 411-413.
8. Garfield, E. "Journal impact factor: a brief review"
Canadian Medical Association Journal, 161(8), 1999; 979-80.
Canadian Medical Association Journal, 161(8), 1999; 979-80.
9. Garfield, E. "Long-Term vs. Short-Term Journal Impact: Does it matter?" The Scientist, 1998; 12(3): 10.
10. Garfield, E. "Long-Term vs Short-Term Impact: Part II. Cumulative Impact Factors" The Scientist, 1998; 12(14):26.
11.Garfield, E. "Impact factors, and why they won't go away"
Nature 411, p. 522, May 31, 2001.
Nature 411, p. 522, May 31, 2001.
12. Garfield, E. "The use of JCR and JPI in measuring short- and long-term journal impact "
Presented at Council of Scientific Editors Annual Meeting, May 9, 2000
13. Garfield, E. "Use of Journal Citation Reports and Journal Performance Indicators in measuring short and long term journal impact" Croatian Medical Journal, 41: (4) 368-374 December 2000.
14. Garfield, E. "Refining the Computation of Topic Based Impact Factors: Some Suggestions"
Occupational Medicine (Oxford) 49:(8) 571, November 1999.
Occupational Medicine (Oxford) 49:(8) 571, November 1999.
15. Garfield, E. "The Impact Factor and using it correctly"
Der Unfallchirurg, 101(6), p.413, June 1998 English Translation
16. Garfield, E. "Dispelling a few common myths about Journal Citation Impacts"
The Scientist,Vol.11(3),p.11, February 3, 1997
17. Garfield, E. "The Impact of Cumulative Impact Factors"
Proceedings of the 8th IFSE Conference, Barcelona, 1995, p.58-81.
Proceedings of the 8th IFSE Conference, Barcelona, 1995, p.58-81.
18. Garfield, E. "Prestige versus Impact: Established images of journals, like institutions are resistant to change"
Essays of an Information Scientist, Vol:10, p.263-264, 1987
Essays of an Information Scientist, Vol:10, p.263-264, 1987
19. Christenson JA, Sigelman L. "Accrediting knowledge: Journal stature and citation impact in social science"
Social Science Quarterly, Vol, 66, No, 4, December 1985
20. Garfield, E. "Which journals attract the most frequently cited articles? Here's a list of the top fifteen."
Essays of an Information Scientist, Vol:1, p.485, 1962-73.
Essays of an Information Scientist, Vol:1, p.485, 1962-73.
21. Gross, P. L. and Gross E. M., College libraries and chemical education, Science, 1927; 66: 385-389.