An analysis of the most cited articles in software

Age : 20 Joined : 15 Mar 2008 Posts : 842

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is
an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited
the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles
published in 2000. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by
citation count. An understanding of which research is viewed by the research community as most valuable to build upon may provide
valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented
here. The intention of the analysis is twofold. First, to identify the most cited articles, and second, to invite the authors of the most
cited articles in 2000 to contribute to a special issue of Information and Software Technology. Five authors have accepted the invitation
and their articles appear in this special issue. Moreover, an analysis of the most cited software engineering journal articles in the last 20
years is presented. The presentation includes both the most cited articles in absolute numbers and the most cited articles when looking at
the average number of citations per year. The article describing the SPIN model checker by G.J. Holzmann published in 1997 is first on
both these lists.
2006 Elsevier B.V. All rights reserved.
Keywords: Citations; Bibliometric research; Software engineering
1. Introduction
Citations are a common way of judging the most influential
work in different fields. The most cited articles often
provide new insights, open a new avenue of research, or
provide a significant summary of the state-of-the-art in
an area. Citations are a way to show how researchers build
their work on existing research to evolve research further.
Basically, they are the backbone of research and hence articles
and authors being cited frequently deserve acknowledgment
for their contribution. The analysis presented
here is intended to highlight which articles are most cited
in software engineering in a given year. The analysis is
intended to complement the work on ranking the most
published scholars and institutions as done by Tse et al. [6].
The objective of the analysis presented here is to list
the most cited journal articles in the field of software
engineering recorded as published in 2000. The analysis
is based on the ISI Web of Science [2]. The ISI web
covers the major journals in the field, creating a web
of references that ensures that the overall picture
obtained from the web is likely to give a representative
view of the most cited articles. In particular, it means
that references from journals included in the ISI Web
of Science also gets included in the web and hence
included in information available from the created web
of references.
The analysis is published as a list of the 20 most cited
articles, or in case of ties the actual number of articles
may be higher, for example, two articles may be tied
for position 20 and hence the list would contain 21 articles.
Authors of the top ranked articles have been invited
to write a new article for this special issue. They havethe previous work attracted high attention or some current
research that they are conducting.
Two main issues should be kept in mind:
• The possible selection of journals is limited to what is
available through the ISI Web of Science.
• The focus is on software engineering. This means that
the actual selection of journals is based on selecting journals
that are perceived as mainly publishing software
engineering articles. All articles in these journals have
been analyzed. This implies that no judgment has been
made whether a specific article is within software engineering
or not. The main reason being that it would
not make the results from the analysis replicable and it
would also create discussions of the borderlines between
fields such as software engineering, computer science
and information systems.
The intention is for the analysis provided here to be conducted
and published on a yearly basis in a special issue of
Information and Software Technology.
The article is structured as follows. Section 2 presents
the ISI Web of Science and the actual selection of journals
in the analysis. In Section 3, the analysis method is
described. This includes some information about the ISI
Web of Science in relation to the analysis method and a
presentation of how the top 20 list was generated. Section
4 presents the results in relation to the top 20 list. In Section
5, lists of the most cited software engineering journal
articles in the last 20 years are presented. A short summary
of the findings is provided in Section 6.
2. Selection decisions
A key issue when looking at citations is what to count.
This includes both, which publications and which references.
When it comes to the publications, any analysis is constrained
by the support given by different databases or
search engines. In the analysis presented here, it was decided
to use the ISI Web of Science. The actual count of citations
is further discussed in Section 3.
2.1. Tool support
The selection of which tool to use to count citations has
a major impact on the actual outcome and hence on the
trustworthiness of the findings. It is worth noting that bibliometric
research is a field of its own. In this field, publication
patterns are studied including both descriptive (for
example counting the number of publications from an
organization) and evaluative (for example counting citations
as a measure of impact). The Institute for Scientific
Information (ISI) has been leading in the field since its
establishment in 1961 [4]. The metrics provided by ISI
are being used for determining impact factors for journals
[1], assess and drive bibliometric research [8], and support
studies like ours in other fields such as medicine [3]. Sample
checks with some of the author’s publications confirmed
that ISI strengths apply to the software engineering domain
as well as in medicine [3]. Hence, the position of the ISI
data as a leading source for bibliometric research and the
actual use of it in other fields were decisive factors when
determining to use this tool for this study of the most cited
articles in software engineering.
2.2. Journals
The objective of the ISI as a database is to provide a
comprehensive coverage of the most important and influential
research. The information about ISI is based on
[5]. The database includes in total more than 8500 journals
and some other sources, for example, Lecture Notes in
Computer Science. However, journals are here used as a
reference to the content of the database. The journals cover
three areas: science, social sciences and arts and humanities.
The ISI staff reviews close to 2000 journals yearly,
but only 10–12% makes it into the database. An interesting
feature is how the references build a web. ISI captures the
cited references and citation information that is included
both from journals in the database and for those journals
not included in the database but which are cited from journals
included. This ensures a good coverage of citations
and also that the data extracted provides a good picture
of actual citations.
The objective was to make a selection of journals that
provide as fair picture as possible of the most cited articles
in software engineering. No database or tool support was
found that was capable of also including all references in
conferences papers and book chapters and hence the analysis
is made under the assumption that journals provide a
representative picture of the most cited type of articles
and authors.
The first column in Table 1 lists the journals selected
from the database as a suitable set of software engineering
journals. However not all of these journals were available
in the database. Thus, the table also includes information
about journals actually in the database in 2000 and
remarks about journals when they appeared some other
years, but not in year 2000. The journals in the database
for 2000 are the journals actually used in the citation study
for 2000.
Several things may be observed from the table and some
issues are worth commenting. A subset of journals has been
in the database several years and appears every year. Some
journals are added specific years and others are removed.
Finally, some journals or issues of journals or specific articles
can be found in the database based on that they are
cited by articles in the database.
3. Method
The analysis is done focusing on the science part of the
ISI Web of Science. The citation search is conducted as follows.
The search is conducted for one specific year, for
been asked to either write a follow-up article given that