2 The databases

The ADS Abstract Service was originally conceived of in the mid 1980's as a way to provide on-line access to bibliographies of astronomers which were previously available only through expensive librarian search services or through the A&A Abstracts series ([Schmadel 1979]; [Schmadel 1982]; [Schmadel 1989]), published by the Astronomisches Rechen-Institut in Heidelberg. While the ideas behind the Abstract Service search engine were being developed (see [Kurtz et al. 2000], hereafter OVERVIEW), concurrent efforts were underway to acquire a reliable data source on which to build the server. In order to best develop the logistics of the search engine it was necessary to have access to real literature data from the past and present, and to set up a mechanism for acquiring data in the future.

An electronic publishing meeting in the spring of 1991 brought together a number of organizations whose ultimate cooperation would be necessary to make the system a reality (see OVERVIEW for details). NASA's Scientific and Technical Information Program (STI) offered to provide abstracts to the ADS. STI's abstracts were a rewritten version of the original abstracts, categorized and keyworded by professional editors. They not only abstracted the astronomical literature, but many other scientific disciplines as well. With STI agreeable to providing the past and present literature, and the journals committed to providing the future literature, the data behind the system fell into place. The termination of the journal abstracting by the STI project several years later was unfortunate, but did not cause the collapse of the ADS Abstract Service because of the commitment of the journal publishers to distribute their information freely.

The STI abstracting approximately covered the period from 1975 to 1995. With the STI data alone, we estimated the completeness of the Astronomy database to be better than 90% for the core astronomical journals. Fortunately, with the additional data supplied by the journals, by SIMBAD (Set of Identifications, Measurements, and Bibliographies for Astronomical Data, [Egret & Wenger 1988]) at the CDS (Centre de Données Astronomiques de Strasbourg), and by performing Optical Character Recognition (OCR) on the scanned table of contents (see Sect. 6 below), we are now closer to 99% complete for that period. In the period since then we are 100% complete for those journals which provide us with data, and significantly less complete for those which do not (e.g. many observatory publications and non-U.S. journals). The data prior to 1975 are also significantly incomplete, although we are currently working to improve the completeness of the early data, primarily through scanning the table of contents for journal volumes as they are placed on-line. We are 100% complete for any journal volume which we have scanned and put on-line, since we verify that we have all bibliographic entries during the procedure of putting scans on-line.

Since the STI data were divided into categories, it was easy to create additional databases with non-astronomical data which were still of interest to astronomers. The creation of an Instrumentation database has enabled us to provide a database for literature related to astronomical instrumentation, of particular interest to those scientists building astronomical telescopes and satellite instruments. We were fortunate to get the cooperation of the SPIE very quickly after releasing the Instrumentation database. SPIE has become our major source of abstracts for the Instrumentation database now that STI no longer supplies us with data.

Our Physics and Geophysics database, the third database to go on-line, is intended for scientists working in physics-related fields. We add authors and titles from all of the physics journals of the American Institute of Physics (AIP), the Institute of Physics (IOP), and the American Physical Society (APS), as well as many physics journals from publishers such as Elsevier and Academic Press (AP).

The fourth database in the system, the Preprint database, contains a subset of the Los Alamos National Laboratory's (LANL) Preprint Archive ([Los Alamos National Laboratory 1991]). Our database includes the LANL astro-ph preprints which are retrieved from LANL and indexed nightly through an automated procedure. That dataset includes preprints from astronomical journals submitted directly by authors.