VizieR is a natural extension of the usage of the metadata stored in the ReadMe files, as an implementation of these metadata in terms of tables managed by a relational database management system (RDBMS).
The first prototype of VizieR was the result of a fruitful collaboration between ESIS (European Space Information System, a project managed by ESRIN, a department of the European Space Agency) and the CDS; VizieR has been under full responsibility of CDS since January 1996. It was presented at the 1996 AAS meeting (Ochsenbein et al. ), and became fully operational in February 1996. This prototype has been significantly upgraded in May 1997, just in time for the implementation of the final catalogues of the Hipparcos mission. The number of catalogues accessible within the VizieR system has grown since that time to 2 374 catalogues (Table 6).
The core of VizieR consists in the organisation of the meta dictionary, i.e. the set of metadata extracted from the standardized ReadMe descriptions discussed in Sect. 4. There are however two main problems which had to be solved: the access to very large catalogues (larger than a few million rows) for which RDBMS proved to be inefficient, requiring therefore dedicated search methods, and the generation of links allowing to connect two related pieces of information, like other tables in the same catalog, or spectra, images from remote services, etc.
|VizieR contents||All||Dealing with objects|
|in terms of:||Catalogues||having positions|
The meta-dictionary consists in 3 main tables detailed below, and about 20 annex tables, all stored in a relational database:
The annex tables of the meta-dictionary contain some definitions, like the list of known data-types (METAtypes) and keywords (METAkwdef); or other details like the acronyms used to designate well-known catalogues like HIP, GSC...(METAcro), the keywords associated to each catalogue (METAkwd), detailed notes and remarks (METAnot), or the list of those objects which are individually quoted in the ReadMe files (METAobj). A special indexing scheme (METAcell), explained briefly in Sect. 5.5, was built to locate the existing objects in all catalogues in a single run. Details on how to generate links are stored in the METAmor table.
The links existing in VizieR may be classified in the following categories:
While links of the first 3 categories can easily be maintained, the maintenance of the external links depends on modifications which are completely outside VizieR's control. These external links are maintained by the GLU system (Fernique et al. ), a system which (i) allows one to use symbolic names instead of hard-coded URLs, and (ii) translates these symbolic names with the help of a distributed dictionary in which the service providers keep up the descriptions of their own services only in terms of URL addresses and actual presentation of the query parameters.
The actual delay required to ingest a new catalogue into the system is currently estimated to something between a few minutes and several days for the preparation of the ReadMe description file, depending on the initial presentation supplied by the authors and on the catalogue complexity -- the delay can be occasionally longer when problems are encountered, requiring interactions with the authors; and a few seconds up to an hour for the actual ingestion into VizieR from the standardized files.
|USNO-A1.0||488.0||The USNO-A1.0 Catalog (Monet 1997)|
|USNO-A2.0||526.3||The USNO-A2.0 Catalog (Monet 1998), calibrated against Tycho data|
|GSC1.1||25.2||HST Guide Star Catalog, 1992 version|
|GSC1.2||25.2||HST Guide Star Catalog, 1996 version|
|GSC-ACT||25.2||HST Guide Star Catalog, calibrated against Tycho data|
|2MASS||20.2||m All Sky Survey, Spring 1999 release (Skrutskie et al. )|
|DENIS||17.5||Deep Near-IR Survey first release (Epchtein et al. )|
calibration made by the Pluto project
The limit of 107 rows corresponds to a limit in performance and time required to ingest the tables into the relational databases; the largest table, in terms of number of rows, currently stored in VizieR is the AC2000 catalog (Urban et al. ), with rows.
The method used to access these very large catalogues consists in grouping the objects within carefully designed groups based essentially on the location in the sky, followed by a lossless compression obtained by replacing the actual values by offsets within the group; details about the actual results and performances are described in another paper (Derriere & Ochsenbein ). Each very large catalogue has presently its own organisation which depends on its actual column contents, and therefore requires a dedicated program for accessing it. VizieR stores in its META dictionary (see Sect. 5.1) which program has to be called to actually access the catalogue, and the description of the columns as they are returned from the dedicated program.
The method adopted for this indexation consists first in a mapping of the celestial coordinates into a set of boxes using a hierarchical spherical-cubic projection similar to the techique used by SIMBAD (Wenger et al. ), but down to a level 8 which corresponds to a granularity of about 20', or ( ) individual boxes. The list of catalogues which exhibit sources in the region of the sky covered by the box is then stored for each of the defined boxes, allowing therefore a fast answer to the question: "what is the list of catalogues which have a fair chance of having at least one source close to a specified target ?'' The final step consists in looking successively into the matching catalogues.
The method offers the particularity of being hierarchical: 6 boxes are defined at level 0, 24 at level 1, ..., and going down one step in the hierarchy consists in dividing each box into four parts. The indexing mechanism recursively groups contiguous non-empty boxes represented by a single box at the upper level, meaning that a dense survey covering the whole sky is just represented by the 6 boxes of level 0 in this index. In practice, the 1247 catalogues with positions are summarized in this index by elements (to be compared to the sources in Table 6), i.e.an average of 3 000 elements per catalogue.
|Figure 2: Histogram of the number of rows among VizieR tables (the darker bars correspond to tables containing celestial coordinates)|
The status of VizieR contents is presented in Table 6, where we distinguished those tables representing data about actual astronomical objects which can be accessed by their position in the sky. In terms of number of available records, those containing celestial positions represent over 78% even when the megacatalogs are omitted, even though only 32% of the tables are concerned. In other words, the average table dealing with actual astronomical objects contains around 16 000 rows -- a theoretical mean, as can be seen from the histogram of the table populations in VizieR represented in Fig. 2 which shows a modal value around tables of 100 objects.
Copyright The European Southern Observatory (ESO)