EOSDIS Core System Data Model

The Earth Observing System Data and Information System (EOSDIS) Core System (ECS) has been developed to support very large amount of data ingested in the ECS system, nominally 2 terabytes per day. These numbers imply a massive data production and storage effort. Data producers must have sufficient information stored with their products in order to effectively calibrate and then re-process data. On the other hand, the data must be searchable and acquirable by scientists from diverse disciplines, such as atmospheric chemists, hydrologists, oceanographers, and so on. The data will also be accessed by non-experts such as lawyers, policy makers, educators, etc.

The above simplistic description suggests an immense problem. Data must be stored, not lost and then found again. Requests for data will come in varied forms. Granules will be acquired by time, space, altitude, data type, etc. Producers will have very different requirements on search and orders than will inter-disciplinary data users. A single granule or collection of similar granules could be requested. Browse or descriptive information might be requested prior to ordering parent data. Information on production history, storage format, and production algorithms may be desired.

The expected diversity of uses and sheer volume and number of granules required a data model that organized and described the data holdings. Data are organized into collections of like granules. For example, data granules containing measurements of fluxes at the top of the atmosphere can be organized into a collection. Collections are described by a series of attributes, including General Description, Data Origin, Spatial Coverage, Temporal Coverage, and Contents. Granules within collections are described by a different set of attributes. These are similar to collection level attributes, except that they are used to describe individual granules within a collection. Additional attributes in the model are intended to describe browse products, documentation and Product quality information. There are about 300 attributes in the ECS data model. These are referenced in a Data Dictionary, which is also used by ECS components to validate metadata. Attributes are documented in DID 311 ECS Data Model. Granule-level attributes are inserted into ECS inventory databases, and used by the system and users to locate data.

In addition to the 300 standard attributes, Product Specific Attributes (PSAs) can be included in data products. These may or may not be searchable, but are added by the data producers to convey information to users, such as statistical averages.

Data Producers choose attributes to describe their data. However, a Minimal set is required by the system to insert science granules into the archives. These attributes include the unique granule shortname. A Limited set identifies science content of the collection to the Global Change Master Directory. An Intermediate set is used for products generated outside EOSDIS, but used and distributed by ECS, such as ancillary data, Landsat 7 data. A Full set of metadata attributes is used for ECS standard products. This latter set enables the full set of services provided by the system, such as search, order and browse. The full set is also designed to enable future services such as subsetting.

The ECS data model was built in consultation with the EOSDIS Data Model Working Group, EOSDIS Version 0, ECS engineers, EOS Instrument Teams, EOS User working groups and other affected organizations. In addition, coordination was carried out with standards organizations: Federal Geographic Data Committee (FCDC), Global Change Master Directory (GCMD) and Spatial Data Transfer Standard (SDTS). These standards were not well developed at the time ECS was being designed. As a result, and due to the sheer volume of data products being inserted into ECS archives, the ECS standard will influence the others. For example, the ECS standard for storing swath data has been adopted as an FGDC standard.

The ECS metadata model is documented in Release 6B Implementation Earth Science Data Model for the ECS Project. Or use the on-line version