1 INTRODUCTION¶
1.1 Purpose and Scope¶
The scope of this document is to be used as an annex for the CSC Common Entity Definition Document [RD-2] in order to describe the Product Entity Properties, the Product Attributes and the corresponding metadata elements that shall be catalogued and queryable for Sentinel products. This document provides the mapping and packaging format for the Sentinel-2 product items.
1.2 Applicable Documents¶
| ID | Document Reference | Document Title | Issue | Date |
|---|---|---|---|---|
| [AD-1] | ESA-EOPG-EOPGC-TN-58 | Copernicus Ground Segment Sentinels Data Flow Configuration | 1.2 | 06/04/2023 |
Table 1-1 – Applicable Documents
1.3 Reference Documents¶
| ID | Document Reference | Document Title |
|---|---|---|
| [RD-1] | S2-PDGS-TAS-DI-PSD | Sentinel-2 Products Specification Document |
| [RD-2] | ESA-EOPG-EOPGC-IF-5 | Copernicus Space Component Ground Segment - Common Entity Definition Document |
| [RD-3] | OGC 17-003r2 | OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard |
| [RD-4] | OGC 10-157r4 | OGC Earth Observation Metadata profile of Observations & Measurements |
| [RD-5] | ESA-EOPG-EOPGC-TN-13 | Copernicus Space Component Ground Segment Operations Glossary |
| [RD-6] | GMES-S2GS-EOPG-TN-13-0004 | Sentinel-2 GRI Format Specification Document |
Table 1-2 – Reference Documents
1.4 Acronyms, Definitions and Abbreviations¶
| Acronym | Description |
|---|---|
| ADG | Auxiliary Data Gathering |
| AUX | Auxiliary |
| CAMS | Copernicus Atmosphere Monitoring Service |
| CSC | Copernicus Space Component |
| DA | Data Access |
| EUP | End User Product |
| GS | Ground Segment |
| ICD | Interface Control Document |
| JSON | JavaScript Object Notation |
| LTA | Long Term Archiving |
| MPC | Mission Performance Cluster |
| ODP | On-Demand Processing |
| OLQC | On Line Quality Control |
| PDGS | Payload Data Ground Segment |
| PDI | Product Data Item |
| PR | Systematic Production |
| TCI | True Colour Image |
| SAFE | Standard Archive Format for Europe |
Table 1-3 – Acronyms and Abbreviations
1.5 Definitions¶
Auxiliary Data – Data which enhance processing and utilization of remote sensing instrument data. The auxiliary data are not necessarily captured by the same data collection process as the instrument data. Auxiliary data include data collected by any other platform or process (e.g. meteorological data from ECWMF or NCEP), data providing processing configuration information typically for data calibration and or instrument characterisation (e.g. processing auxiliary files), and data providing information on satellite position and velocity (orbit auxiliary files). Auxiliary data help in data processing, but are also data sets in their own right and may be relevant for end-users in specific User Level Data exploitation scenarios.
2 SENTINEL-2 PRODUCTS AND AUXILIARY DATA¶
2.1 Sentinel-2 Product Items¶
[AD-1] describes the list of Sentinel-2 product items which are required to be circulated within the Copernicus Space Component (CSC) GS elements (e.g. from Systematic Production to Long Term Archiving or Data Access systems).
Each Sentinel-2 PDI will be packaged independently of other related PDIs (e.g. of the same datatake), and it is these PDIs that will be circulated between the various CSC GS elements. Each Granule and Datasstrip PDI is a tar file consisting of data and metadata. The structures of these Granules and Datasstrip PDIs are defined in [RD-1].
A Sentinel-2 Granule PDI contains among its metadata the “Parent_ID” / “DATASTRIP_ID” (corresponding to the “datastripId” attribute, Table 4-1) related to the corresponding Datasstrip PDI linked to the Granule. This link establishes the hierarchy between Granule and Datasstrip, thus maintaining the relationship at metadata level. Moreover, each Datasstrip PDI contains among its metadata the “Group_ID” (“productGroupId” attribute) and this link establishes the hierarchy between Granule/Datasstrip and the Datatake.
Level-1C True Colour Image products will be generated and archived independently as well. A Level-1C TCI product will consist of a JPEG 2000 file (jp2), which will have the product metadata embedded in the jp2 file structure.
The Level-1C and Level-2A Granule PDIs represent ortho-images/tiles of 100x100km² in UTM/WGS84 projection, the Granule is in this case a Tile.
Sentinel-2 Level-1 and Level-2 products disseminated via the Data Access service are currently not included in this document. These products will not be disseminated as PDIs, instead, they will be assembled into final User Level Data.
The specifications and structures of all Sentinel-2 Product Data Items and User Products are defined in the Sentinel-2 Products Specification Document [RD-1].
2.2 Sentinel-2 Auxiliary Data¶
[AD-1] and Table 2-1 presents the list of Sentinel-2 Auxiliary Data which are required to be circulated between the CSC GS elements.
| Product Type | Data Provider | Description | Format | Band-ID Discriminator |
|---|---|---|---|---|
| AUX_ECMWFD | ECMWF | Auxiliary Data File which contains weather forecast parameters for 3 geophysical variables: total column ozone, total column water vapour and mean sea level pressure. | TGZ | |
| AUX_PREORB | POD | This file contains the predicted Orbit State Vectors (OSV) generated by the orbit determination process of the POD Service. The OSV is calculated in Earth- | EOF | |
| fixed Coordinate Frame, whenever the S/C crosses the J2000 equator. | ||||
| AUX_RESORB | POD | This file contains the restituted Orbit State Vectors (OSV) based on the orbit determination performed by the POD Service. The State Vectors will be in Earth-fixed reference. | EOF | |
| AUX_GNSSRD | POD | Daily GNSS L1b RINEX files | EOF | |
| AUX_PROQUA | POD | Processed Quaternions Files | EOF | |
| AUX_UT1UTC | IERS | The International Earth Rotation Service (IERS) Bulletin A, which contains rapid determinations for earth orientation parameters (Earth orientation parameters x/y pole, UT1-UTC and their errors). | TGZ | |
| AUX_CAMSFO | CAMS | Copernicus CAMS 'Forecast' Dataset with a sub- selection of 10 geophysical variables | TGZ | |
| AUX_CAMSAN | CAMS | Copernicus CAMS 'Analysis' Dataset | TGZ | |
| AUX_CAMSRE | CAMS | Copernicus CAMS 'Re-analysis' Dataset | TGZ | |
| GIP_ATMIMA | Cal/Val (MPC) | IAS AnaTm image parameters file (Type_1) | TGZ | |
| GIP_ATMSAD | Cal/Val (MPC) | IAS AnaTm image parameters file (Type_1) | TGZ | |
| GIP_DATATI | Cal/Val (MPC) | IAS Datation parameters file (Type_1) | TGZ | |
| GIP_LREXTR | Cal/Val (MPC) | LR Extraction parameters file (Type_1) | TGZ | |
| GIP_INVLOC | Cal/Val (MPC) | InitLoc Inv parameters file (Type_1) | TGZ | |
| GIP_VIEDIR | Cal/Val (MPC) | Pixel line of sight for each bands in the focal plane reference frame (Type_1) | TGZ | X |
| GIP_SPAMOD | Cal/Val (MPC) | Platform model (Type_1) | TGZ | |
| GIP_BLINDP | Cal/Val (MPC) | List of blind pixels (Type_1) | TGZ | |
| GIP_CLOINV | Cal/Val (MPC) | CloudInv parameter file (Type_1) | TGZ | |
| GIP_CLOPAR | Cal/Val (MPC) | Cloud mask file | TGZ | |
| GIP_PRDLOC | Cal/Val (MPC) | InitLoc production parameters file (Type_1) | TGZ | |
| GIP_R2PARA | Cal/Val (MPC) | RadioS2 parameters file (Type_1) | TGZ | |
| GIP_R2SWIR | Cal/Val (MPC) | SWIR detectors arrangement parameters (Type_1) | TGZ | |
| GIP_R2EQOG | Cal/Val (MPC) | Radiometric equalization parameters on ground (on-ground correction) (Type_1) | TGZ | X |
| GIP_R2DEPI | Cal/Val (MPC) | List of defective pixels (Type_1) | TGZ | |
| GIP_R2DEFI | Cal/Val (MPC) | Deconvolution filter for each deconvoluted band (Type_1) | TGZ | X |
| GIP_R2WAFI | Cal/Val (MPC) | Wavelets filters (Type_1) | TGZ | X |
| GIP_R2L2NC | Cal/Val (MPC) | L2 Norm coefficients (denoising) (Type_1) | TGZ | X |
| GIP_R2DENT | Cal/Val (MPC) | Denoising thresholds (denoising) (Type_1) | TGZ | X |
| GIP_R2DECT | Cal/Val (MPC) | Threshold file for deconvolution through wavelet Packets (Type_1) | TGZ | X |
| GIP_R2NOMO | Cal/Val (MPC) | Noise model (denoising) (Type_1) | TGZ | |
| GIP_R2ABCA | Cal/Val (MPC) | Absolute calibration parameters (Type_1) | TGZ | |
| GIP_R2BINN | Cal/Val (MPC) | Binning for 60m bands parameters (filters and undersampling) (Type_1) | TGZ | |
| GIP_R2CRCO | Cal/Val (MPC) | Crosstalk correction (Type_1) | TGZ | |
| GIP_G2PARA | Cal/Val (MPC) | GeoS2 parameters file (preProc) (Type_1) | TGZ | |
| GIP_G2PARE | Cal/Val (MPC) | Geometric parameter to refine (Type_1) | TGZ | |
| GIP_EARMOD | Cal/Val (MPC) | Earth model (Type_1) | TGZ | |
| GIP_GEOPAR | Cal/Val (MPC) | Global geometrical parameters (Type_1) | TGZ | |
| GIP_INTDET | Cal/Val (MPC) | Description of the inter detectors overlapping area (Type_1) | TGZ | |
| GIP_TILPAR | Cal/Val (MPC) | TilingS2 parameters file (Type_1) | TGZ | |
| GIP_RESPAR | Cal/Val (MPC) | ResampleS2 parameters file (preProc) (Type_1) | TGZ | |
| GIP_MASPAR | Cal/Val (MPC) | MaskS2 parameters file (Type_1) | TGZ | |
| GIP_JP2KPA | Cal/Val (MPC) | Compression JP2K parameters file (Type_2) | TGZ | |
| GIP_ECMWFP | Cal/Val (MPC) | ECMWF parameters file (Type_2) | TGZ | |
| GIP_DECOMP | Cal/Val (MPC) | On board decompression parameters file (Type_2) | TGZ | |
| GIP_OLQCPA | Cal/Val (MPC) | OLQC configuration parameters file (Type_2) | TGZ | |
| GIP_PROBAS | Cal/Val (MPC) | Processing Baseline parameters (Type_2) | TGZ | |
| GIP_CONVER | Cal/Val (MPC) | Mapping parameters from 16 to 18 bits (Type_2) | TGZ | |
| GIP_R2EOB2 | Cal/Val (MPC) | Radiometric equalization parameters on board (dark current, on-board inter pixel calibration) (Type_3) | TGZ | X |
| GIP_L2ACAC | Cal/Val (MPC) | Calibration parameter for the atmospheric correction algorithm | TGZ | |
| GIP_L2ACSC | Cal/Val (MPC) | Calibration parameter for the scene classification algorithm | TGZ | |
| GIP_PROBA2 | Cal/Val (MPC) | Processing baseline identifier of the generated L2A products | TGZ | |
| GIP_HRTPAR | Cal/Val (MPC) | Auxiliary file from S2 IPF 6.x | TGZ | |
| AUX_GRIXXX1 | Cal/Val (MPC) | Global Reference Image provided to the Sentinel-2 operational processor, for the Level-1B product generation |
Table 2-1 – Sentinel-2 Auxiliary Data list
Some Sentinel-2 Auxiliary Products come as separate files, one for each spectral band. These are indicated in the last column in Table 2-1 and concern the following products: GIP_VIEDIR, GIP_R2EQOG, GIP_R2DEFI, GIP_R2WAFI, GIP_R2L2NC, GIP_R2DENT, GIP_R2DECT and GIP_R2EOB2. Each of these products shall be indexed separately, by spectral band. The productType attribute of these products (Table 4-1) shall be concatenated by extracting the “File Type” (e.g. GIP_R2DECT) and the band number (e.g. B01) from the product filename and then joining them together, separating them by an underscore: <file_type>_<band number=“”> (e.g. GIP_R2DECT_B01).
2.3 Sentinel-2 OLQC Reports¶
The On-Line Quality Control (OLQC) performs essential quality checks on each product generated by the processing chain. The results of the quality checks are presented as XML reports. OLQC reports may be issued for each Checklistname for each Sentinel-2 PDI and the contents of each report are configurable. These files are described in detail in [RD-1].
The OLQC reports will be exposed by the Production Service on the Production Interface delivery Point (PRIP).
¹ Archived manually as a one-off. Where: XXX = Relative Orbit (001-143)
3 SENTINEL-2 PRODUCT ENTITY PROPERTIES MAPPING¶
[RD-2] defines the Generic CSC Common Entity Properties. Table 3-1, below, provides the mapping between these Generic Product Entity Properties and the Sentinel-2 product metadata. Most of the properties defined in [RD-2] are generated by the CSC services. The following table describes the properties that map directly to the Sentinel-2 product metadata.
| OData Product Entity Properties | Type | Description | Cardinality | Corresponding Sentinel-2 Metadata | Example | |||||
| L0, L1B, L1C, L2A Datastrip | L0, L1B Granule L1C, L2A Tile | L1C, L2A TCI | HKTM | SAD | AUX | |||||
| Name | String | Data file name (according to the Sentinel file naming conventions) plus the file extension | 1 | n/a Based on file name. | n/a Based on file name. | n/a Based on file name. | n/a Based on file name. | n/a Based on file name. | n/a Based on file name. | S2A_MSIL1C_201807 10T100031_N0208_R 122_T32TQL_201807 10T110910.zip |
| ContentDate | TimeRange | The sensing range period. Compound property with start and end times in UTC in the format YYYY- MM- DDThh:mm:ss.sssZ | 1 | Validity_Start Validity_Stop | Validity_Start Validity_Stop | Validity_Start Validity_Stop | acquisition Period/ startTime acquisition Period/ stopTime | Validity_ Start Validity_ Stop | Validity_ Start Validity_ Stop | "ContentDate": { "Start":"2018-07- 10T10:00:31.022Z", "End":"2018-07- 10T10:00:31.022Z" } |
| Footprint | Geography | Footprint of the product expressed as Edm.GeographyPolyg on | 0..1 | The footprint coordinates are to be constructed from the Geographic_Localization metadata (i.e. the Geo_Pnt LATITUDE – LONGITUDE information in the Inventory_Metadata.xml) | n/a | n/a | n/a | geography'SRID=432 6; Polygon((),(-41.15749 66.766701,- 31.740927 67.629135,- 31.479883 66.860405,- 40.616844 66.011871,-41.15749 66.766701))' | ||
| GeoFootprint Geography | Geography | Footprint of the product following the GeoJSON format | 0..1 | The footprint coordinates are to be constructed from the Geographic_Localization metadata (i.e. the Geo_Pnt LATITUDE – LONGITUDE information in the Inventory_Metadata.xml) | n/a | n/a | n/a | "GeoFootprint":{ "type": "Polygon", "coordinates": [ [ [-59.3169, 2.6367], [-63.105, -14.0539], [-60.8506, - 14.4245], [-57.1309, 2.3269], [-59.3169, 2.6367] ] ] } | ||
Table 3-1 – Product Entity Properties mapping to Sentinel-2 metadata
4 SENTINEL-2 PRODUCT ATTRIBUTES MAPPING¶
All relevant metadata elements of the Sentinel-2 products shall be indexed in the Product Attributes; additional metadata elements may also be identified if appropriate. In order to provide a harmonised model across the Sentinel missions the JSON property naming from [RD-3] is preferred for the Attribute Names. In case an attribute is not defined in [RD-3] the naming used in the mission specific metadata files, harmonised across the Sentinel missions, is preferred. The Product Attributes mapping for all Sentinel-2 products is described in the Excel file “Sentinel-2 Product Attributes Mapping”, which is packaged with this document. A summary of the Sentinel-2 Product Attributes mapping across all products is presented in Table 4-1. It should be highlighted that, in order to harmonise the metadata model across the Sentinel missions, in some cases an “alias” for the value defined in the metadata is mapped to a standardised value to be used by the CSC GS elements, in other cases a standard value is defined where it is otherwise unavailable from the product.
| OData Attribute Name | Sentinel-2 Metadata Mapping | Applicability | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| L0, L1B, L1C, L2A Datastrip | L0, L1B Granule | L1C, L2A Tile | L1C, L2A TCI | HKTM | SAD | AUX | OLQC Report | PR | LTA | ADG | DA | |
| beginningDateTime | x | x | x | x | x | x | x | x | x | x | x | x |
| endingDateTime | x | x | x | x | x | x | x | x | x | x | x | x |
| productType | x | x | x | x | x | x | x2 | x | x | x | x | x |
| processorVersion | x | x | x | x | x | x | x | x | x | x | ||
| processingCenter | x | x | x | x | x | x | x | x | x | x | x | x |
| processingDate | x | x | x | x | x | x | x | x | x | x | x | x |
| platformShortName | x | x | x | x | x | x | x | x | x | x | x | x |
| platformSerialIdentifier | x | x | x | x | x | x | x3 | x | x | x | x | x |
| instrumentShortName | x | x | x | x | x | x | x | x | x | |||
| operationalMode | x | x | x | x | ||||||||
| orbitNumber | x | x | x | x | x | x | x | x | x | |||
| lastOrbitNumber | x | x | x | x | x | |||||||
| relativeOrbitNumber | x | x | x | x | x | |||||||
| cloudCover | x | x | x | x | x | x | ||||||
| productGroupId | x | x | x | x | x | x | x | |||||
| datastripId | x | x | x | x | x | x | ||||||
| tileId | x | x | x | x | x | |||||||
| illuminationZenithAngle | x | x | x | |||||||||
| qualityStatus | x | x | x | x | x | x | x | x | ||||
| qualityInfo | x | x | x | x | x | x | x | x | ||||
| checklistResult | x | x | ||||||||||
| checklistParentId | x | x | ||||||||||
| checklistName | x | x | ||||||||||
| inspectedProduct | x | x | ||||||||||
| inspectedProductType | x | x | ||||||||||
Table 4-1 – Sentinel-2 Product Attributes Mapping
² For AUX Products separated by bands the productType shall be concatenated using the File Type and the Band Number
³ Used where available