EOPF
Explore the architecture
EOPF provides a complete environment
for developing and running
Earth Observation processors
It connects structured EO data with development tools, libraries, and scalable computing resources. This allows developers to create reliable processing workflows that can be tested, validated, and deployed in operational environments.
The architecture includes:
EO Data and Products
Standardised data formats and structured datasets designed for efficient cloud access and processing.
Development Environment
(EOPF Playground)
A workspace that provides computing resources, libraries, and tools for building and testing EO processors.
Structure & format definition
The Structure & Format Definition within the EO Processing Framework (EOPF) refers to the standardized organization and technical specifications for instrument data products and Auxiliary Data Files (ADFs)
It is a key component of the EOPF Products and ADFs service and includes:
Format Definitions & Product Tutorials:
Detailed technical guides on how data is formatted within the framework
Data Samples:
Actual samples of EOPF products and ADFs used for development and testing
Product Structure Examples:
Visual or descriptive examples showing the hierarchical layout of data products
Technically, the framework is moving towards industry standards by replacing custom data structures with xarray’s DataTree and utilizing the Zarr storage format.These definitions are essential for the Core Python Modules (CPM) to perform efficient data management and conversion across the processing chain.
Libraries & Stack
The EO Processing Framework (EOPF) software stack is primarily built on the Core Python Modules (CPM), which harmonize best-in-class open-source Python libraries into a single framework. This stack provides critical functionalities such as data management, data conversion, distributed and parallel computing, and base algorithms specifically for Earth Observation sensors.
The library is currently transitioning to CPM V3, a major refactoring that replaces custom data structures with xarray’s DataTree to align with industry standards. For storage, the framework utilizes the Zarr format, which has been updated to support relative paths.
In addition to CPM, the stack includes the ASGARD and ASGARD-Legacy libraries, which provide specialized tools for Sentinel-1, Sentinel-2, and Sentinel-3 missions. The entire environment is supported by GitLab for collaboration and Jupyter notebooks for interactive documentation and development.
CPM
The Core Python Modules (CPM) is a harmonized framework that integrates top-tier open-source Python modules to provide a comprehensive set of features for EO data processing. Its core functionalities include data management, data conversion, distributed and parallel computing, logging, and base algorithms for Earth Observation sensors.
The framework is currently transitioning to CPM V3, a major refactoring that replaces custom data structures with xarray’s DataTree to align with modern ecosystem standards.
Asgard
ASGARD is a specialized library designed to cover the functional requirements for Sentinel-1, Sentinel-2, and Sentinel-3 missions.
It manages complex sensor-related tasks such as geometric modeling, the integration of Digital Elevation Models (DEM) like GETAS DEM, and correcting high shifts observed in specific sensors like the S3 SLSTR. Both the current ASGARD and ASGARD-Legacy versions are maintained to support various stages of processor development and re-engineering.
The framework is currently transitioning to CPM V3, a major refactoring that replaces custom data structures with xarray’s DataTree to align with modern ecosystem standards.
SDE
The Software Development Environment (SDE) is a complete development environment centered on GitLab, the renowned open-source collaboration platform. It covers the full development cycle of instrument data processors by leveraging modern DevOps practices.
The SDE provides two primary services:
Test Data Management Service: Responsible for providing all the data required for the development and validation of re-engineered processors.
Modern Documentation Management Service: Includes automatic documentation generation from source code, an advanced search engine, and interactive documentation based on Jupyter notebooks
Build Tools CI/CD Pipeline
The Build Tools and CI/CD pipelines within the EO Processing Framework (EOPF) are integrated into the Software Development Environment (SDE), which is designed to support the full development cycle of instrument data processors.
Key characteristics of these tools include:
GitLab Integration:
The SDE is centered on GitLab, utilizing it as the primary collaboration platform to implement modern DevOps practices.
Automated Documentation:
The pipeline includes a modern documentation management service that performs automatic documentation generation from source code.
Traceability and Release Management:
The framework provides dedicated tools for traceability and managing software releases, ensuring that every version of a processor is properly tracked.
Triggering and Orchestration:
The Core Python Modules (CPM) provide functionalities for triggering and orchestration, which are essential for automating tasks within the CI/CD workflow.
Validation Support:
The pipeline works in conjunction with a test data management service, which provides the necessary data for the validation of re-engineered processors during the build process.
By leveraging these tools, developers can push their code to GitLab, where the framework handles the automated steps required to build, document, and validate the software.
Join the Community
Forum
Engage with our global network of specialists. Participate in technical discussions, share insights, and get expert help on the Discourse EOPF Forum.
Issue Reporting
Help us strengthen the framework. Report bugs or suggest performance enhancements for our core components.
Developers
Built for the community. EOPF source code is distributed under the Apache License. We welcome contributions from developers worldwide.