1. Introduction¶
1.1 Purpose¶
This System Design Document (SDD) provides a comprehensive view on the design and overall architecture of the DestinE Core Service Platform and in particular on the Platform Management Services (section 4).
1.2 Reference Documents¶
| Ref. | Title | Reference and Version |
|---|---|---|
| RD-1. | OpenStack | https://www.openstack.org/ |
| RD-2. | Kubernetes | https://kubernetes.io/ |
| RD-3. | JIRA | https://www.atlassian.com/software/jira |
| RD-4. | OpenStack Stein API Reference Documentation | https://docs.openstack.org/stein/api/index.h tml |
| RD-5. | OVHcloud API | https://api.ovh.com/ |
1.3 Acronyms and Abbreviations¶
| Acronym | Definition |
|---|---|
| AD | Applicable Document |
| API | Advanced Programming Interface |
| DCMS | Data Cache Management Service |
| DestinE platform | DestinE Core Service Platform |
| DTE | Digital Twin Earth |
| DT | Digital Twin |
| ESA | European Space Agency |
| GDPR | General Data Protection Regulation |
| GUI | Graphical User Interface |
| HDA | Harmonised Data Access |
| IAM | Identity and Access Management |
| ICD | Interface Control Document |
| IDE | Integrated Development Environment |
| RD | Reference Document |
| SDD | System Design Document |
1.4 Terms and definitions¶
Ecosystem:
Ecosystem defines a governed perimeter leveraging data and services plus a set of members (participants) that are interlinked for reciprocal interactions and for data and services exploitation.
Data Space:
Open ecosystem of distributed and federated actors sharing data, applications, services, and infrastructure.
In our framework, the Ecosystem concept is applied to DestinE Ecosystem. Please refer to Section 2.
System/Engineered System:
A system is an arrangement of parts or elements that together exhibit behaviour or meaning that the individual constituents do not. An engineered system is a system designed or adapted to interact with an operational environment to achieve one or more intended purposes while complying with applicable constraints.
The system’s properties (as a whole) result, or emerge from:
the parts or elements and their individual properties; AND
the relationships and interactions between and among the parts, the system and its environment.
In our framework, the engineered system is the DestinE Platform.
System Element:
A system element is a member of a set of elements that constitutes a system. It can be any or all of people, products, services, information, processes, and natural elements which constitute it (the term subsystem is sometimes used instead).
In this document, system elements are all the Services and Infrastructure elements constituting DestinE Platform Management Services. The architecture of the elements composing the Data Management Services is not discussed in this document.
System Component:
System component forms a system element. They can be software package, a web service, a web resource, etc. Each component is an essential part of the system element and is necessary for it to work properly.
Workflow:
A Workflow is any user coding architecture, including the data management with inputs, retrieval, transformation and writing outputs.
2. DestinE Platform design overview¶
DestinE Core Service Platform (DestinE Platform) is a user-friendly platform, funded by the European Commission in the frame of the DestinE initiative, for the delivery of DestinE services to users.
DestinE Platform is attracting user communities for data exploitation on local platform or on remote environment. It is based on an open, flexible, federated, scalable and evolvable secure cloud-based architecture. In phase 2 of DestinE, the platform connects to existing and future HPC resources as well as public cloud computing infrastructures.
2.1 DestinE Platform Design Drivers¶
The main drivers, i.e., the guidelines used to define the architecture are derived from the following documentation:
[DP-SOW], which provides the full set of contractual requirements, and [AD-DSP-TSR] providing the technical requirements.
Services Portfolio, described in “[DSP-USR-SDP] DestinE platform Services Portfolio” which document all services provided as part of the contract.
Service Level Agreement which defines the expected levels for service delivery.
the Architectural Principles of the European Initiative SIMPL.
Analysis of these set of documents and information is conducted to derive design drivers and constraints for subsequent development of the system.
Main drivers related to system design activities are detailed in Table 1:
Table 1: DestinE Platform Design Drivers
| Driver ID | Driver | Description |
| DRIVER#1 | Cloud-native applications | system elements shall be deployed onto a cloud environment |
| DRIVER#2 | Data-driven approach | system functions shall be applied as soon as data is available, compatibly with the overall dataflow design. |
| DRIVER#3 | Data agnostic approach | system functions shall be as much as possible independent on the data characteristics (e.g, type, format, timeliness, size). |
| DRIVER#4 | High availability | system functions shall be as much as possible made redundant, implementing the high-availability paradigm. |
| DRIVER#5 | Federation | System functions providing the capability of making interacting actors to directly or indirectly consume, produce, or provide resources. |
| DRIVER#6 | Modularity | System functions shall be modular, allowing to plug in/out, change or replace any system element or component. |
| DRIVER#7 | Loose coupling | System elements work independently with respect to others, without affecting the way in which other elements or actors interface with them. |
| DRIVER#8 | Resilience | System functions shall ensure that the failure of one element or component has the minimum impact on others interacting with it. |
| DRIVER#9 | Openness and agnosticism | System functions shall make use of open standards, open interfaces and open-source components, so that to ease deployment on any infrastructure. |
| DRIVER#10 | Composability and extensibility | Following microservices design approach, system elements or components shall be headless and independent, shall be implemented via containerization. |
| DRIVER#11 | Interoperability | System functions shall rely on a shared information model and adhere to common standards guaranteeing the communication with complementary ecosystems of users and services. |
| DRIVER#12 | Scalability and elasticity | System components shall scale resources according to the infrastructure layer. |
| DRIVER#13 | Security, privacy and trust | System functions shall implement security principles and standards and be compliant with the GDPR. |
| DRIVER#14 | Discoverability | System functions shall aim to making it easy for users to find, access, and understand the service's value proposition. |
2.2 DestinE Platform context¶
In Figure 1, the System context diagram of the DestinE Platform is depicted.
Looking at the system as a black box (depicted in dark grey), its external interfaces are depicted as black dashed arrows, listed in Table 2 (Section 2.3). The light-grey background draws the boundaries of the DestinE Ecosystem. External elements to the platform are represented in orange. Relevant external interfaces are depicted as black dashed arrows.
DestinE Platform is the entry point to the DestinE ecosystem, i.e., it is the DestinE entry point for users.
Figure 1: DestinE Platform context diagram.
2.3 External elements¶
DestinE Platform external elements are defined as systems/services deployed outside the DestinE Platform cloud infrastructure and run by administrators according to an agreed service level.
Table 2: DestinE Platform External elements
| External elements | Description |
|---|---|
| DEDL | DestinE Data Lake (DEDL): A space fulfilling the storage and access requirements for any data that is offered to DestinE users. It provides eligible DestinE Platform users with a harmonized access to datasets, regardless of data type and location, as well as with near-data processing capabilities. This service is implemented by EUMETSAT. |
| DTE | DestinE Digital Twin (DT) Engine (DTE): An engine capable of providing a common system approach to a unified orchestration of Earth-system simulations, delivering data from digital replicas of the Earth through the fusion of observations with models. This service is implemented by ECMWF. |
| Actors | All the actors and user categories are described in the OCD Document. |
3. DestinE Platform system design¶
Design is performed via the following steps:
generation of System Functional model (Section 3.1) which defines how the System satisfy requirements and user scenarios;
identification of System Elements able to implement the identified functions.
association of System Functions to Elements.
3.1 Functional model¶
The features of the DestinE Platform are realized through various functions, which are enumerated in the subsequent paragraphs and distinguished by a selected identifier.
Their implementation through the System Elements chosen to compose the DestinE Platform Management Services is fully described in Section 4, which reports the functional decomposition models.
The chosen approach is to define high-level functional areas, which group main functions.
3.1.1 F1 Identity and access management¶
3.1.1.1 F1.1 User Registration¶
The registration function covers the registration of any kind of user into DestinE Platform, thanks to self-registration via Information Dissemination & Onboarding Support Service - which redirects to the IAM Service dedicated panel.
The registration function also includes the possibility to find on the Web Portal the DestinE Platform “Code of Conduct”, “Terms and Conditions”, “Privacy Policy”, “Cookie Policy” and “Legal Notice”, read and access them.
3.1.1.3 F1.3 Login¶
The login function allows users to access their accounts on the DestinE Platform by providing their credentials.
Once a user is logged-in, he/she may access:
User Profile area storing personal account information
All services which are not providing users the possibility to access DestinE data
Onboarding request form
Upgraded access request form
“Submit an idea” form, to suggest new features
Moreover, depending on the level of authorization, users can access:
Services allowing access to the original DestinE data and
Service allowing access to information/data derived from DestinE data.
3.1.1.4 F1.4 User Profile management¶
This function implements the user accounts management by the DestinE Platform Registered users and Administrators.
In particular:
Users will be able to manage their user attributes.
Users will be able to change their password and email.
Users will be able to manage/monitor Tenancy feature.
Administrator will have the grants to edit user information including its token and roles.
EC will have the authority to grant DestinE Platform users access to services that are not available by default (i.e., services immediately accessible after registration).
3.1.1.5 F1.5 Federation¶
This function allows to federate the access to Third party services. This translates into the possibility for Service Providers to allow the publication of their services to the DestinE Platform user community i.e. third-party services could be accessed by DestinE Platform users using their DestinE Platform accounts.
3.1.1.6 F1.6 Management Web Interface¶
This function allows DestinE Platform Administrators to:
Register services (i.e. client, applications) that can be used for authentication by users
Register service resources
Manage the user authorization over its resources bases on policies over custom attributes and permissions schemes
3.1.1.7 F1.7 Tenancy¶
This functionality enables group of Users to access dedicated Resources provided by DestinE Platform Services.
It relies on:
Resource Access Control Layer provided by the IAM to ensure dedicated and secure access
Clearing House and Accounting component to track DestinE Platform Resource Consumptions
Tenancy Management component provided by the IAM to support Tenancy Feature scenarios:
Tenancy Members and Project Participants management
Tenancy Projects Management
3.1.2 F2 Accounting/Clearing House¶
Figure 2 Accounting and Clearing House diagram
3.1.2.1 F2.6 Transaction and Consumption Logging¶
From Accounting/Clearing House perspective, a Service is composed by one or more Transactions which are the description of every Service functionality that may trigger User Consumptions. An Accounting Transaction describes the characteristics of a Service functionality wrapped as a Tenancy Resource and enables the Service App to log related Consumptions.
3.1.2.2 F2.7 Query User Consumptions¶
This function allows Users to check their Resource Consumptions through all the Resources provided by every Service of the DestinE Platform.
3.1.2.3 F2.8 Query Service Consumptions¶
This function allows Service Providers to check the Resource Consumptions happened through the provided Resources.
3.1.2.4 F2.9 Query Tenancy Consumptions¶
This function allows Tenancy/Project Admins to check the Resource Consumptions happened through the managed Projects for the related Allocated Resources.
3.1.3 F3 Service Registration and Discovery¶
Figure 3: Service registration and discovery Functional Area
3.1.3.1 F3.2 Service Registration¶
This function covers the registration of a new service into the Service Registry – after its successful revision and approval by the Integration Manager. A registered service can also be unregistered.
3.1.3.2 F3.3 Services repository management¶
This function implements the storing and maintaining of the list of the available DestinE Platform registered Services list and their relevant metadata
3.1.3.3 F3.4 Service Discovery¶
This function - allowed to both unregistered and registered users - covers the search and identification of available registered services to be consumed.
3.1.3.4 F3.6 Service Feedback (Phase 2)¶
This function covers the process of leaving feedback for a service. Only DestinE Platform users (registered or federated) and Service Providers can leave feedback for a service.
3.1.4 F4 User Community Management¶
Figure 4: User Community Management Functional Area
3.1.4.1 F4.1 Access to Web Portal¶
This function covers the access to the Information Dissemination and Onboarding Support Service (Web Portal), without registration and authentication.
3.1.4.2 F4.2 Website content search¶
This function allows users to search for any content published in the DestinE Platform Information Dissemination and Onboarding Support Service (Web Portal), thanks to dedicated filters and search engines.
Included in the documentation published and available via the Web Portal, registered users can find the DestinE Platform Information Model and its relevant Ontology.
3.1.4.3 F4.3 Website content management¶
This function comprehends the creation and management of the DestinE Platform Information Dissemination and Onboarding Support Service (Web Portal) digital content, covering:
publication of information concerning DestinE Platform and its Services, including relevant news.
users access to the content, including “DestinE platform Code of Conduct”, “DestinE platform Terms and Conditions” and “DestinE platform Privacy Policy”.
knowledge base management enabling users to easily search information (e.g. User Guides).
3.1.4.4 F4.4 Community feedback management¶
This function covers the feedback and suggestions features managements on the platform (community ideas) and service ratings, ensuring the collections of valuable data to improve services and support. It fosters users’ interaction and is supported by sharing of information via online messages (Forum – Phase 2).
3.1.4.5 F4.5 Issue tracking and management¶
This function covers:
handling incoming requests. These requests may include technical support issues, questions about products or services, or requests for information or assistance.
tracking and resolving issues, to track and manage incoming requests, assigning them to the appropriate staff member, and monitoring their progress until resolution.
Resolution confirmation to the ticket originator automatically sent with the option to provide feedback.
3.1.4.6 F4.6 User feedback analysis and reporting¶
This function covers the collection, analysis and reporting of user’s feedback.
In their reply to a ticket resolution confirmation, users will have the choice to express their level of satisfaction of the service, with the choice of varying degree of satisfaction levels. Additionally, a dedicated feedback field will allow users to provide their level of satisfaction on a scale of 1 to 5 and leave suggestions for improvement or other comments relating to each of the services.
Collected feedback will be analysed and will feed the envisaged reports (quarterly/annual). Moreover, such data will be published on the Executive Dashboard in a page accessible to ESA.
3.1.5 F5 Monitoring and Reporting¶
Figure 5: Monitoring and reporting Functional Area
3.1.5.1 F5.1 Monitoring data gathering¶
This function implements the data gathering, performed by agents, of raw input data from Platform Management Services and Data Management Services for monitoring purposes.
3.1.5.2 F5.2 Monitoring data brokering and manipulation¶
This function covers the data brokering and collection of gathered information as a queue of message, as well as data filtering and pre-processing before storing them for reporting purposes in the Real Time Monitoring (Service Operational Monitoring Dashboard) and in the Long-Term Monitoring (Executive Dashboard).
3.1.5.3 F5.3 Monitoring data storage¶
The purpose of this function is to store the information in suitable datastores structures.
Data storages can be used for different purposes, such as:
provide a persistent and virtually infinite store of records.
provide access with minimal delay to the parameters of interest.
3.1.5.4 F5.4 Data Mining¶
This function implements the retrieval of data archived in the datastores and their aggregation and population into a dedicated DWH to reduce the execution time of a predefined list of data mining queries acting at producing a report. This ensures the provisioning of statistical data for reporting purposes.
3.1.5.5 F5.5 Reporting¶
This function implements the handling of statistical data, provided by the mining, to create automatic reporting and service reports.
3.1.5.6 5.6 Alerting¶
This function aims at feedback authorized users about significant events related to monitored systems.
3.1.5.7 5.7 Real-time monitoring¶
This function implements publication and visualization of metrics allowing up-to-date real-time monitoring of the status and performance for all Services, via the Service Operation Monitoring Dashboard Service. It allows to create, explore, share dashboards via GUI or API and export the published information to a format suited to the type of exported information (e.g., csv, excel, pdf) interactively.
3.1.5.8 F5.8 Statistics visualization¶
This function ensures unregistered and registered users to visualize statistics exposed by the DestinE Platform public Dashboard
The Executive Dashboard which is accessible to ESA supports bar-charts, curves, histograms, pie-charts, Gantt Chart tables, textual tables, etc. It will be configurable by means of filtering the visualized information per mission/unit/service etc. It will allow to select any time range for the retrieved information.
3.1.5.9 F5.9 Statistics export¶
This function ensures DestinE Platform registered users to export the Executive Dashboard statistics, in different formats, for ad-hoc further analysis.
3.1.6 F10 Infrastructure Management¶
Figure 6: Infrastructure Management Functional Area
3.1.6.1 F10.1 Infrastructure provisioning¶
This function covers the infrastructure provisioning of:
the Runtime Platform – integrated with the OVH Cloud Infrastructure layer.
Cloud Services provided by the OVH Cloud Infrastructure layer.
3.1.6.2 F10.2 Container orchestration¶
This function allows to deploy, manage, and scale containers across a cluster of machines. It automates container lifecycle management, including deployment, scaling, and recovery.
3.1.6.3 F10.3 Storage Orchestration¶
This function provides mechanisms to manage storage resources and their allocation to containers.
3.1.6.4 F10.4 Deployment management¶
This function supports smooth updates and rollbacks of containerized applications without impacting the overall system.
3.1.6.5 F10.5 Load Balancing¶
This function handles load balancing capabilities to distribute traffic across the containers providing a particular service.
3.1.6.6 F10.6 Scaling and Auto-scaling¶
This function enables horizontal scaling by allowing adding or removing instances of containers based on workload demands. It can automatically scale the number of replicas based on metrics such as CPU utilization or custom-defined metrics.
3.1.6.7 F10.7 Fault Tolerance and Self-healing¶
This function enables the monitoring of the health of containers and nodes within the Runtime platform. It ensures the availability and resilience of applications by automatically restarting or rescheduling containers in case of failures.
3.1.6.8 F10.8 Resource Allocation and Management¶
This function allows specifying resource requirements and limits for containers, ensuring optimal allocation of resources within the cluster. It also provides monitoring and metrics for resource utilization.
3.1.6.9 F10.9 Security and Access Control¶
This function offers security features and network policies. It helps enforce security best practices and provides a secure environment for containerized applications.
3.1.7 F11 Service Handling¶
3.1.7.1 F11.1 Calculation and estimation process¶
This function oversees the estimation of the pricing needed to host services on DestinE Platform. Reserved to Service Providers, it allows them to select OVH Cloud or Runtime Platform resources they intend to use, choose hourly or monthly type, and compute an estimation of the price in a specific time range.
3.1.7.2 F11.2 Carbon Calculator Estimation¶
This function offers registered users the capability to calculate carbon footprint estimation based on user service configuration.
3.1.7.3 F11.3 Service boosting¶
This function provides to eligible DestinE Platform Services access to additional resources allowing Service Providers to boost their Service and enhance user experience.
4. Overall architecture¶
The final architecture we are going to show in Table 3 reflects the mapping among:
Functions (Section 3.1) Elements involved in the function.
The functions described in Section 3.1 are implemented by DestinE Platform external and internal elements (this latter, the Core Services).
Table 3: Functions, Services & Elements mapping matrix
| Functional Area | Function | Element(s) |
| F1 Identity and Access Management | F1.1 User Registration | IAM Service |
| F1.2 Authentication & authorization | IAM Service | |
| F1.3 Login | IAM Service | |
| F1.4 User Profile management | IAM Service | |
| F1.5 Federation | IAM Service | |
| F1.6 Management Web Interface | IAM Service | |
| F1.7 Tenancy | IAM Service | |
| F2 Accounting/Clearing House | F2.6 Transaction Logging | Accounting/Clearing House Service |
| F2.7 Query own services | Accounting/Clearing House Service | |
| F 2.9 Consumption Report | Accounting/Clearing House Service | |
| F3 Service Registration and Discovery | ||
| F3.2 Service Registration | Service Registry | |
| F3.3 Service repository management | Service Registry | |
| F3.4 Service Discovery | Service Registry | |
| F3.6 Service Feedback | Service Registry | |
| F4 User Community Management | F4.1 Access to Web Portal | Information dissemination & onboarding service |
| F4.2 Website content search | Information dissemination & onboarding service | |
| F4.3 Community feedback management | Information dissemination & onboarding service | |
| F4.4 Website content management | Information dissemination & onboarding service | |
| F4.5 Issue tracking and management | Service Desk | |
| F4.6 User feedback analysis and reporting | Service Desk | |
| F5 Monitoring and reporting | F5.1 Monitoring data gathering | Service Operations Monitoring Dashboard Service |
| F5.2 Monitoring data brokering and manipulation | Service Operations Monitoring Dashboard Service | |
| F5.3 Monitoring data storage | Service Operations Monitoring Dashboard Service | |
| F5.4 Data Mining | Service Operations Monitoring Dashboard Service | |
| F5.5 Reporting | Service Operations Monitoring Dashboard Service | |
| F5.6 Alerting | Service Operations Monitoring Dashboard Service | |
| F5.7 Real-time monitoring | Service Operations Monitoring Dashboard Service | |
| F5.8 Statistics visualization | Executive Dashboard / Information dissemination & onboarding service | |
| F5.9 Statistics export | Executive Dashboard | |
| F10 Infrastructure Management | F10.1 Infrastructure Provisioning | Cloud Infrastructure layer |
| F10.2 Container orchestration | Runtime Platform | |
| F10.3 Storage Orchestration | Runtime Platform | |
| F10.4 Deployment management | Runtime Platform | |
| F10.5 Load Balancing | Runtime Platform | |
| F10.6 Scaling and Auto-scaling | Runtime Platform | |
| F10.7 Fault Tolerance and Self-healing | Runtime Platform | |
| F10.8 Resource Allocation and Management | Runtime Platform | |
| F10.9 Security and Access Control | Runtime Platform (plus IAM Service) | |
| F11 Service Handling | F11.1 Calculation and estimation process | Calculator |
| F11.2 Carbon calculator estimation | Carbon Calculator | |
| F11.3 Service boosting | Booster |
In accordance with Table 3, a general block view of the DestinE Platform architecture is shown in Figure 4: functions and involved external / internal elements are depicted together within functions they belong to.
The colour of the arrows identifies the functions belonging to one of the Functional areas (see Section 3.1). The arrow direction represents the direction of the action from the subject to the object. When the same software component is deployed in close interaction with another component, it is represented with a small shape. Badges represent in an intuitive way functions involving more services – while badges in parentheses are facultative.
Figure 7: DestinE Platform Overall Architecture.
DestinE Platform supports an open ecosystem of services for DestinE data exploitation and information sharing, including the provision of Core Services, referred to as Platform and Data Management services, for the benefit of the DestinE users and Third-Party entities. Services shall be available for all potential users as well as for any potential Third-Party applications or services.
Platform Management Services: trusted entities within the ecosystem, competing in creating a business model around services, establishing trust among participants. These services are the pillars of the platform set up and are unique.
Data Management Services: all services allowing consumers to access data and applications. The access to these services is protected.
The Platform management services, on which this document focuses on, are intended to be used by the other services to integrate inside the platform. The Data Management Services, on the other hand, are services highly dependent on the user demand. They will strongly leverage the scalability and elasticity of the platform.
In following Section 4.1, a high-level description of DestinE Platform system elements is provided.
4.1 DestinE Platform Elements¶
4.1.1 Platform Management Services¶
The DestinE Platform elements implementing the functions described in Section 3.1 are the Platform Management Services.
These, described in the following paragraphs, are:
Identity and Access Management (IAM) Service
Accounting Service/Clearing House
Service Registry
Information Dissemination & Onboarding Support Service
Service Desk
Dashboard Services
4.1.1.1 IAM Service¶
The Identity and Access Management (IAM) Service is the unified DestinE Platform user management service, granting access to all authorised registered Services based on the same digital identity.
It provides the means of authenticating end-users and authorizing their access to resources depending on the specific resource and access privileges.
The IAM Service provides also means for other Entities and Ecosystems to federate with DestinE Platform, leveraging standard protocols like SAML and OpenID Connect.
Tenancy Management Component
Tenancy Management Component is a section of the User Profile management web application provided by the IAM Service.
This component provides graphical user interface to support Tenancy scenarios which include:
Tenancy Management
Project Management
Consumptions Tracking
4.1.1.2 Clearing House (Accounting)¶
The Clearing House collects and stores all the transactions executed by DestinE Platform Services. This will track information about the services consumption, by platform active users.
4.1.1.3 Service Registry¶
The Service Registry catalogues all services, integrated in DestinE Platform, including the Core Services delivered as part of this contract, onboarded services, DEDL Services operated by EUMETSAT and DT Services operated by ECMWF.
It maintains a searchable database of all the available services and data portfolio, as per the DestinE Platform Service Catalogue.
As part of the Service Onboarding, information and metadata of new Services are registered in the Service Registry and made available to DestinE users.
4.1.1.4 Information dissemination & onboarding service¶
The Information dissemination & onboarding support service take cares of:
Web information Presence layer (website, knowledge base, video tutorials, technical documentation, …) of the DestinE Platform
user experience and community engagement activities Onboarding support to users (aiming to provide information, suggestions or help to register services on the platform).
4.1.1.5 Service Desk¶
The Service Desk is available for users to interact with DestinE Platform and Community by submitting issues, support and/or information request, and has the responsibility to manage the relevant ticketing system. Moreover, it oversees user feedback analysis and reporting.
The Service Desk will also take care of the interaction with the DestinE elements and/or third-party application services as part of the anomaly and maintenance management processes. The User Support Team will raise internal tickets for specific issues, e.g., when they discover anomalies or faults in the system or want to share suggestions on improvements for workflows or processes.
4.1.1.6 Dashboard Services¶
DestinE Platform Dashboard
The DestinE Platform Dashboard provides live information about the DestinE Platform usage in terms of total registered users and trend by user profile, registered users by country, Monthly active users per service, Number of tasks of the public roadmap for each phase, number of services. This page is public and accessible to anyone visiting the platform.
Executive Dashboard Service
The Executive Dashboard Service provides live information about the DestinE Platform usage, performance and status as well as relevant statistics coming from the platform registered Services.
Pages of the executive dashboard present a set of dedicated statistic panels designed to match reporting needs and will be accessible to the authorized users (ESA, EC).
Service Operations Monitoring Dashboard Service
The Service Operations Monitoring Dashboard Service allows real-time monitoring of DestinE Platform operations and performance, with restricted access targeted to Serco as Service Administrator and ESA Technical Officer (including individuals authorised by the ESA Technical Officer).
All the Events are delivered by parsing relevant logs and system metrics useful for operations.
4.1.2 Infrastructure elements¶
Public Cloud Figure 8: Deployment model of DestinE Platform system elements
4.1.2.1 Infrastructure as a Service (IaaS)¶
The DestinE Platform is hosted on the infrastructure provided by OVH Cloud. This infrastructure is composed by several Kubernetes clusters managed by TAS and hosted on VMs in OVH Public Cloud (OpenStack-based [RD-1]).
4.1.2.2 Runtime platform (PaaS and Caas)¶
The DestinE Platform Runtime Platform, built by Thales Alenia Space, is a common layer for the deployment and operation of all the DestinE Platform services.
The Runtime Platform integrates the following components:
The cloud infrastructure layer provided by OVH;
The Virtual Machines built on top of the cloud infrastructure using TAS provided secured operating System;
The Orchestration platform (Kubernetes-based [RD-2] CaaS) provided by TAS. Containers are the infrastructure core building block on which the exposed services physically reside, in which one or more applications may be run;
The Orchestration support services provided by TAS.
On the Runtime Platform, the Platform Management Services will provide the needed functions to enable Data Management Services to perform their ones.
The Runtime Platform will ensure the automation relevant to the configuration, coordination, and management of computing resources and applications. The Orchestrator distributes the Containers above the Compute, Storage and Network layers, providing each Container with the necessary Compute, Storage, Network.
The Orchestration Platform may also leverage also other infrastructure providers, being these DEDL computing services, HPC providers or other cloud providers that aim to federate with the DestinE Platform.
4.1.3 New registered Services¶
Within the DestinE Platform, a Service is a specific function operated by a Service Provider based on a service-level agreement.
A Destine Platform Registered Service is listed in the Service Registry, and available with free access to authorized registered users. Each registered Service providing data access, information, or software shall state the applicable access conditions and licenses and required registered Users acceptance.
External service providers, data providers and infrastructure providers can grow and expand the DestinE Platform offering by registering their own data and services on the platform or by offering their infrastructural resources to host the Services.
From a design perspective, a new registered service should adhere to the following principles:
NRSD#1: It should follow the FAIR (Findability, Accessibility, Interoperability, Reuse) principles for scientific data management.
NRSD#2: It should foster interaction and collaboration amongst the user community by providing dedicated collaborative interfaces and frameworks.
NRSD#3: It should enable seamless access to a complete suite of services, which shall serve expert development activities while allowing discovery of the Earth system and interaction with its representation by the general public.
NRSD#4: It could enable smooth integration of resources not funded by DestinE but that provide operational continuity and create a seamless scalability for registered user operations.
As a minimum, a new registered service:
Shall be registered on the Service Registry.
Shall be integrated with the IAM Service.
5. Interface Control Document¶
This Section provides the Master ICD of DestinE Platform, listing the internal and external interfaces. Each Interface is labelled with a unique code as follows:
<component id=“”>-<ext=external int=“internal” |=“”>-IF-<p r=“”>-<progressive number=“”>
Where:
<component id=“”> is the letters acronym of DestinE Platform element. The identified acronyms, listed in alphabetic order, are:
ACC = Clearing House/Accounting Service
CACHE = Data Cache Management Services
DARS = Data Access & Retrieval Services
DCDS = Data Catalogue & Discovery Services
DCMS = Data Cache Management Service
DSK = Service Desk
DSP = entire DestinE Platform system (as a black box)
EXED = Executive Dashboard
IAM = Identity and Access Management Service
IDOS = Information dissemination & Onboarding Service
OVH = Cloud Infrastructure layer
RUN = Runtime Platform
SOMD = Service Operations Monitoring Dashboard Service
SR = Service Registry
TRAC = Traceability Services
TRFO = Data Transformation Services
UWS = User Workflow Services
VIS = Visualization Services
<ext=external int=“internal” |=“”> defined the interface as EXT if external or INT if internal.
<p r=“”> indicated the interface type, P if provided or R if required.
<progressive number=“”> is a zero-padded four digits number (e.g., 0001, 0010, 0020, etc.).
Table 4 provides the list of the interfaces with their short description.
For each identified interface, the following properties are reported:
Interface unique identifier
Description
Source (i.e. provided service, and owner of the interface details description in the relevant applicable service documentation)
Destination
Table 4: List of the External and Internal interfaces of DestinE Platform
| Interface ID | Interface Description | From | To | Notes |
| DestinE platform system | ||||
| DSP-EXT-IF- P-0010 | Interface allowing authorized EUMETSAT Copernicus Data Access users to access DestinE platform Registered Services | DestinE platform | EUMETSAT Copernicus Data Access users | |
| DSP-EXT-IF- R-0020 | DestinE platform Users (registered) authorized to access the ESA Copernicus Data Access services | DestinE platform Registered Users | ESA Copernicus Data Access services | CDS Ecosystem documentation |
| DSP-EXT-IF- R-0030 | DestinE platform Users (registered) authorized to access the EUMETSAT Copernicus Data Access services | DestinE platform Registered Users | EUMETSAT Copernicus Data Access services | |
| DSP-EXT-IF- R-0040 | Access to DTE data | DestinE platform | DTE (ECMWF) | |
| DSP-EXT-IF- P-0050 | Access to immediate service | DTE (ECMWF) | DestinE platform | |
| DSP-EXT-IF- P-0070 | On-demand data access & processing | DestinE platform | DEDL (EUMETSAT) | |
| IAM Service | ||||
| IAM-INT-IF- P-0001 | DestinE platform user registration, received by the IDOS, is redirected to IAM Service | IAM Service | Information Dissemination & Onboarding Support Service (Web Portal) | |
| IAM-INT-IF- P-0010 | Interface allowing authentication and authorization of a DestinE platform Registered Service by IAM Service | IAM Service | DestinE platform Registered Services | Applicable to each DestinE platform Registered Service. |
| IAM-EXT-IF- P-0020 | Interface between IAM Service of DestinE platform and a Federated IAM Service | IAM Service | Federated IAM Service | |
| IAM-EXT-IF- P-0030 | Registered DestinE platform user can access DestinE platform IAM directly | IAM Service | DestinE platform Registered Users | |
| Accounting/Clearing House Service | ||||
| ACC-INT-IF- P-0001 | DestinE platform Data Management Services provides transaction information to the Accounting/Clearing House Service | t DestinE platform Data Management Services | Accounting/Clearing House Service | Applicable to all the Data Management Services. Facultative for a new Registered Service. |
| ACC-INT-IF- R-0010 | Accounting/Clearing House Service | Accounting/Clearing House Service Accounting/Clearing House Service | IAM Service | |
| ACC-INT-IF- P-0020 | DestinE platform services can interact with Accounting/Clearing House Service to check user balance, transaction logging, etc. | DestinE platform Data Management Services | Accounting/Clearing House Service | |
| Service Registry | ||||
| SR-INT-IF- R-0080 | Interface allowing a backend operator to add a new Service in the registry | Service Desk | Service Registry | |
| SR-INT-IF- R-0080 | Interface allowing the Web portal to visualize and browse the contents of the Service Registry | Service Registry | Information Dissemination & Onboarding Support Service (Web Portal) | |
| Information Dissemination & Onboarding Support Service | ||||
| IDOS-INT- IF-R-0050 | Redirection to the 'Support' Area present in the Web Portal | DestinE platform Registered Service | Information Dissemination & Onboarding Support Service (Web Portal) | Applicable to each DestinE platform Registered Service. |
| IDOS-INT- IF-R-0060 | Redirection to the 'User Guide' Area present in the Web Portal | DestinE platform Registered Service | Information Dissemination & Onboarding Support Service (Web Portal) | Applicable to each DestinE platform Registered Service. |
| IDOS-INT- IF-P-0070 | Support request from registered user (generate ticket) | Information Dissemination & Onboarding Support Service (Web Portal) | Tracking tracking tool | |
| IDOS-INT- IF-P-0080 | Interface allowing the visibility of each DestinE platform Registered Services in the Service Registry | Information Dissemination & Onboarding Support Service (Web Portal) | Service Operations Monitoring Dashboard | |
| IDOS-INT- IF-R-0090 | Registered users can provide feedback for the services | Users | Information Dissemination & Onboarding Support Service (Web Portal) | |
| IDOS-INT- IF-P-0100 | Registered and unregistered users can discover the services | Information Dissemination & Onboarding Support Service (Web Portal) | Unregistered and Registered users | |
| IDOS-INT- IF-P-0110 | After new service is approved, Service Registry needs to inform IDOS for creating new Keycloak client and secret and send it to the service provider | Ticketing tracking tool | Information Dissemination & Onboarding Support Service | |
| Service Desk | ||||
| DSK-EXT-IF- P-0001 | Interface allowing users to send feedback on quality of service and satisfaction, raise tickets and report suggestions – via email. | DestinE platform Registered Users | Service Desk | |
| DSK-EXT-IF- P-0010 | Interface allowing users to send feedback on quality of service and satisfaction, raise tickets and report suggestions – via web form. | DestinE platform Registered Users | Service Desk | |
| DSK-EXT-IF- P-0020 | Feedbacks and analytics report on user satisfaction | Service Desk | Users | |
| DSK-EXT-IF- R-0030 | Link with DTE's support service support service to transmit specific requests in relation to DTE services | Service Desk | DTE | |
| DSK-EXT-IF- R-0040 | Link with DEDL's support service support service to transmit specific requests in relation to DEDL services | Service Desk | DEDL | |
| DSK-INT-IF- R-0050 | Interface allowing the Service Desk to track tickets in the Ticket Tracking Tool. | Service Desk | Ticket Tracking Tool | [RD-3] |
| Executive Dashboard Service | ||||
| EXED-INT- IF-P-0001 | Interface allowing the exposure of services metrics results | Executive Dashboard Service | Registered users | Accessible only to SERCO teams and stakeholders (ESA) |
| Service Operations Monitoring Dashboard | ||||
| SOMD-INT- R-IF-0001 | Interface allowing up-to-date real- time monitoring of the DestinE platform Core Services operations status and performance. | DestinE platform Core Services | Service Operations Monitoring Dashboard (Agents) | Applicable to each DestinE platform Platform Management Service and Data Management Service |
| Runtime Platform | ||||
| RUN-INT-R- 0001 | Interface allowing Services to request needed cloud resources to deploy and operate their functions | Data Management Service | Runtime Platform | |
| OVH | ||||
| OVH-INT-P- 0001 | Interface allowing Runtime Platform to deploy OVH cloud resources | OVH Cloud Infrastructure layer | Runtime Platform | [RD-4] |
| OVH-EXT-IF- R-0020 | Interface allowing the DestinE platform Calculator to query OVH price lists | Calculator | OVH Cloud Infrastructure layer | |
| Calculator | ||||
| CALC-EXT- IF-P-0001 | Registered DestinE platform users can access the DestinE platform Calculator via the website GUI, accessible on the DestinE platform runtime platform | Calculator | DestinE platform Registered users | |
| CALC-EXT-IF | Registered DestinE platform users can access the DestinE platform Calculator via external API client and receive estimations directly | Calculator | DestinE platform Registered users | |
End of the document