Infrastructure

Kubernetes

Determine how and if we want to give access to the consortium for k8s API’s. Other option is to use deployment pipelines for everything.

Object storage

ESA cloud as ‘glacier’ and redundant storage. CDSE (Cloudferro) as fast access mechanism close to processing resources.

For ESA cloud, we are asked what the best design would be. The data should be accessible over HTTP, preferably with very fast access. Firewalls and proxies in between are known to reduce performance in a number of cases.

Identity and access management

Partner: Inuits

ESA EOIAM needs to be integrated as one of the identity providers. Earth code project will use GitHub as IDP.

Question: will Earth code also use the teams API, for authorization purposes? https://docs.github.com/en/rest/teams/teams

APEx project requirements for authorization: SSO across APEx Avoid use of generic accounts, require proper identity. Projects will need to group people in teams.

Proposal: For MVP, skip creation of groups and group-based permissions. This means that per-component, permissions may need to be configured per user.

This allows us to better align with Earth-Code/EOEPCA+/CDSE initiatives for group management.

Group management alternatives

Github has teams, user-friendly interface but proprietary. EGI-Checkin has virtual organizations, following AARC blueprint architecture, so more standardized.

Relationship with platform identities

APEx will rely on NoR services, but these services may have a different IDP. It could be a requirement for services to integrate the same IDP?

Monitoring & Metrics

Docker registry

Shared components

Continuous integration

Github actions

Implement custom runners that run within the APEx k8s environment.

Terradue requires ArgoCD to be able to deploy instantiation services. MVP: basic actions + custom runner using kubectl + helm

Elasticsearch

Used by catalog Other usages, or not really a shared component?

Vault

To store secrets

Binary artifact management

User algorithms may have dependencies on binary files. These are usually not stored in Git.

The most basic approach to serving such files is an object storage bucket, but more advanced solutions also exist.