EarthSelf will provide basic preconfigured services for your clusters i.e:
Solution 1: Infrastructure-as-a-Service Integration
1. Description
“IaaS” stands for Infrastructure as a Service. A first level of integration, designed for “IT professionals”. At this level of integration users access directly to the virtual infrastructure (such as virtual networks, virtual machines or storage partitioning) and can build fully customized services.
2. Advantages
The biggest advantage of the IaaS integration solution is the flexibility and amount of control available to operators of the infrastructure.
The choice of cloud provider benefits both infrastructure and application operators. Integrating at the virtual machine level allows the provisioning of computing resources tailored to the need of the application as closely as possible, depending on the underlying provider’s offerings and capabilities.
The IaaS integration solution permits tight control over the cost of deploying, operating and migrating an application depending on its resource requirements and the cost of infrastructure and capabilities offered by such or such cloud provider.
Operators in charge of the IaaS integration can provision virtual networks, virtual machines and storage capacities to form clusters fitting the exact requirements of the application they are deploying and operating. They can also dynamically provision or release those resources on the fly, in order to closely match the availability and performance constraints throughout the lifecycle of the application they operate.
3. Drawbacks
Integrating at the IaaS level means that integrators and/or application operators and/or application developers understand both the exact resources requirements of their applications, in order to maximize its efficiency and reduce costs, and the technical details of provisioning and managing cloud infrastructure.
Such integration solution may be uneasy for inexperienced operators and developers alike due to its increased complexity level.
Deploying an application in this context means provisioning virtual networks, machines and storage capacities as a forethought, meaning, in this instance, both planning in advance and consideration for the future of the application. The infrastructure, thus provisioned, is ready to host the application, and infrastructure operators must use or provide ways for application operators or developers to deploy the application onto the hosts and configure it to use the available resources.
Infrastructure operators must apply constant care to the health of the computing resources and interact with application operators or developers to notify them of any perturbation or inconsistencies at the infrastructure layer, while at the same time accommodate for static application requirements and its elasticity requirements.
Such ability requires visibility over the whole system provided by monitoring and alerting features which must be maintained by infrastructure operators as an additional burden.
Solution 2: Kubernetes-as-a-Service Integration
1. Description
“Kaas” stands for Kubernetes as a Service. A second level of integration, positioned on top of the IaaS layer, which includes all the functionalities offered by this layer but by providing an abstraction layer over virtualized computing resources and, a fortiori, over hardware resources. SNAPERATH users can then deploy and manage their applications using Kubernetes management tools they are the most familiar with by focusing on building, deploying and operating their applications without worrying about the underlying architecture.
The KaaS layer is also intended for “IT professionals” users, but optimizing Time To Market and Total Cost of Operation. The KaaS integration solution offers them the following services:
- Orchestration of micro services: To accelerate the Time To Market by simplify service deployments
- Observability and security enhancing: tackling the Total Cost of Operation by offering ergonomic solution to monitor performance and security and to understand and troubleshot applications.
2. Advantages
As a higher layer of abstraction on top of virtualized computing resources, integration at the container level, through Kubernetes, allows operators to deal with applications’ processing, memory and storage requirements and deployment topology rather than hosts and volumes. It frees operators and application developers from the burden of assigning services and replicas to specific hosts, also called scheduling.
Because Kubernetes provides a higher level of service than infrastructure management, it sits on top of higher level concepts which are abstraction over application deployment concepts. Examples of such concepts are Secrets, Pods, Volume Claims or Services, which represent opaque data available to application components, groups of containers running on a same host, dynamically allocated storage volumes or service discovery capabilities, respectively.
Built-in monitoring features provides integrators with automated ways to track and add more or retract unnecessary resources depending on the runtime performance and availability requirements of the application. Such elasticity thereby grants efficient cost savings by reducing the computing resources allocation at its lowest while sustaining the running application.
Git/Helm repositories watching and automated continuous deployment further simply integration with the platform. By pushing Kubernetes management files, or manifests, a background task will automatically deploy the Kubernetes resources described in those.
For a higher ease of use, the SnapEarth platform will also provide support for continuous integration and continuous deployment (CI/CD) through capabilities such as automated Git/Helm repositories watching (GitOps) so that integrators can easily push forward deployment changes through configuration management alone, without resorting to direct communication/interaction with their cluster.
3. Drawbacks
Container technologies and Kubernetes, while allowing operators and developers to manage high level concepts, come with additional unfamiliarity and an arguably steeper learning curve than traditional or cloud-based infrastructure management tools and techniques.
Managing computing resources at the container/Kubernetes level also means that neither the underlying virtualized hosts nor the underlying physical hardware is accessible to operators, which can be limiting for certain types of maintenance work or access to low level details and debug information.
As a relatively new technology, Kubernetes and Kubernetes management tools are still under very active development. Furthermore, managed Kubernetes clusters’ features may differ from one cloud provider to another. Parameters such the supported Kubernetes versions, maximum number of nodes in a cluster and SLAs (Service Layer Agreement) may defer quite a lot, even though Kubernetes concepts themselves are based on the lowest common denominator between infrastructure and deployment components.
Deployment container based applications using Kubernetes is solving the problem of easily deploying and updating scattered application services. And in doing so, it requires additional monitoring, logs/metrics aggregation, tracing, dashboards and alerting mechanisms in order to provide visibility on how the application is behaving throughout its lifespan.
Solution 3: Workflow-as-a-Service Integration
1. Description
“WaaS” stands for Workflow as a Service. The last level of integration, positioned on top of the KaaS and IaaS layers, includes all the functionalities offered by the layers underneath it in order to allow users to quickly create operational and secure services without requiring any knowledge of neither the underlying container orchestration mechanism nor its underlying infrastructure. This layer is intended for users, who are not IT professionals but who have good ideas and want to implement them by combining existing services to produce added value. This layer, allow them to focus on developing their individual processing tasks, using whichever technology they are the most familiar with; then to push them through Git repositories which will be managed through a workflow management layer. Thus, users will be able to access all the services on top of this layer (such as access to the EarthSignature database and the other SNAPEARTH services). In addition, they will not have to worry about the large-scale processing, (e.g. to reach national, continental or even global land cover computing areas).
2. Advantages
As yet another layer of indirection, WaaS provides a higher level of integration than KaaS or IaaS. The workflow abstraction allows application developers and operators to reason about applications as a series, or graph, of tasks to be executed, rather than an application to be deployed and maintained.
Resource constraints can be specified on individual units of work without having to worry about provisioning, allocation or deallocation of computing resources. Usage of the underlying infrastructure is thus tailored to the exact runtime needs of individual processing jobs or applications, increasing resource utilization efficiency and reducing costs.
Application developers can focus on developing individual processors, or tasks, using whichever technology they are the most familiar with such as any programming language packaged application or simple scripts. Data access can be made assuming data locality on the task’s POSIX-like file system by referring to distant objects via configuration only, leveraging efficient, automatic and cost saving data retrieval offered by the SnapEarth platform.
Workloads or applications requiring the consumption of external data sources or streams of events/data can also benefit from built-in ways to connect their processing tasks to ingoing or outgoing Web API/Web Hooks, event queues, storage watchers or scheduled (CRON-like) planning.
For a higher ease of use, the SnapEarth platform will also provide support for continuous integration and continuous deployment (CI/CD) through capabilities such as automated Git repositories watching (GitOps) so that operators and developers can easily push forward new workflow definitions, and let the SnapEarth platform perform deployment changes, without resorting to direct communication/interaction with their cluster.
3. Drawbacks
In the same manner that integration at the KaaS solution level abstracts the complexity of the IaaS solution, integration at the WaaS level abstracts the complexity of the KaaS solution, but does so by introducing new concepts that operators need to learn.
Moreover, workflow technologies, such as the workflow framework provided by the SnapEarth platform, but other too, don’t abstract the underlying container orchestration solution they are based on in its totality. This means that, sometimes, for specific tasks, operators and/or application developers may need to familiarize themselves with some concepts of the underlying stack, such as Kubernetes, in order to perform tasks and leverage the power of the platform.
Formation and resource docs will be available to users of the SnapEarth platform in order to get started with container orchestration and workflow technologies.