DOWNLOAD : The Cloud Data Integration Checklist from TDWI

How to Manage Access to 3rd Party Resources in Kubernetes with Helm

How to Manage Access to 3rd Party Resources in Kubernetes with Helm

  • Sébastien Gandon
    I worked for different compagnies such as Bouygues Telecom or Canal + at building robust software before joining Talend in 2010 for contributing to the Talend Studio architecture and especially to the Unified Platform concept. I am now part of the growing Talend Architecture department working on cross products concerns in our cloud offerings and helping shape the future of Talend.

Post By Sébastien Gandon and Iosif Igna

Kubernetes provides the ability to easily deploy and run containerized applications in cloud, on-premise, or hybrid environments. Kubernetes has gained a lot of attention recently and it has become a platform for innovation in containerized applications. One technology which has probably helped Kubernetes grow a lot is Helm, which provides the means to package, install, and manage Kubernetes applications.

At Talend, we are using Kubernetes and Helm for our cloud applications. In this post, we will show how we have used Kubernetes resources and a Helm chart to address a specific deployment challenge. 

The specific use case is a Kubernetes application that needs to connect to a PostgreSQL database.   First, we create a Helm chart to manage the deployment of the application. We don’t yet know how to provision PostgreSQL and how to manage the connection from the application to PostgreSQL.

Although the answers might seem trivial at first, it becomes more complicated when we consider different deployment strategies for PostgreSQL. Therefore, let’s start by looking at several possible PostgreSQL deployment scenarios, which we have tried over the course of our journey with Kubernetes and Helm.

Embedded Deployment Scenario

In this scenario, PostgreSQL is deployed alongside the application inside the Kubernetes cluster, as shown in the following diagram.

Figure 1 - Embedded Postgresql provisioning

While it might not be the perfect scenario for a production system, it provides an easy and flexible way to get a PostgreSQL up and running in a very short time.

The official Kubernetes Helm charts repository provides a PostgreSQL chart which installs and configures a PostgreSQL database inside the cluster. The database name, database user name, and database password can be provided in the values.yaml file or as input parameters at install time. The chart stores the database password in a Kubernetes secret which is then used by the pod that hosts the PostgreSQL container and by the applications which need to connect to the database.

OSBA Deployment Scenario

OSBA (Open Service Broker API) enables service providers to deliver services to applications running in a cloud-native platform, such as Kubernetes. The idea is to provision resources that are managed by a cloud provider using Kubernetes manifests.

In this scenario, we are using the Kubernetes service catalog to connect to a Microsoft Azure service broker and provision a PostgreSQL database.

Figure 2 - OSBA Postgresql provisioning

Azure Catalog service proposes 3 different PostgreSQL configurations:

  • Provision only the cluster
  • Provision the cluster and the DB
  • Provision a new DB on an existing cluster

We have chosen to provision both the cluster and the DB to have a setup similar to the embedded model.

Once the cluster/database has been provisioned, the service broker creates a secret inside the Kubernetes cluster which contains all parameters required to access the PostgreSQL database, such as: host, port, database name, user, password, etc. Like in the embedded scenario, this secret can be used by the applications which need to connect to the database.

It is important to notice that the created secret is highly dependent on the way the service broker provider decides how it is going to be formatted. The keys of the secret may vary from one cloud provider to another.

Learn more about the Open Service Broker for Azure and PostgreSQL.

External Deployment Scenario

This scenario implies that a PostgreSQL database is provisioned and managed outside the Kubernetes cluster. This could be a managed service in a cloud environment (i.e. Amazon RDS) or a self-managed PostgreSQL cluster in a cloud or on-premise environment.

Kubernetes provides a special service called “service without selector” to enable the communication from resources inside the cluster to resource outside the cluster. In this scenario, we are using a service without selector to connect from our application deployed inside the cluster to an external PostgreSQL database.

Figure 3 - External Postgresql Provisioning

The database host and access credentials are stored in a Kubernetes secret which can be accessed by our application inside the cluster. In this case the secret attributes are freely chosen to match our service’s required parameters.

The Problem

When we look at these three deployment scenarios, we see that in each case the application that uses PostgreSQL needs to be aware of the different secrets with their own names and content. The embedded scenario uses values and a secret provided by the official PostgreSQL Helm chart, the OSBA scenario uses a secret with vendor-specific attribute names, and in the external scenario we are free to define the secret name and its content.

This is fine if we only need to work with one deployment scenario. However, this might not always be the case. For instance, in the development phase we might want to start with an embedded deployment, but later in production, we may need to use a managed service from a cloud provider - therefore needing an OSBA deployment. We can see that there is a need for abstraction here, so that the application can access the database in the same way, independent of how and where the database is deployed.

The Solution

The solution we have chosen consists of a generic secret that provides the abstraction layer required by an application to connect to a database without having any knowledge of its whereabouts. The advantage of a secret, apart from being the resource to handle sensitive data, is that it also provides a way to synchronize the launch of pods. If you have a pod depending on a secret file or some environments variables depending on the secret, the pod will not even start before the secret is available.

Figure 4 - Generic Secret abstracts custom provisionings

Below is an example of the generic secret we create for accessing the PostgreSQL cluster and database.

Secret key

Secret value description

postgresql.database

the name of the DB

postgresql.host

host name of the cluster, can be an IP or a K8s service name

postgresql.password

master password

postgresql.port

cluster port, usually 5432

postgresql.user

master user

To create this generic secret for each of the deployment scenarios above, we use different mechanisms as described below.

Embedded scenario

The generic secret is created by a Kubernetes job from the values.yaml file and the PostgreSQL secret is created by the official PostgreSQL chart. We use the environment variables of the job to wait for the embedded PostgreSQL secret to be created. You can find an example here.

OSBA scenario

The OSBA provisioning process involves two Kubernetes resources, a service instance, and a service binding. The service binding describes the name of the secret to be created after the successful provisioning. So, just like in the embedded scenario, we are using a Kubernetes job that waits for the OSBA secret to be created and then creates our generic secret out of it. You can find an example here.

External scenario

This is the easiest scenario because the credentials data comes from outside, and therefore can be provided during the helm install process. So, a simple secret template is enough to create the generic secret. You can find an example here.

A New Level of Managing Connections

Kubernetes has taken the orchestration of containerized applications to a different level and is helping software vendors reduce the gap between development, QA, and production environments. At the same time, the options for 3rd party resources/services have increased significantly, and software vendors are faced with the challenge to build their applications in a way that provides enough flexibility to switch between a locally managed service and a cloud-based service.

In our example, we have shown how you can leverage Kubernetes and Helm to manage the connection from a Kubernetes application to a PostgreSQL database. This could be deployed either alongside the application inside the cluster, on-demand in a cloud environment, or pre-deployed outside the cluster. We have used a Helm chart to create a single generic secret which provides an abstraction layer between the application and the database. In this way, the application should never know where the database is deployed or change the way it connects to the database.

You may find the related helm charts that where created using this approach here: https://github.com/sgandon/helm-postgresql-multi/tree/master/tpostgresql

This post was inspired by a workshop that Talend did with Microsoft. We’d like to thank Gil Isaacs for organizing this workshop and Julien Corioland for his great knowledge and skills around Kubernetes and Azure.

 

Join The Conversation

0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *