Techblog: Connect GitLab with DigitalOcean’s Kubernetes

Techblog: Connect GitLab with DigitalOcean’s Kubernetes

Techblog: Connect GitLab with DigitalOcean’s Kubernetes

How cool would it be to have a Kubernetes Cluster managed by DigitalOcean and use it to test each branch in your self-hosted GitLab repository separately, as a Review App? If you want to find out, stick to this blog and we’ll guide you through it step by step.

 

Prerequisites:

  1. DigitalOcean account
  2. Knowledge about DigitalOcean’s dashboard
  3. Knowledge about GitLab
  4. Basic UNIX experience

Log in or register at https://cloud.digitalocean.com/ and create a new droplet. Choose the GitLab one-click app image and during the setup, add your SSH keys to the droplet. Register a domain or sub-domain, and point it to the IP address of the GitLab instance.

 

Connect to your GitLab droplet

To connect to your droplet you need an IP address. You can find your droplet’s IP address in the DigitalOcean dashboard. Open a terminal and type in the following command:

ssh root@<DROPLET_IP>

You should see Welcome to DigitalOcean's One-Click GitLab Droplet.

 

Configure the external URL for GitLab

In order for GitLab to display correct repository clone links to your users it needs to know the URL under which it is reached by your users, e.g. https://gitlab.example.com.

By default, when you specify an external_url starting with ‘https’, Nginx will no longer listen for unencrypted HTTP traffic on port 80. If you want to redirect all HTTP traffic to HTTPS you can use the redirect_http_to_https setting.

Add or edit the following lines in /etc/gitlab/gitlab.rb:

 

Enable Let’s Encrypt for GitLab

Add the following entries to /etc/gitlab/gitlab.rb to enable Let’s Encrypt support for the primary domain:

 

Reboot your GitLab instance

Run sudo gitlab-ctl reconfigure for the changes to take effect.

 

 

 

Create a DigitalOcean Kubernetes Cluster

DigitalOcean provides a nice way to create a Kubernetes Cluster without manual hassle.

- Go to your DigitalOcean dashboard.

- Create a Kubernetes cluster.

- Download the Kubernetes YAML file at the bottom of the page of your DigitalOcean Managed Kubernetes dashboard and move it somewhere you can find it on your local machine.

To connect to your Kubernetes Cluster you need to setup kubectl on your local machine with your Kubernetes Cluster YAML config.

- Install kubectl locally (https://kubernetes.io/docs/tasks/tools/install-kubectl/) and follow the install instructions for your platform.

- You can either copy the contents of the YAML file directly in the configfile in the ~/.kube directory or create an environment variable KUBECONFIG=/path/to/config/file in your ~/.bashrc.

 

 

 

Configure Service Accounts in your Kubernetes Cluster

In order for GitLab to authenticate with your Kubernetes Cluster we need to create a Service Account. Open a new terminal session and let’s start creating a Service Account!

First off we need to create a new namespace for GitLab.

kubectl create namespace gitlab-namespace

Let’s set our context to the newly created namespace so we don’t have to consistently add the --namespace=gitlab-namespace option to the commands.

kubectl config set-context $(kubectl config current-context) --namespace=gitlab-namespace

Now we are going to create the actual Service Account.

kubectl create -f service-account.yml

Next up is setting the correct permissions for the gitlab-user in the gitlab-namespace.

kubectl create clusterrolebinding gitlab-cluster-admin-binding --clusterrole=cluster-admin --serviceaccount=gitlab-namespace:gitlab-user

 

And

kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts

Kubernetes dashboard

To enter your Kubernetes dashboard you can follow the steps on: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/.

TL;DR
Execute the following commands in your terminal.

Now setup a proxy to your Kubernetes Cluster.

Use the link below to visit your Kubernetes Cluster:

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/overview?namespace=default

 

 

GitLab Kubernetes Integration

To use your newly created or existing Kubernetes Cluster in GitLab you will need connect it to a project. Go to your project and navigate to Operations/Kubernetes in the sidebar on the left.

Once you are on the Kubernetes page in GitLab you will see a big green button with the text ‘Add Kubernetes Cluster’. Click on it to continue adding your Kubernetes Cluster.

You will see a panel with 2 tabs; Create new Cluster on GKE and Add existing cluster. You should go to Add existing cluster since we created ours on DigitalOcean.

Now it is time to enter your existing Kubernetes Cluster details. You can retrieve this information by performing the following steps in your terminal.

ProTip: You should save the details in a file for later use.

 

Get Service Account definition

Execute the following command kubectl describe serviceAccounts gitlab-user to get information about the service-account we created earlier. Below is a dump of the result that we got in the terminal, you should see something similar.

The things that are important here are the Mountable secrets and Tokens.

 

Get secret definition

Execute the following command kubectl describe secret [gitlab-user-token-b468q] to get the token. The result below is something you should be seeing in your terminal.

 

Getting the token’s certificate

We are interested in the token which is base64 encoded. The token has a payload with the certificate in it. Execute the following to retrieve the certificate.

In the terminal you see the certificate starting with -----BEGIN CERTIFICATE-----. Copy everything including -----END CERTIFICATE----- to your the file your are using to keep track of the details.

 

Get your Kubernetes Cluster API URL

You can find the API URL by executing the following command kubectl config view. You should see something similar to:

 

Enter details in GitLab

Now that you have all the details we can start entering them in GitLab. Go back to GitLab and you should see a form with the following input fields (Some may not be visible to you).

Kubernetes Cluster name
The name you wish to give the cluster.

Environment scope
The default environment scope is *, which means all jobs, regardless of their environment, will use that cluster. Each scope can only be used by a single cluster in a project, and a validation error will occur if otherwise. Also, jobs that don’t have an environment keyword set will not be able to access any cluster.

API URL
It’s the URL that GitLab uses to access the Kubernetes API. Kubernetes exposes several APIs, we want the “base” URL that is common to all of them, e.g., https://kubernetes.example.com rather than https://kubernetes.example.com/api/v1. We got the URL from the kubectl config view command we did earlier.

CA Certificate
If the API is using a self-signed TLS certificate, you’ll also need to include the ca.crt contents here. We decoded this from the token.

Token
GitLab authenticates against Kubernetes using service tokens, which are scoped to a particular namespace. Enter the base64 encoded token here.

Project namespace (optional, unique)
You don’t have to fill it in; by leaving it blank, GitLab will create one for you.

RBAC-enabled cluster
Enable it, your cluster is role-based this was done in chapter 2.

Once you filled in all the fields with the correct values. You can add the Kubernetes Cluster and you will be redirected to the Kubernetes Cluster page in Gitlab. Next step is to install some Helm Charts.

 

 

 

Install Helm Charts

The Kubernetes cluster page gives you some packages that you can install. Begin with Helm Tiller. Once this package is installed you get access to the other packages.

If you only want to use your Kubernetes cluster for pipelines, you will need to install the Gitlab Runner package.

If you want to run Review Apps and use Let’s Encrypt you will need to install the following package as well:

 

 

Gitlab’s Kubernetes Review Apps

This is an example of a simple Nginx web-server that does nothing more than to serve the Nginx index.html.

The .gitlab-ci.yml now has a review_app and a stop_review_app job. The review_app job applies 3 ConfigMaps to your Kubernetes Cluster. It creates a Deployment, Service and an Ingress LoadBalancer.

Below you will find the Kubernetes ConfigMaps used in the review_app stage in the .gitlab-ci.yml. Don't forget to substitute the environment variables, YAML will not do this for you. You can substitute the environment variables like this in your CI job, before executing the kubectl commands:

envsubst < /path/to/templates/kubernetes-service.yml > /path/to/kubernetes-service.yml

When a request is sent to your server, the Ingress LoadBalancer will try to resolve it based on the host. By default it will be forwarded to the default backend controller. To forward the request to your Nginx webserver you will need to create a service.

Deployments are used to create a set of containers, just like with docker-compose you have the freedom to choose an image, set ports and bind volumes.

Last but not least is the Ingress LoadBalancer. This will map any host to a service in the rules block. Basically all you have to do is say to the Ingress LoadBalancer that any traffic on the host ${CI_ENVIRONMENT_SLUG}.example.com will be forwarded to the ${CI_ENVIRONMENT_SLUG}-app-service.

 

Kubernetes CronJob

Sometimes you want to run scheduled commands which are executed in a different container. Kubernetes has a type for that, it’s called CronJob. Below is a example of how you can integrate this in your Review App.

Create a bash script with the following contents with filename cronjob.

This will run a Job every 15 minutes and dynamically resolves the pod name of the app container we created in the configuration in chapter 6. In the .gitlab-ci.yml go to the job review_app and add the following lines to the script array.

The reason we need the sleep 5 is so we give Kubernetes the time to create the pods we need for the CronJob.

Don’t forget to remove the CronJob in the stop_review_app job with the following command.

 

That’s all folks

That was quite a blog, but we hope you managed to create your Kubernetes Cluster and integrated it in your GitLab repository. If you have any questions or run into any problems, drop a comment below and we hope to give you an answer.

Due to the fact that multiple authors are not supported by Medium. I would like to add that Robin Bänffer is co-author in this blog.