GitPod Self Host Installation on K3S

GitPod Installation

Step 1: Create a Kubernetes Cluster

Gitpod is a Kubernetes application that uses Kubernetes internally to provision workspaces as Kubernetes pods. Before you can start installing Gitpod, you need to create a compatible Kubernetes cluster.

Install K3S Kubernets Cluster

Cluster Set-Up

Gitpod is a Kubernetes application running with certain expectations on the characteristics of the cluster it is running on.

Gitpod requires Kubernetes as an orchestration technology in order to spin up and down workspaces—ideally in combination with cluster autoscaling to minimise cost. We strongly recommend deploying a dedicated Kubernetes cluster just for Gitpod Self-Hosted.

In this article we will use k8s to setup a self managed kubernetes cluster

K3s is a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.

On each node, we install K3s. We configure K3s by setting the following environment variables on the nodes.

K3s config for main node node0:

The below configure is for a single node setup with only master node

export INSTALL_K3S_EXEC="server --disable traefik --flannel-backend=none --node-label gitpod.io/workload_meta=true --node-label gitpod.io/workload_ide=true -node-label gitpod.io/workload_workspace_services=true --node-label gitpod.io/workload_workspace_regular=true --node-label gitpod.io/workload_workspace_headless=true"
export K3S_CLUSTER_SECRET="44dc0c2d471f50bc151aa72515d53067"
curl -sfL https://get.k3s.io | sh -

After setting the environment variables, install K3s on every node like this:

$ curl -sfL https://get.k3s.io | sh -

# Check for Ready node,
takes maybe 30 seconds
k3s kubectl get node

You can run the below command to start the server

systemctl start k3s.service
# Kubeconfig is written to /etc/rancher/k3s/k3s.yaml
k3s kubectl get node

To enable any other node to join the cluster run the command

# On a different node run the below. NODE_TOKEN comes from /var/lib/rancher/k3s/server/node-token
# on your server
export INSTALL_K3S_EXEC="agent --node-label gitpod.io/workload_workspace_services=true --node-label gitpod.io/workload_workspace_regular=true --node-label gitpod.io/workload_workspace_headless=true"
export K3S_CLUSTER_SECRET="<your random secret string that is the same on all nodes>"
export K3S_URL="https://node0:6443"

Now, you have to install Calico.

Method 1

Download the Calico manifest and add the following line to the plugins section of the cni_network_config:

"container_settings": { "allow_ip_forwarding": true }

/var/lib/rancher/k3s/server/manifests/

Copy that file to node0 in the following folder (create folder if missing):

Method 2

Install the Calico operator and custom resource definitions.

kubectl create -f https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml

Install Calico by creating the necessary custom resource. For more information on configuration options available in this manifest, see the installation reference.

kubectl create -f https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml

Note: Before creating this manifest, read its contents and make sure its settings are correct for your environment. For example, you may need to change the default IP pool CIDR to match your pod network CIDR.

Final checks

1. Confirm that all of the pods are running using the following command.

watch kubectl get pods --all-namespaces

Wait until each pod shows the STATUS of Running.

Confirm that you now have a node in your cluster with the following command.

kubectl get nodes -o wide

References

• https://www.gitpod.io/docs/self-hosted/latest/cluster-set-up/on-k3s
• https://www.gitpod.io/docs/self-hosted/latest/getting-started#step-2-install-cert-manager
• https://projectcalico.docs.tigera.io/getting-started/kubernetes/k3s/quickstart

Step 2: Install Cert-Manager

Once you have created your Kubernetes cluster you need to install cert-manager. cert-manager is needed in any case even when you bring your own TLS certificate for your domain.

Install cert-manager on kubernetes cluster

cert-manager is a Kubernetes add-on to automate the management and issuance of TLS certificates from various issuing sources.

It will ensure certificates are valid and up to date periodically, and attempt to renew certificates at an appropriate time before expiry.

on the same node where k3s master was installed run the below command to install the cert manager

k3s kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.yaml
    

By default, cert-manager will be installed into the cert-manager namespace

Creating TLS certs for your domain with cert-manager

cert-manager provides the Gitpod installation with certificates for internal communication. Besides this, cert-manager can also create a TLS certificate for your domain. Since Gitpod needs wildcard certificates, you must use the DNS-01 challenge.

You can use either an Issuer or ClusterIssuer.

Issuers, and ClusterIssuers, are Kubernetes resources that represent certificate authorities (CAs) that are able to generate signed certificates by honoring certificate signing requests. All cert-manager certificates require a referenced issuer that is in a ready condition to attempt to honor the request.

we will use the domain miko-robot.co.in , DNS provide is onlydomains . We will first delegate the domain to cloudflare and then use cloudflare for DNS01 challenge

Create and Account and Login into cloudflare . Click on Add a Site option to start the domain delegation process

For now choose the Free option and continue

Log in to the administrator account for your domain registrar . In this case domain registrar is onlydomain.com .

1. By Default the following nameservers are configured

   ns2.onlydomains.com , ns3.onlydomains.com , ns1.onlydomains.com
   

2. Go to DNS Settings Menu

Choose the option to delegate name server Add Cloudflare’s nameservers

tegan.ns.cloudflare.com

zod.ns.cloudflare.com

This will update the DNS Settings and allow the DNS to be managed via cloudflare

Create CloudFlare API Tokens

Tokens can be created at User Profile > API Tokens > API Tokens. The following settings are recommended:

• Permissions:
  • Zone - DNS - Edit
  • Zone - Zone - Read
• Zone Resources:
  • Include - All Zones

Copy The token and save it as it will not be displayed again for security purposes

Verify that the token is working

curl -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify"
-H "Authorization: Bearer {TOKEN}"
-H "Content-Type:application/json"

If token is working then you will see a output similar to one below

{"result":{"id":"cf4a06f05d43d58468667ba715145c34","status":"active"},"success":true,"errors":[],"messages":[{"code":10000,"message":"This API Token is valid and active","type":null}]}

Create a new Issuer

To create a new Issuer, first make a Kubernetes secret containing your new API token:

apiVersion: v1
kind: Secret
metadata:
 name: cloudflare-api-token-secret
 namespace: cert-manager
type: Opaque
stringData:
 api-token: {TOKEN}

Please note that the namespace has to be cert-manager for the key else you may encounter error while creating the certificates

To apply the configuration

k3s kubectl apply -f cloudflare_token.yaml

Create Issuer configuration file

kind: ClusterIssuer
metadata:
  name: gitpod-issuer
spec:
  acme:
    email: prashant@miko.ai
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: gitpod-issuer
    solvers:
    - dns01:
        cloudflare:
          email : aaa@aaa.com
          apiTokenSecretRef:
            name: cloudflare-api-token-secret
            key: api-token

Once you complete the gitpod installation create the below certificates in gitpod and kube-system workspace both

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
 name: https-certificates
 namespace : kube-system
spec:
 secretName: https-certificates
 issuerRef:
   name: gitpod-issuer
   kind: ClusterIssuer
 dnsNames:
  - gitpod.miko-robot.co.in
  - "*.gitpod.miko-robot.co.in"
  - "*.ws.gitpod.miko-robot.co.in"

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
 name: https-certificates
 namespace : gitpod
spec:
 secretName: https-certificates
 issuerRef:
   name: gitpod-issuer
   kind: ClusterIssuer
 dnsNames:
  - gitpod.miko-robot.co.in
  - "*.gitpod.miko-robot.co.in"
  - "*.ws.gitpod.miko-robot.co.in"

To apply the configuration

k3s kubectl apply -f cert.yaml

While certificate issuance process is in progress you will see the status as False

kubectl get certificate
NAME                        READY   SECRET                      AGE
https-certificates          False    https-certificates          5m

After a few minutes, you should see the https-certificate become ready.

kubectl get certificate
NAME                        READY   SECRET                      AGE
https-certificates          True    https-certificates          5m

Once the DNS record has been updated, you can delete all Cert Manager pods to retrigger the certificate request

kubectl delete pods -n cert-manager --all

References

https://cert-manager.io/docs/configuration/acme/dns01/cloudflare/

Step 3: Configure DNS

To install Gitpod you need a domain with a TLS certificate. The DNS setup to your domain needs to be configured such that it points to the ingress of your Kubernetes cluster.

Configure DNS

To install Gitpod you need a domain with a TLS certificate. The DNS setup to your domain needs to be configured such that it points to the ingress of your Kubernetes cluster. You need to configure your actual domain (say example.com) as well as the wildcard subdomains *.example.com as well as *.ws.example.com.

Step 4: Install Gitpod

GitPod Installation

kubectl kots install gitpod

You will be asked for the namespace you want to install Gitpod to as well as a password for the admin console. After some time, you will see the following output:

  • Press Ctrl+C to exit
  • Go to http://localhost:8800 to access the Admin Console

To access the Admin Console again, run kubectl kots admin-console –namespace gitpod

Open your favorite browser and go to http://localhost:8800 (port 8800 is opened on your node on localhost only—you may want to forward the port to your workstation in order to access the admin console).

we will use ngrok for port forwarding

ngrok http 8800

The first page will ask you to upload your Gitpod license. Gitpod provides community license for free (right click and save link as here)

.

Enter the domain name as :gitpod.miko-robot.co.in

User all the default settings for incluster container registry , mysql database , storage provide

User cert-manager for SSL certificates and use Issuer type as “Issuer”

Mark Allow login by into workspace via ssh

Check if all the pods are running states

kubectl -n gitpod patch svc proxy -p '{"spec": {"type": "LoadBalancer", "externalIPs":["xxx.xxx.xxx.xx"]}}'

Once the installation has been finished successfully, you will see the status “Ready” with a small green indicator next to the Gitpod logo. You see which version you installed and which license you are using.