diff --git a/DOC/KUBERNETES.md b/DOC/KUBERNETES.md index 6479ebf4..3e032a3c 100644 --- a/DOC/KUBERNETES.md +++ b/DOC/KUBERNETES.md @@ -1,109 +1,24 @@ # Running rqlite on Kubernetes -This document provides an example of how to run rqlite as a Kubernetes [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/). +This document provides an example of how to run rqlite as a Kubernetes [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/). The [source for the configuration below is available in the Kubernetes Configuration repo](https://github.com/rqlite/kubernetes-configuration). ## Creating a cluster ### Create Services The first thing to do is to create two [Kubernetes _Services_](https://kubernetes.io/docs/concepts/services-networking/service). The first service, `rqlite-svc-internal`, is [_Headless_](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services) and allows the nodes to find each other and cluster automatically. It shouldn't be used by rqlite clients. It is the second service, `rqlite-svc`, that is for clients to talk to the cluster -- this service will get a Cluster IP address which those clients can connect to. A key difference between `rqlite-svc-internal` and `rqlite-svc` is that the second will only contain Pods that are ready to serve traffic. This makes it most suitable for use by end-users of rqlite. - -```yaml -apiVersion: v1 -kind: Service -metadata: - name: rqlite-svc-internal -spec: - clusterIP: None - publishNotReadyAddresses: True - selector: - app: rqlite - ports: - - protocol: TCP - port: 4001 - targetPort: 4001 ---- -apiVersion: v1 -kind: Service -metadata: - name: rqlite-svc -spec: - selector: - app: rqlite - ports: - - protocol: TCP - port: 4001 - targetPort: 4001 -``` -Apply the configuration above to your Kubernetes deployment. - ```bash -kubectl apply -f headless-service.yaml +curl -s https://raw.githubusercontent.com/rqlite/kubernetes-configuration/master/service.yaml -O rqlite-service.yaml +kubectl apply -f rqlite-service.yaml ``` -where the file `headless-service.yaml` contains the configuration shown above. ### Create a StatefulSet -For a rqlite cluster to function properly in a production environment, the rqlite nodes require a persistent network identifier and storage. This is what a _StatefulSet_ can provide. The example belows shows you how to configure a 3-node rqlite cluster. -```yaml -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: rqlite -spec: - selector: - matchLabels: - app: rqlite # has to match .spec.template.metadata.labels - serviceName: rqlite-svc-internal - replicas: 3 # by default is 1 - podManagementPolicy: "Parallel" - template: - metadata: - labels: - app: rqlite # has to match .spec.selector.matchLabels - spec: - terminationGracePeriodSeconds: 10 - containers: - - name: rqlite - image: rqlite/rqlite - args: ["-disco-mode=dns","-disco-config={\"name\":\"rqlite-svc-internal\"}","-bootstrap-expect","3", "-join-interval=1s", "-join-attempts=120"] - ports: - - containerPort: 4001 - name: rqlite - readinessProbe: - httpGet: - scheme: HTTP - path: /readyz - port: 4001 - initialDelaySeconds: 15 - periodSeconds: 5 - timeoutSeconds: 2 - livenessProbe: - httpGet: - scheme: HTTP - path: /readyz?noleader - port: rqlite - initialDelaySeconds: 15 - timeoutSeconds: 5 - failureThreshold: 3 - volumeMounts: - - name: rqlite-file - mountPath: /rqlite/file - volumeClaimTemplates: - - metadata: - name: rqlite-file - spec: - accessModes: [ "ReadWriteOnce" ] - storageClassName: "standard" - resources: - requests: - storage: 1Gi -``` -Apply this configuration to your Kubernetes system, and a 3-node rqlite cluster will be created. +For a rqlite cluster to function properly in a production environment, the rqlite nodes require a persistent network identifier and storage. This is what a _StatefulSet_ can provide. Retrieve the Stateful set configuration, apply it, and a 3-node rqlite cluster will be created. ```bash -kubectl apply -f stateful-set.yaml +curl -s https://raw.githubusercontent.com/rqlite/kubernetes-configuration/master/statefulset-3-node.yaml -o rqlite-3-nodes.yaml +kubectl apply -f rqlite-3-nodes.yaml ``` -where the file `stateful-set.yaml` contains the configuration shown above. -Note the `args` passed to rqlite. The arguments tell rqlite to use `dns` discovery mode, and to resolve the DNS name `rqlite-svc-internal` to find the IP addresses of other nodes in the cluster. Furthermore it tells rqlite to wait until three nodes are available (counting itself as one of those nodes) before attempting to form a cluster. +Note the `args` passed to rqlite in the YAML file. The arguments tell rqlite to use `dns` discovery mode, and to resolve the DNS name `rqlite-svc-internal` to find the IP addresses of other nodes in the cluster. Furthermore it tells rqlite to wait until three nodes are available (counting itself as one of those nodes) before attempting to form a cluster. ## Scaling the cluster You can grow the cluster at anytime, simply by setting **both** the replica count **and** `bootstrap-expect` argument to the desired cluster size. Shrinking the cluster, however, will require some manual intervention. As well reducing `replicas` and `bootstrap-expect`, you also need to [explicitly remove](https://github.com/rqlite/rqlite/blob/master/DOC/CLUSTER_MGMT.md#removing-or-replacing-a-node) the deprovisioned nodes, or the Leader will continually attempt to contact those nodes.