StatefulSets

StatefulSets represent a set of Pods with unique, persistent identities and stable hostnames. It provides guarantees about the ordering of deployment and scaling.

StatefulSets are valuable for applications that require one or more of the following:

  • Stable, unique network identifiers
  • Stable, persistent storage
  • Ordered, graceful deployment and scaling
  • Ordered, graceful deletion and termination

If an application doesn’t require any stable identifiers or ordered deployment, deletion, or scaling, you should deploy your application with a controller such as Deployments or ReplicaSets that provides a set of stateless replicas.

Creating StatefulSets

Components

  • A Headless Service
  • A StatefulSet
  • A PersistentVolume

The following is an example of a Service and StatefulSet manifest file:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
$ kubectl create -f statefulsets.yaml 
service "nginx" created
statefulset.apps "web" created

It will create three Pods named web-0,web-1,web-2

$ kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE
web-0     1/1       Running   0          1m
web-1     1/1       Running   0          46s
web-2     1/1       Running   0          18s
$ kubectl get svc nginx
NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
nginx     ClusterIP   None                 80/TCP    2m

Kubernetes creates one PersistentVolume for each VolumeClaimTemplate. In the nginx example above, each Pod will receive a single PersistentVolume and 1 Gib of provisioned storage.

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM               STORAGECLASS   REASON    AGE
pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec   1Gi        RWO            Delete           Bound     default/www-web-0   standard                 4m
pvc-5a7494e2-327b-11e9-a03c-42010a8000ec   1Gi        RWO            Delete           Bound     default/www-web-1   standard                 4m
pvc-6b820255-327b-11e9-a03c-42010a8000ec   1Gi        RWO            Delete           Bound     default/www-web-2   standard                 4m
$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
www-web-0   Bound     pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       4m
www-web-1   Bound     pvc-5a7494e2-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       4m
www-web-2   Bound     pvc-6b820255-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       4m

In this example:

  • A Service object named nginx is created. The Service targets an app called nginx, indicated by labels: app: nginx and selector: app: nginx.
  • The Service exposes port 80 and names it web. This Service controls the network domain and to route Internet traffic to the containerized application deployed by the StatefulSet.
  • A StatefulSet named web is created with three replicated Pods (replicas: 3).
  • The Pod template (spec: template) indicates that its Pods are labeled app: nginx.
  • The Pod specification uses the web port opened by the Service.
  • template: spec: volumeMounts specifies a mountPath, which is named www. The mountPath is the path within the container at which a storage volume should be mounted.
  • The StatefulSet provisions a PersistentVolumeClaim, www, with 1GB of provisioned storage.

Pod specification contains the following instructions:

  • Label each Pod as app: nginx.
  • In each Pod, run one container named nginx.
  • Have Pods use port 80.
  • Save data to the mount path.

Scaling a StatefulSet

Scaling a StatefulSet refers to increasing or decreasing the number of replicas.

Scaling Up

$ kubectl scale statefulset web --replicas=5
statefulset.apps "web" scaled
$ kubectl get pods -l app=nginx
NAME      READY     STATUS    RESTARTS   AGE
web-0     1/1       Running   0          11m
web-1     1/1       Running   0          10m
web-2     1/1       Running   0          10m
web-3     1/1       Running   0          33s
web-4     1/1       Running   0          19s
$ kubectl get pvc
NAME        STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
www-web-0   Bound     pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       11m
www-web-1   Bound     pvc-5a7494e2-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       11m
www-web-2   Bound     pvc-6b820255-327b-11e9-a03c-42010a8000ec   1Gi        RWO            standard       10m
www-web-3   Bound     pvc-c8c7c417-327c-11e9-a03c-42010a8000ec   1Gi        RWO            standard       1m
www-web-4   Bound     pvc-d1086c7a-327c-11e9-a03c-42010a8000ec   1Gi        RWO            standard       53s

Scaling Down

$ kubectl get pods -w -l app=nginx
NAME      READY     STATUS        RESTARTS   AGE
web-0     1/1       Running       0          13m
web-1     1/1       Running       0          12m
web-2     0/1       Terminating   0          12m
web-3     0/1       Terminating   0          2m
web-4     0/1       Terminating   0          1m
$ kubectl get pods -l app=nginx
NAME      READY     STATUS    RESTARTS   AGE
web-0     1/1       Running   0          13m
web-1     1/1       Running   0          12m

Updating StatefulSets

The strategy used is determined by the spec.updateStrategy field of the StatefulSet API Object.

There are two valid update strategies, RollingUpdate and OnDelete.

Deleting StatefulSets

Use kubectl delete to delete the StatefulSet.

$ kubectl delete statefulset web
statefulset.apps "web" deleted

Get the Pods.

$ kubectl get pods -l app=nginx
No resources found.

You must delete the nginx Service manually.

$ kubectl delete service nginx
service "nginx" deleted

Cleaning up

You will need to delete the persistent storage media for the PersistentVolumes used in the example.

$ kubectl delete pvc --all
persistentvolumeclaim "www-web-0" deleted
persistentvolumeclaim "www-web-1" deleted
persistentvolumeclaim "www-web-2" deleted
persistentvolumeclaim "www-web-3" deleted
persistentvolumeclaim "www-web-4" deleted
$ kubectl delete pv --all
persistentvolume "pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec" deleted
persistentvolume "pvc-5a7494e2-327b-11e9-a03c-42010a8000ec" deleted
persistentvolume "pvc-6b820255-327b-11e9-a03c-42010a8000ec" deleted
persistentvolume "pvc-c8c7c417-327c-11e9-a03c-42010a8000ec" deleted
persistentvolume "pvc-d1086c7a-327c-11e9-a03c-42010a8000ec" deleted

Limitations:

  • StatefulSet was a beta resource prior to 1.9 and not available in any Kubernetes release prior to 1.5.
  • The storage for a given Pod must either be provisioned by a PersistentVolume Provisioner based on the requested storage class, or pre-provisioned by an admin.
  • Deleting and/or scaling a StatefulSet down will not delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources.
  • StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.
  • StatefulSets do not provide any guarantees on the termination of pods when a StatefulSet is deleted. To achieve ordered and graceful termination of the pods in the StatefulSet, it is possible to scale the StatefulSet down to 0 prior to deletion.