StatefulSets represent a set of Pods with unique, persistent identities and stable hostnames. It provides guarantees about the ordering of deployment and scaling.
StatefulSets are valuable for applications that require one or more of the following:
- Stable, unique network identifiers
- Stable, persistent storage
- Ordered, graceful deployment and scaling
- Ordered, graceful deletion and termination
If an application doesn’t require any stable identifiers or ordered deployment, deletion, or scaling, you should deploy your application with a controller such as Deployments or ReplicaSets that provides a set of stateless replicas.
Creating StatefulSets
Components
- A Headless Service
- A StatefulSet
- A PersistentVolume
The following is an example of a Service and StatefulSet manifest file:
apiVersion: v1 kind: Service metadata: name: nginx labels: app: nginx spec: ports: - port: 80 name: web clusterIP: None selector: app: nginx
--- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: selector: matchLabels: app: nginx serviceName: "nginx" replicas: 3 template: metadata: labels: app: nginx spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: nginx ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi
$ kubectl create -f statefulsets.yaml service "nginx" created statefulset.apps "web" created
It will create three Pods named web-0,web-1,web-2
$ kubectl get pods NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 1m web-1 1/1 Running 0 46s web-2 1/1 Running 0 18s
$ kubectl get svc nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx ClusterIP None 80/TCP 2m
Kubernetes creates one PersistentVolume for each VolumeClaimTemplate. In the nginx example above, each Pod will receive a single PersistentVolume and 1 Gib of provisioned storage.
$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec 1Gi RWO Delete Bound default/www-web-0 standard 4m pvc-5a7494e2-327b-11e9-a03c-42010a8000ec 1Gi RWO Delete Bound default/www-web-1 standard 4m pvc-6b820255-327b-11e9-a03c-42010a8000ec 1Gi RWO Delete Bound default/www-web-2 standard 4m
$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE www-web-0 Bound pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 4m www-web-1 Bound pvc-5a7494e2-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 4m www-web-2 Bound pvc-6b820255-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 4m
In this example:
- A Service object named nginx is created. The Service targets an app called nginx, indicated by labels: app: nginx and selector: app: nginx.
- The Service exposes port 80 and names it web. This Service controls the network domain and to route Internet traffic to the containerized application deployed by the StatefulSet.
- A StatefulSet named web is created with three replicated Pods (replicas: 3).
- The Pod template (spec: template) indicates that its Pods are labeled app: nginx.
- The Pod specification uses the web port opened by the Service.
- template: spec: volumeMounts specifies a mountPath, which is named www. The mountPath is the path within the container at which a storage volume should be mounted.
- The StatefulSet provisions a PersistentVolumeClaim, www, with 1GB of provisioned storage.
Pod specification contains the following instructions:
- Label each Pod as app: nginx.
- In each Pod, run one container named nginx.
- Have Pods use port 80.
- Save data to the mount path.
Scaling a StatefulSet
Scaling a StatefulSet refers to increasing or decreasing the number of replicas.
Scaling Up
$ kubectl scale statefulset web --replicas=5 statefulset.apps "web" scaled
$ kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 11m web-1 1/1 Running 0 10m web-2 1/1 Running 0 10m web-3 1/1 Running 0 33s web-4 1/1 Running 0 19s
$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE www-web-0 Bound pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 11m www-web-1 Bound pvc-5a7494e2-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 11m www-web-2 Bound pvc-6b820255-327b-11e9-a03c-42010a8000ec 1Gi RWO standard 10m www-web-3 Bound pvc-c8c7c417-327c-11e9-a03c-42010a8000ec 1Gi RWO standard 1m www-web-4 Bound pvc-d1086c7a-327c-11e9-a03c-42010a8000ec 1Gi RWO standard 53s
Scaling Down
$ kubectl get pods -w -l app=nginx NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 13m web-1 1/1 Running 0 12m web-2 0/1 Terminating 0 12m web-3 0/1 Terminating 0 2m web-4 0/1 Terminating 0 1m
$ kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 13m web-1 1/1 Running 0 12m
Updating StatefulSets
The strategy used is determined by the spec.updateStrategy field of the StatefulSet API Object.
There are two valid update strategies, RollingUpdate and OnDelete.
Deleting StatefulSets
Use kubectl delete to delete the StatefulSet.
$ kubectl delete statefulset web statefulset.apps "web" deleted
Get the Pods.
$ kubectl get pods -l app=nginx No resources found.
You must delete the nginx Service manually.
$ kubectl delete service nginx service "nginx" deleted
Cleaning up
You will need to delete the persistent storage media for the PersistentVolumes used in the example.
$ kubectl delete pvc --all persistentvolumeclaim "www-web-0" deleted persistentvolumeclaim "www-web-1" deleted persistentvolumeclaim "www-web-2" deleted persistentvolumeclaim "www-web-3" deleted persistentvolumeclaim "www-web-4" deleted
$ kubectl delete pv --all persistentvolume "pvc-4eaf7a74-327b-11e9-a03c-42010a8000ec" deleted persistentvolume "pvc-5a7494e2-327b-11e9-a03c-42010a8000ec" deleted persistentvolume "pvc-6b820255-327b-11e9-a03c-42010a8000ec" deleted persistentvolume "pvc-c8c7c417-327c-11e9-a03c-42010a8000ec" deleted persistentvolume "pvc-d1086c7a-327c-11e9-a03c-42010a8000ec" deleted
Limitations:
- StatefulSet was a beta resource prior to 1.9 and not available in any Kubernetes release prior to 1.5.
- The storage for a given Pod must either be provisioned by a PersistentVolume Provisioner based on the requested storage class, or pre-provisioned by an admin.
- Deleting and/or scaling a StatefulSet down will not delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources.
- StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.
- StatefulSets do not provide any guarantees on the termination of pods when a StatefulSet is deleted. To achieve ordered and graceful termination of the pods in the StatefulSet, it is possible to scale the StatefulSet down to 0 prior to deletion.