Create Longhorn Volumes
In this tutorial, you’ll learn how to create Kubernetes persistent storage resources of persistent volumes (PVs) and persistent volume claims (PVCs) that correspond to Longhorn volumes. You will use kubectl to dynamically provision storage for workloads using a Longhorn storage class. For help creating volumes from the Longhorn UI, refer to this section.
This section assumes that you understand how Kubernetes persistent storage works. For more information, see the Kubernetes documentation.
First, you will create a Longhorn StorageClass. The Longhorn StorageClass contains the parameters to provision PVs.
Next, a PersistentVolumeClaim is created that references the StorageClass. Finally, the PersistentVolumeClaim is mounted as a volume within a Pod.
When the Pod is deployed, the Kubernetes master will check the PersistentVolumeClaim to make sure the resource request can be fulfilled. If storage is available, the Kubernetes master will create the Longhorn volume and bind it to the Pod.
Use following command to create a StorageClass called longhorn
:
kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v1.7.2/examples/storageclass.yaml
The following example StorageClass is created:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: longhorn
provisioner: driver.longhorn.io
allowVolumeExpansion: true
parameters:
numberOfReplicas: "3"
staleReplicaTimeout: "2880" # 48 hours in minutes
fromBackup: ""
fsType: "ext4"
# mkfsParams: "-I 256 -b 4096 -O ^metadata_csum,^64bit"
# diskSelector: "ssd,fast"
# nodeSelector: "storage,fast"
# recurringJobSelector: '[
# {
# "name":"snap",
# "isGroup":true,
# },
# {
# "name":"backup",
# "isGroup":false,
# }
# ]'
In particular, starting with v1.4.0, the parameter mkfsParams
can be used to specify filesystem format options for each StorageClass.
Create a Pod that uses Longhorn volumes by running this command:
kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v1.7.2/examples/pod_with_pvc.yaml
A Pod named volume-test
is launched, along with a PersistentVolumeClaim named longhorn-volv-pvc
. The PersistentVolumeClaim references the Longhorn StorageClass:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: longhorn-volv-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 2Gi
The persistentVolumeClaim is mounted in the Pod as a volume:
apiVersion: v1
kind: Pod
metadata:
name: volume-test
namespace: default
spec:
containers:
- name: volume-test
image: nginx:stable-alpine
imagePullPolicy: IfNotPresent
volumeMounts:
- name: volv
mountPath: /data
ports:
- containerPort: 80
volumes:
- name: volv
persistentVolumeClaim:
claimName: longhorn-volv-pvc
More examples are available here.
It is possible to use a Longhorn StorageClass to bind a workload to a PV without creating a StorageClass object in Kubernetes.
Since the Storage Class is also a field used to match a PVC with a PV, which doesn’t have to be created by a Provisioner, you can create a PV manually with a custom StorageClass name, then create a PVC asking for the same StorageClass name.
When a PVC requests a StorageClass that does not exist as a Kubernetes resource, Kubernetes will try to bind your PVC to a PV with the same StorageClass name. The StorageClass will be used like a label to find the matching PV, and only existing PVs labeled with the StorageClass name will be used.
If the PVC names a StorageClass, Kubernetes will:
Since the Longhorn volume already exists while creating PV/PVC, a StorageClass is not needed for dynamically provisioning Longhorn volume. However, the field storageClassName
should be set in PVC/PV, to be used for PVC bounding purpose. And it’s unnecessary for users to create the related StorageClass object.
By default the StorageClass for Longhorn created PV/PVC is longhorn-static
. Users can modify it in Setting - General - Default Longhorn Static StorageClass Name
as they need.
Users need to manually delete PVC and PV created by Longhorn.
Now users can create PV/PVC via our Longhorn UI for the existing Longhorn volumes. Only detached volume can be used by a newly created pod.
Creating a Longhorn volume will fail if there are no available nodes, disks, or insufficient storage. The failures are categorized into:
The failure results in the workload failing to use the provisioned PV and showing a warning message
# kubectl describe pod workload-test
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedAttachVolume 14s (x8 over 82s) attachdetach-controller AttachVolume.Attach
failed for volume "pvc-e130e369-274d-472d-98d1-f6074d2725e8" : rpc error: code = Aborted
desc = volume pvc-e130e369-274d-472d-98d1-f6074d2725e8 is not ready for workloads
In order to help users understand the error causes, Longhorn summarizes them in the PV annotation, longhorn.io/volume-scheduling-error
. Failures are combined in this annotation and separated by a semicolon, for example, longhorn.io/volume-scheduling-error: insufficient storage;disks are unavailable
. The annotation can be checked by using kubectl describe pv <pvc name>
.
# kubectl describe pv pvc-e130e369-274d-472d-98d1-f6074d2725e8
Name: pvc-e130e369-274d-472d-98d1-f6074d2725e8
Labels: <none>
Annotations: longhorn.io/volume-scheduling-error: insufficient storage
pv.kubernetes.io/provisioned-by: driver.longhorn.io
...
© 2019-2024 Longhorn Authors | Documentation Distributed under CC-BY-4.0
© 2024 The Linux Foundation. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page.