If users want to create nodes with large storage spaces and/or CPU resources for Longhorn only (to store replica data) and reject other general workloads, they can taint those nodes and add tolerations for Longhorn components. Then Longhorn can be deployed on those nodes.

Notice that the taint tolerations setting for one workload will not prevent it from being scheduled to the nodes that don’t contain the corresponding taints.

For more information about how taints and tolerations work, refer to the official Kubernetes documentation.

Setting up Taints and Tolerations

Longhorn consists of user-deployed components (for example, Longhorn Manager, Longhorn Driver, and Longhorn UI) and system-managed components (for example, Instance Manager, Backing Image Manager, Share Manager, CSI Driver, and Engine Image). You need to set tolerations for both types of components. See more details below.

Setting up Taints and Tolerations During installing Longhorn

1. Set taint tolerations for user-deployed components (for example, Longhorn Manager, Longhorn Driver, and Longhorn UI).

   • If you install Longhorn through Rancher, you must copy and paste the following parameters into the YAML on the Rancher UI (click Edit as YAML during the installation) to apply the value to all user-deployed components.

       global:
         tolerations:
         - key: "key"
           operator: "Equal"
           value: "value"
           effect: "NoSchedule"
     

   * You can also specify the tolerations for each user-deployed component and it will override the global setting.
    ```yaml
      longhornManager:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
      longhornDriver:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
      longhornUI:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
   

   • If you install Longhorn by using kubectl to apply the deployment YAML, you need to modify the taint tolerations section for Longhorn Manager, Longhorn UI, and Longhorn Driver Deployer. Then apply the YAMl files.
   • If you install Longhorn using Helm, you can change the Helm values for global.tolerations, longhornManager.tolerations, longhornUI.tolerations, longhornDriver.tolerations in the values.yaml file before installing the chart.

2. Set taint tolerations for system-managed components (for example, Instance Manager, CSI Driver, and Engine images)

   Follow the Customize default settings to set taint tolerations by changing the value for the taint-toleratio default setting

   Note: Because of the limitation of Rancher 2.5.x, if you are using Rancher UI to install Longhorn, you need to click Edit As Yaml and add setting taintToleration to defaultSettings.

   For example:

   defaultSettings:
     taintToleration: "key=value:NoSchedule"
   

Setting up Taints and Tolerations After Longhorn has been installed

Warning:

Before modifying the toleration settings, users should make sure all Longhorn volumes are detached.

Since all Longhorn components will be restarted, the Longhorn system is unavailable temporarily. If there are running Longhorn volumes in the system, this means the Longhorn system cannot restart its components and the request will be rejected.

Don’t operate the Longhorn system while toleration settings are updated and Longhorn components are being restarted.

1. Prepare

   Stop all workloads and detach all Longhorn volumes. Make sure all Longhorn volumes are detached.

2. Set taint tolerations for user-deployed components (for example, Longhorn Manager, Longhorn Driver, and Longhorn UI).

   • If you install Longhorn through Rancher, you must copy and paste the following parameters into the YAML on the Rancher UI (click Edit as YAML during the upgrade) to apply the value to all user-deployed components.

       global:
         tolerations:
         - key: "key"
           operator: "Equal"
           value: "value"
           effect: "NoSchedule"
     

   * You can also specify the tolerations for each user-deployed component and it will override the global setting.
    ```yaml
      longhornManager:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
      longhornDriver:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
      longhornUI:
        tolerations:
        - key: "key"
          operator: "Equal"
          value: "value"
          effect: "NoSchedule"
   

   • If you install Longhorn by using kubectl to apply the deployment YAML, you need to modify the taint tolerations section for Longhorn Manager, Longhorn UI, and Longhorn Driver Deployer. Then reapply the YAMl files.
   • If you install Longhorn using Helm, you can change the Helm values for global.tolerations, longhornManager.tolerations, longhornUI.tolerations, longhornDriver.tolerations in the values.yaml file, and then run helm upgrade to upgrade to the new version of the chart.

3. Set taint tolerations for system-managed components (for example, Instance Manager, Backing Image Manager, Share Manager, CSI Driver, and Engine Image).

   The taint toleration setting can be found at Longhorn UI under Setting > General > Kubernetes Taint Toleration.

History

Available since v0.6.0

• Original feature request
• Resolve the problem with GitOps