Use BOS

Last Updated：2022-01-14

Preparation

The user must make the following preparations before mounting the BOS instance in the container.

[Register Baidu Account](UserGuide/Register Baidu Account.md#Register Baidu Account) and complete the Identity Verification.
Go to the BOS page to create a bucket.
Create a usable container cluster.

Create Bucket

To create a BOS Bucket, refer to Create Bucket for the operation steps.

Note: The created bos bucket and mounting point must be in the same vpc/subnet as the cluster node.

Create Container Cluster

To create a container cluster, refer to Create Cluster for the operation steps.
Download the command-line client kubectl and connect to the cluster. Refer to Connect to Kubernetes Cluster via kubectl).

Note: K8S cluster version>=1.16

Operation Guide

Deploy Storage Plugin

Select in turn: Cloud Container Engine CCE -> Helm Template -> Baidu AI Cloud Template

Search for template by the template name cce-csi-bos-plugin.

Click Install and enter the corresponding parameters.

* Instance name: the plug-in instance name, e.g.: bos;
* Deploy the cluster: select the cluster where you need to deploy the BOS CSI plug-in;
* Namespace: The namespace that manages the instance's helm metadata, for example: kube-system;
* kubernetets version: enter the version of the cluster deployed actually. Currently, the versions 1.18, 1.16, and 1.13 are available;
* nodes: If you specify the data directory of kubelet when deploying the nodes of the cluster, you need to fill in the specific data directory to this list, otherwise keep the default;
* region: You need to modify the actual region of the cluster, e.g.: bj (Beijing), gz (Guangzhou), su (Suzhou), hkg (Hong Kong), bd (Baoding), and fwh (Wuhan).

Mount BOS in Static PV/PVC Mode

Create an AK/SK secret in the cluster to access the BOS storage.

kubectl create secret generic csi-bos-secret \
  --from-literal=ak=<Your AK> \
  --from-literal=sk=<Your SK>

For more information about AK/SK, see How to Get AK and SK.

Create PV and PVC resources in cluster

Use kubectl, and execute kubectl create -f bos-pv.yaml to complete the creation of PV.

The corresponding bos-pv.yaml file is as follows:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-bos
  namespace: "default"
spec:
  accessModes:
  - ReadWriteOnce
  - ReadOnlyMany
  capacity:
    storage: 5Gi
  storageClassName: csi-bos
  csi:
    driver: "csi-bosplugin"
    volumeHandle: "v-XXXXXX"
    nodePublishSecretRef:
      name: "csi-bos-secret"
      namespace: "default"
  mountOptions:
  - "-o meta_expires=0"
  persistentVolumeReclaimPolicy: Retain

Considerations and Parameter Descriptions:

VolumeHandle in yaml: corresponds to the bucketName of BOS, supports mounting BOS bucket subdirectories, such as bucketName/dirName

nodePublishSecretRef: fill in the secret name in step 1

mountOptions: Since the BOS bucket is mounted on Bosfs, you can specify the startup parameters of Bosfs through the “mountOptions”. For details of the supported parameters, see the BOS parameter specification.

BOS supports one write and multiple reads, but the read-only pod cannot read the latest written data: the corresponding accessMode only supports ReadWriteOnce + ReadOnlyMany

After creating the PV, enter kubectl get pv to see a PV in the available state, as shown below:

$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE
bos-pv    5Gi        RWO,ROX        Retain           Available             csi-bos                         3s

Create a PVC that can be bound to the PV.

Use kubectl and execute kubectl create -f bos-pvc.yaml to complete the creation of PVC.

The corresponding bos-pvc.yaml file is as follows:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: bos-pvc
spec:
  accessModes:
  - ReadWriteOnce
  - ReadOnlyMany
  resources:
    requests:
      storage: 5Gi
  storageClassName: csi-bos

Note: The storageClassName field in yaml is used to associate with PV. It is recommended to enter it. If the PV of a multi-class storage system is used in the cluster, the PVC is pending before binding.

$ kubectl get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
bos-pvc   Pending                                       csi-bos        2s

After binding, the status of PV and PVC becomes Bound.

$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM             STORAGECLASS   REASON    AGE
bos-pv    5Gi        RWX            Retain           Bound     default/bos-pvc                            36s
$ kubectl get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
bos-pvc   Bound     bos-pv    5Gi        RWO,ROX        csi-bos        1m

For more settings and field descriptions of PV and PVC, see k8s Official Documentation.

Mount the PVC in the Pod

Specify the corresponding PVC name in the Pod spec, use kubectl to execute kubectl create -f demo-bos-pod.yaml to complete the creation of the pod.

The corresponding demo-bos-pod.yaml file is as follows:

apiVersion: v1
kind: Pod
metadata:
  name: nginx01
  namespace: default
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nginx01
    volumeMounts:
    - mountPath: /var/lib/www/html
      name: bos-pvc
    - mountPath: /var/lib/www/html000
      name: bos-pvc
      readOnly: true
    livenessProbe:
      exec:
        command:
        - ls
        - /var/lib/www/html
  volumes:
  - name: bos-pvc
    persistentVolumeClaim:
      claimName: bos-pvc
      readOnly: false

After the Pod is created, you can read and write the path of /var/lib/www/html in the container to access the content on the corresponding BOS storage. At the same time, the path supports read-write and /var/lib/www/html000 read-only.

Note: Because CSI bosplugin relies on bosfs, configuring livenessProbe in Pod can avoid the problem that the container cannot perceive the mount point failure after bosfs restarts abnormally.

At the same time, it supports mounting read-only disks on other machines and kubectl create -f demo-bos-pod1.yaml creating a pod containing a read-only bos bucket.

apiVersion: v1
kind: Pod
metadata:
  name: nginx01-bbaa
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nginx01
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/lib/www/html000
      name: bos-pvc
      readOnly: true
    livenessProbe:
      exec:
        command:
        - ls
        - /var/lib/www/html
  volumes:
  - name: bos-pvc
    persistentVolumeClaim:
      claimName: bos-pvc
      readOnly: true

Release PV and PVC resources

After the use of the storage resources, the PVC and PV resources can be released.

Use the following command to release the PVC.

$ kubectl delete -f  bos-pvc.yaml

After the release of PVC, the status of the PV originally bound to it changes to Release, as shown below:

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM             STORAGECLASS   REASON    AGE
bos-pv    5Gi        RWO,ROX        Retain           Released   default/bos-pvc   csi-bos        16m

Enter the following command to release PV resources.

$ kubectl delete -f  bos-pv.yaml

Use Baidu Object Storage (BOS)

Monitoring Alarm

Cloud Container Engine