GPU Exclusive and Shared Usage Instructions

If the cluster supports GPU computing power and memory sharing and isolation, you can configure whether to exclusively occupy or share GPU resources through console settings or the YAML file submitted during the creation of a task or workload.

Prerequisites

• The CCE GPU Manager and CCE AI Job Scheduler components have been successfully installed. You can find them in Cluster - Component Management - Cloud-Native AI.
• Memory sharing has been enabled for the node. You can configure this in Cluster - Node Management - Worker - Memory Sharing Configuration.

Resource names corresponding to GPU card models

To ensure proper use of GPU resources, ensure the specified resource name matches the GPU card model. For instance, if this document uses the number of GPU cards as an example, add “_core” (for computing power) or “_memory” (for memory) after the resource name to specify computing or memory resources.

Resource description

Example of creating a task via console

If you create a task through the CCE console (for operation steps, refer to Cloud-Native AI Task Management), you can specify the GPU type as “Exclusive” or “Shared” in the task’s pod group configuration.

Exclusive occupation example(single-GPU/multi-GPU)

• When specifying the GPU type as exclusive GPU, to use the full resources of a GPU card for the task:

1. Choose the GPU model.
2. Enter the number of GPU cards. The number must be a positive integer, ranging from [1 to the maximum number of GPU cards per node in the current cluster].

• To use only CPU and memory resources (no GPU resources):

Leave the GPU Model field blank and only input the required CPU and memory resources.

Single-GPU sharing example [isolation for memory only, no isolation for computing power]

When specifying the GPU type as shared GPU, to isolate only memory (no computing power isolation), follow steps below:

1. Choose the GPU model.
2. Do not fill in the computing power value.
3. Enter the required GPU memory. The memory size must be a positive integer, ranging from [1 to the memory size of the selected GPU card].

Single-GPU sharing example [isolation for both memory and computing power]

When specifying the GPU type as shared GPU, to isolate memory and computing power, follow steps below:

1. Choose the GPU model.
2. Enter the required computing power percentage. The percentage must be a positive integer between [5 and 100].
3. Enter the required GPU memory. The memory size must be a positive integer, ranging from [1 to the memory size of the selected GPU card].

Example of creating a workload via console

If you create a workload through the CCE console (for operation steps, refer to Workloads), you can specify the GPU type as “Exclusive” or “Shared” in the workload’s container configuration. The resource input restrictions for exclusive and shared modes are the same as those for creating AI tasks.

Example of creating a task/workload via YAML

If you create a task or workload via YAML (for detailed configurations, refer to Cloud-Native AI Task Management and Workloads), you can specify the required GPU card resources as “Exclusive” or “Shared” in the YAML configuration. Specific examples are as follows:

Note:
1. When creating a task via YAML, the scheduler must be specified as: schedulerName: volcano
2. GPU memory resources must be applied for. Additionally, only one of baidu.com/v100_32g_cgpu_memory and baidu.com/v100_32g_cgpu_memory_percent can be filled in; they cannot be filled in simultaneously

Exclusive occupation example (single-GPU)

1resources:
2  requests:
3 baidu.com/v100_32g_cgpu: 1 // 1 GPU card
4    cpu: "4"
5    memory: 60Gi
6  limits:
7 baidu.com/v100_32g_cgpu: 1 // Limit and request must be the same
8    cpu: "4"
9    memory: 60Gi

Exclusive occupation example (multi-GPU)

1resources:
2  requests:
3 baidu.com/v100_32g_cgpu: 2 // 2 GPU card
4    cpu: "4"
5    memory: 60Gi
6  limits:
7 baidu.com/v100_32g_cgpu: 2 // Limit and request must be the same
8    cpu: "4"
9    memory: 60Gi

Single-GPU sharing example [isolation for memory only, no isolation for computing power]:

1resources:
2  requests:
3    baidu.com/v100_32g_cgpu: 1
4    baidu.com/v100_32g_cgpu_memory: 10 // 10GB
5    cpu: "4"
6    memory: 60Gi
7  limits:
8    baidu.com/v100_32g_cgpu: 1
9    baidu.com/v100_32g_cgpu_memory: 10
10    cpu: "4"
11    memory: 60Gi

Single-GPU sharing example [isolation for both memory and computing power]:

1resources:
2  requests:
3    baidu.com/v100_32g_cgpu: 1
4 baidu.com/v100_32g_cgpu_core: 50 // 50% (i.e., 0.5 of a card’s computing power)
5    baidu.com/v100_32g_cgpu_memory: 10 // 10GB
6    cpu: "4"
7    memory: 60Gi
8  limits:
9    baidu.com/v100_32g_cgpu: 1
10    baidu.com/v100_32g_cgpu_core: 50 //
11    baidu.com/v100_32g_cgpu_memory: 10
12    cpu: "4"
13    memory: 60Gi

Example of using encoding/decoding function (only supported by performance-optimized components):

1resources:
2  requests:
3    baidu.com/v100_32g_cgpu: 1
4 baidu.com/v100_32g_cgpu_core: “20” // 20% (i.e., 0.2 of a card’s computing power)
5    baidu.com/v100_32g_cgpu_memory: "2" // 2
6 baidu.com/v100_32g_cgpu_encode: 100 // Enable encoding function
7 baidu.com/v100_32g_cgpu_decode: 100 // Enable decoding function
8    memory: "2Gi"
9    cpu: "1"
10  limits:
11    baidu.com/v100_32g_cgpu: 1
12    baidu.com/v100_32g_cgpu_core: "20"
13    baidu.com/v100_32g_cgpu_memory: "2"
14    baidu.com/v100_32g_cgpu_encode: 100
15    baidu.com/v100_32g_cgpu_decode: 100