CCE GPU Manager Description

Component introduction

A bundle of GPU device plugins, paired with a compatible scheduler, delivers GPU resource scheduling capabilities in complex scenarios. The CCE GPU Manager component supports isolation-optimized mode, facilitating shared use and isolation of computing power and memory.

Component function

• Topology allocation: Enables a GPU topology allocation function. When more than one GPU card is assigned to a Pod, the system automatically chooses the fastest topology connection mode to allocate GPU devices.
• GPU sharing: Provides the option to enable memory sharing for GPU devices on a node and supports allocating GPU cards to multiple Pods based on memory size.
• Memory and computing power isolation: Ensures isolation of memory and computing power when multiple Pods share a single GPU card.
• Fine-grained scheduling: When enabled, you can select specific GPU models during the creation of queues and tasks. When disabled, only quota input is allowed while creating queues and containers, and specific GPU models cannot be selected.
• Encoding/decoding instances: Submit encoding/decoding tasks using the independent encoding/decoding units of GPUs for hardware-based encoding/decoding.
• For detailed component usage instructions, please refer to: GPU Exclusive and Shared Instructions

Application scenarios

Running GPU applications in CCE clusters addresses the issue of resource waste caused by exclusively using entire GPU cards in scenarios like AI training, thereby improving resource utilization and reducing costs.

Limitations

• Only supports Kubernetes clusters of version v1.18 and above.
• Currently, this component depends on the CCE AI Job Scheduler. If required, please install both components together; otherwise, the functions of this component may be unavailable.
• GPU-shared virtualization supports the following mainstream GPU CUDA and Driver versions. The isolation-optimized mode imposes additional requirements on OS kernel versions and others. For adaptation to other versions, please submit a request. The current support details are as follows.

Install component

1. Sign in to the Baidu AI Cloud Official Website and enter the management console.
2. Go to Product Services - Cloud Native - Cloud Container Engine (CCE) to access the CCE management console.
3. Click Cluster Management > Cluster List in the left navigation bar.
4. Click on the target cluster name in the Cluster List page to navigate to the cluster management page.
5. On the Cluster Management page, click O&M & Management > Component Management.
6. Select the CCE GPU Manager component from the component management list and click on Install.
7. In the installation confirmation dialog box, isolation-optimized mode is selected by default.
8. The default unit for GPU memory sharing is GiB.
9. Fine-grained scheduling is enabled by default.
10. Click the OK button to finalize the component installation.

Version records