CCE AI Job Scheduler Description

Component introduction

The task scheduling component allows for organizing and managing diverse AI tasks. When used with the CCE Deep Learning Frameworks Operator, it enables direct training of deep learning models on CCE.

Component function

• Support a variety of scheduling strategies and advanced job management capabilities.
• Scheduling strategies include two types: "spread" and "binpack." "Binpack" prioritizes centralized GPU card sharing and usage by multiple Pods, ideal for improving GPU resource utilization. "Spread" aims to distribute multiple Pods across different GPU cards for scenarios requiring high GPU availability.
• Preemption modes support intra-queue priority preemption and inter-queue oversell/preemption. Intra-queue priority preemption: In the same queue, high-priority tasks can preempt resources of low-priority tasks to ensure the operation of high-priority tasks; inter-queue oversell/preemption: When queue A is fully utilized and queue B has idle resources, new tasks submitted to queue A will be scheduled to queue B. If queue B later receives new tasks and lacks resources, the oversell tasks will be killed to ensure the running of queue B tasks.
  For the use of preemption functions, refer to the relevant descriptions in Queue Management and Task Management.

Application scenarios

Run deep learning tasks directly on CCE clusters to boost AI engineering efficiency.

Limitations

• Supports only Kubernetes clusters of version 1.18 or higher.

Install component

1. Sign in to the Baidu AI Cloud official website and enter the management console.
2. Go to Product Services - Cloud Native - Cloud Container Engine (CCE) to access the CCE management console.
3. Click Cluster Management - Cluster List in the left navigation bar.
4. Click on the target cluster name in the Cluster List page to navigate to the cluster management page.
5. On the Cluster Management page, click Component Management.
6. From the component management list, choose the CCE AI Job Scheduler Component and click Install.
7. On the Component Configuration page, finish setting up the deep learning framework.

• Scheduling strategies include two options: spread and binpack. Binpack prioritizes centralized use of the same GPU card for multiple Pods, ideal for enhancing GPU resource usage. Spread aims to distribute multiple Pods across different GPU cards, ensuring high availability of GPU resources.
• Preemption modes include intra-queue priority preemption and inter-queue oversell/preemption. For intra-queue priority preemption: Within the same queue, high-priority tasks can seize resources from lower-priority tasks to maintain their operations. For inter-queue oversell/preemption: If queue A is fully utilized and queue B has idle resources, tasks submitted to queue A are scheduled to queue B. In case queue B later experiences a resource shortage due to new tasks, oversell tasks will be terminated to maintain the operation of tasks in queue B.

8. Click the OK button to finalize the component installation.

Version records