Overview

When users are ready to use Baidu AI Cloud BOS, they often need to migrate data from other cloud storage to Baidu AI Cloud BOS. If the amount of data migrated is large, the traditional manual migration method of "Download and Upload" is often time-consuming and laborious, and it is difficult to meet the business requirements of rapid migration of users.

Therefore, BOS provides online data import function for the vast number of users. Users only need to configure the import rules, and the system will automatically grab data from the source directory and store it to the designated path on BOS, thus greatly improving the migration efficiency of business data.

Create Tasks

1.Log in the management console and select "Product Services > object Storage BOS" to enter the global overview page of BOS.

2.Click the create task button in the "Data Online Import" card on the right to enter the create task page.

Note: In order to ensure the performance of task transmission, at present, the system only supports up to two tasks with "Running" status.

3.Fill in the task name to start the task creation.

4.Select a transfer target and a created bucket as the storage location for online data import.

Note:

Data migrated to BOS will be transferred to the bucket selected in "Select Bucket" by default.

If there is a file with the same name in the target bucket, it will be overwritten directly

5.Select the transmission source and click the "Select File" button to upload the locally prepared URL list file to BOS.

Note: For the format requirements of this URL list file, please refer to [URL list file format requirements](#URL list file format requirements) .

6.Select the configuration execution time by clicking "Limited Operation Period" to select a reasonable task execution time.

Note:

1.The source directory must be in the allowed reading status to ensure that the object can be obtained by BOS.

2.The source object size is limited to 0-10G.

3.Please be aware that, when the transmission service is running, the third-party source path may incur a read fee; when BOS writes to the target bucket, it may also incur a normal request fee.

4.The online import task will occupy a certain amount of bandwidth resources, so users need to arrange the task execution time reasonably to avoid the impact on other services when the bandwidth is full.

URL List File Format Requirements

The URL list file contains the three parts of information as follows:

1.The location of the object in the source directory will be defined by URL. 2.Path and object name of the object after it is transferred to the specified bucket of BOS. 3.The storage types of objects uploaded to BOS include STANDARD, STANDARD-IA, COLD, and ARCHIVE, which correspond to standard storage, infrequency storage, cold storage and archive storage types respectively. If it is not filled in, the default is the standard storage type.

BOS supports users to define the above three parts of information in a txt file. Each row represents an object, the first column represents the storage path of the object transferred to BOS, the second column represents the address in the source directory, and the third column represents the storage type in BOS, as shown in the following figure:

Taking the object represented by the first row as an example, its storage path in the source directory is http://abc001.com/A/1.jpg; after online import, it exists in the path cjbucket1/A/1.Jpg under the specified bucket in BOS. The storage type of its file in BOS is STANDARD.

Note: Three columns of information should be separated by TAB.

Transmission List

The created tasks will be displayed in the transmission list. The list shows the name, target bucket, current task status, task creation time and task progress information of each task.

Users can suspend running tasks, delete or resume stopped tasks, and delete completed tasks.

Users can view more detailed task information in the drop-down status of the taskbar for each task.

For completed tasks, BOS will automatically generate a task log to display the transmission of each object during task execution in detail, so that users can locate the failed objects.

It is required to just click "Click Download" in the figure below, and the task log will also be uploaded to the root directory of the task transmission target bucket by default.