Migration from other clouds

Cross-cloud migration of stock data

For customers transitioning from other cloud platforms to BOS, Baidu AI Cloud Object Storage (BOS) offers three migration methods: the CloudFlow data flow platform, the BOS import tool, and the online data import feature available in the console.

• Data Flow Platform CloudFlow (recommended) is a fully managed migration platform for storage data launched by Baidu AI Cloud Storage. It supports cross-cloud migration and synchronization for both existing and incremental data;
• BOS Import Tool: It is a reliable, secure, and stable scripted migration tool. It needs to be downloaded and installed locally before use, and is suitable for data migration of 1 TB or more;
• BOS mirror back-to-origin: You can configure mirror back-to-origin for the bucket. When you access an object in the bucket that does not exist in BOS, BOS will immediately request the object from the pre-configured back-to-origin address to ensure normal access. During the request process, the data returned by the origin server is automatically delivered to the requesting user without their awareness. Meanwhile, this data will also be automatically stored in BOS, improving the smoothness of such requests in the future;

Method 1: CloudFlow data flow platform (recommended)

Data Flow Platform CloudFlow is a fully managed migration platform for storage data launched by Baidu AI Cloud Storage. It realizes data migration from third-party clouds to Baidu AI Cloud through visual configuration. It supports migration of third-party object storage data to Baidu AI Cloud object storage, and allows viewing the progress and results of migration tasks. For failed migration tasks, it supports re-initiating migration retries, and also supports operations such as interrupting and deleting migration tasks.

The data flow platform supports the following functions:

• Task creation: Before starting a data migration process, you need to create a corresponding migration task first;
• Supported sources: The data flow platform supports migration from Alibaba Cloud OSS, Tencent Cloud COS, Kingsoft Cloud KS3, Qiniu Cloud Kodo, Huawei Cloud OBS, AWS China region, and local self-built S3 clusters to Baidu AI Cloud Object Storage (BOS);
• Task management: After the migration task is created, you can manage the task, such as interrupting, deleting, downloading failed files, restarting, etc.;
• Task progress viewing: After the migration task is created, you can view the migration progress of the task, such as the number of files that have been migrated, the number of successfully migrated files in progress, the number of failed files, the number of files to be migrated, etc.;
• Support migration of stock data and incremental migration;

You can log in to the Baidu AI Cloud BOS console, click Data Migration in the left navigation to create a migration task, or directly log in to the CloudFlow Console to create a migration task and initiate data migration. For detailed migration steps, please refer to CloudFlow Help Document.

Method 2: BOS import tool

The BOS import tool, developed by Baidu AI Cloud Object Storage (BOS), facilitates data migration from other cloud storages to BOS. It offers features such as "high scalability" and "support for on-the-fly starting and stopping of migration tasks." The tool is based on a client-server (C-S) architecture. The client manages functions such as reading file lists, scheduling tasks, and tracking progress. All migration-related parameters/configurations are set on the client side. Migration tasks are executed on the server side, and the client communicates with the server via RPC requests to indicate which files to migrate.

For specific usage methods and precautions, please refer to: BOS Import Tool

Incremental cross-cloud data synchronization

You can achieve cross-cloud synchronization of incremental data by setting up mirror back-to-origin rules or utilizing the CloudFlow data flow platform.

Method 1: CloudFlow data flow platform

Data Flow Platform CloudFlow is a fully managed migration platform for storage data launched by Baidu AI Cloud Storage. It realizes data migration from third-party clouds to Baidu AI Cloud through visual configuration. It supports cross-cloud migration and synchronization for both existing and incremental data.

Method 2: Mirror back-to-origin

The mirror back-to-origin feature is a bucket-level configuration designed to help users perform hot data migration without service interruption and address incremental data migration needs from other clouds to BOS.

Once mirror back-to-origin is configured for a bucket, if a user attempts to access (GetObject) an object in BOS that does not exist, BOS will fetch the object from the back-to-origin address and store the retrieved data in BOS while delivering it to the requesting user.

For specific usage methods and precautions, please refer to: BOS mirror back-to-origin