This paper introduces the best practices for migrating the data of MongoDB database replica set instance or self-built MongoDB database replica set instance to MongoDB database replica set instance on the cloud by using Baidu smart cloud data transmission service (hereinafter referred to as DTS).

Preconditions

Description:

At present, DTS data migration service only supports MongoDB replica set instance. For shard set instance, data migration can be completed by using mongoshake tool.

• An instance of MongoDB database replica set has been created as the migration source, version 3.4 or 3.6
• An instance of MongoDB database replica set has been created as the migration target, version 3.4 or 3.6
• To migrate the database replica set instances from source to target, users need to provide accounts with root role for data migration.
• At present, the data migration from MongoDB to MongoDB only supports the migration at instance level

Migration type

• All data migration：Migrate all the data in the source database to the target database, if there is new data in the source database during the process of migration, and only full migration is selected, the new data will not be migrated to the target database.。
• Incremental synchronous migration：Synchronize the newly added data in the migration process of the source database to the target database, mainly used for real-time synchronization of incremental data between the source database and the target database.。

If both full migration and incremental synchronization are selected when creating the data migration task, the stock data in the source database will be migrated to the target database first, and then the incremental data will be synchronized to the target database. The incremental synchronization migration will not end automatically. If required, user can manually suspend or end the migration task in the management console as needed.

Preparation

Create an instance of cloud database DocDB for MongoDB replica set in Baidu smart cloud service to be used in the target location of database migration, the creation steps refer to the Create Replica Set Instance.

Operation procedures

1. Create DTS migration task

Click the Data Transmission Task tab on the left navigation bar of DTS management console, then click the button Create Data Transmission Instance

If your source is a cloud database DocDB for MongoDB instance, then select Baidu Smart Cloud Database as the source location; if it is a public network /BBC/BCC/DCC self-built MongoDB instance, then select Self-built Data Storage as the source location.

If your target is a cloud database DocDB for MongoDB instance, then select Baidu Smart Cloud Database as the target location; if it is a public network /BBC/BCC/DCC self-built MongoDB instance, then select Self-built Data Storage as the source location.

Select the cross-region or same region according to your actual situation.

Then click Next to complete the purchase configuration. On the data transmission task list page of DTS management console, a DTS task in unconfigured status will be added at the top of the list page, that is the migration task you just created. Click Configure Task to configure the task.If no new tasks appear on the list page jumped to, it is recommended to wait for a while and refresh the page.

2. Configure DTS migration task

Click Configure Task to enter the task connection configuration page. In the following instance, the configuration takes the source as Self-built Data Storage and the target as Baidu Smart Cloud Database.

If the configuration source is a self-built data storage, you only need to select your corresponding self-built instance at the source. In this instance, select the access type cloud server BCC, and the data type MongoDB, fill in corresponding instance account and password, then click the button Test Connectivity to test the connectivity of the source instance.

If the target is Baidu smart cloud database, the data type of the target will be matched automatically according to the data type of the source. Select the cloud database DocDB for MongoDB instance to be migrated to, enter corresponding account password, and click the button Test Connectivity to test the connectivity of the target.

Click the Authorize Whitelist to Enter into Next Step, authorization whitelist to enter the next step, and you can choose the migration type of the source instance as needed, including full migration, incremental migration or both. At present, the data migration from MongoDB to MongoDB only supports the migration at instance level.

After selecting the migration type, you can click Save and Pre-check button to save the configuration and pre-check, or click Savebutton to save the configuration.

3. Start the migration

After saving the DTS migration task configuration, you can start the pre-check on the task list page. If you click the Save and Pre-checkbutton when saving the configuration task, there is no need to perform pre-check again.

If the pre-check fails, you can check relevant configuration according to the error prompt.

After successful pre-check, click Start to start the migration task.

After the task is started, the task status will change to Running，When the data migration is completed, the task status will change to Task End, then you can release the migration task.

If both Full Migration and Incremental Synchronization are selected when creating the data migration task, the stock data in the source database will be migrated to the target database first, and then the incremental data will be synchronized to the target database. The incremental synchronization migration will not end automatically. If you need to end real-time synchronization, you can do the following in the management console as you need:

• Click the button Pause to pause the migration task, and the task status changes to Paused, then you can continue the migration task later.
• Click Stop to end the migration task, and the task status will change to Ended, after that, you can release the migration task.

After the data migration task is finished, you can connect to the cloud database instance on the target and view the migrated data information.