Migration Type

• Structure migration: The structure objects define the syntax migration, including the syntax migration of structure objects such as table, view, etc. The mapping of data type can be realized during the migration of the heterogeneous database, and the syntax of the source database can be adjusted to the syntax corresponding to the target database.
• Full data migration: Migrates all data in the source database to the target database, excluding syntax defined by structure objects. If there is new data in the source database during migration, and only full migration is selected as the migration type, the added data cannot be migrated to the target database.
• Incremental data migration: Synchronizes the data added during the migration of the source database to the target database, mainly used for real-time synchronization of incremental data of the source database and the target database. If both full data migration and incremental data migration are selected when creating the data migration task, static data in the source database is migrated to the target database first. Then incremental data is synchronized to the target database. The incremental data migration cannot end automatically. If you need to end the real-time synchronization, you need to end the migration task at the console manually.

Migration Restrictions

• The primary key must exist at the source. Otherwise, duplicate data may appear at the target.
• Currently, the migration of storage process, storage function, trigger, and foreign key is not supported, and the user needs to migrate manually.
• The source binlog_format must be row.
• If the source is MySQL 5.6 and above,The source binlog_row_image must be full.

Migration Privilege Requirements

When the source is a self-built database of public network or the BCC self-built database, the user needs to provide a migration account meeting the privilege requirements.

Users can authorize the existing account for migration or create a new account for migration. Please see the following authorization statement: GRANT SELECT, LOCK TABLES, REPLICATION SLAVE, REPLICATION CLIENT, SHOW VIEW ON ?.? TO 'Migrated Account' @ 'CVM Server Name' IDENTIFIED BY 'Migration Password'; Please fill in the "Migration Account" and the "Migration Password" as required; fill in the "Hostname" with reference to DTS Public Network IP Address Range, to release access restriction only to DTS server IP, or fill in % open access to all IPs.

After authorization, continue to refresh the system privilege table, and execute FLUSH PRIVILEGES;; ensure that the account privilege takes effect immediately.

Create a Data Migration Task

The data migration task consists of 3 steps: Source database and target database configuration > migration type and database table filtering > pre-check.

1.Log in to Baidu AI Cloud Console.

2.Select "Product Service > Data Transmission Service DTS" to enter the data migration list.

3.Click "Create Migration Task" and fill in relevant information of the data migration task.

Note:

1. The source database types supported by DTS include Instance on RDS, BCC self-built database, and public network self-built database.
2. The target database types supported by DTS include: Instance on RDS, and BCC self-built database.

4.After adding the task, enter the Migration Type and Database Table tab, select the migration type and the source database object, and add the source database object to be migrated to the right through "Add Selected".

Baidu AI Cloud supports three types of data migration: Structure migration, full data migration, and incremental data migration. For more information, please see Migration Types.

During the migration process, the user can filter the table data according to row filtering or column filtering. When filtering conditions are not filled in, no filtering is performed by default.

• Column filtering: When it is only required to obtain partial column data and changes, fill in the column name of data required, using ',' for separation, and the syntax is the same with the database standard SQL select condition, such as id, name, and data.
• Row filtering: When it is only required to obtain partial row data and changes, fill in the row filtering conditions, and the syntax is the same with the database standard SQL where condition, such as id>10 and status=' success'.

5.Click "Save and Pre-check" to complete the creation of the task, and then view the task status in the task list.

• The status column displays "Pre-check Passed". You can select and start the migration task. After the task is started, you can view the migration progress in the task progress column.
• The status column displays "Pre-check Failed". Click the button next to view the reason for failure and modify it. Restart the check and start the migration task until successful.

6.After the task is started, you can view the information on each task through the task details page, including configuration information, operation details, and traffic information.

Pre-check

After creating the DTS data migration task, you need to check the connectivity of the source database and the target database. In addition, the detailed content check is performed before starting data migration. Please see the following interpretation for detailed check items.

• The source database connectivity pre-check is interpreted as follows:

Ping test	Telnet test	Mysql connect test	Interpret
Successful	Successful	Successful	Connection successful
Successful	Successful	Failed	The source database connection failed. Please check whether the database service has enabled the corresponding port and whether the authorization is added. Fr Baidu's exit ip is not a fixed one. You need to authorize 10.% of IP address range or authorize % directly and check the firewall setting
Successful	Failed	Failed	The source database port connection failed. Please check whether the database service is enabled, or whether the corresponding port is enabled, check the firewall setting, and whether to block the port access
Failed	Failed	Failed	There is no network connection between the source ip and the DTS server. Please check whether the ip accessible by the external network is filled in or check the firewall setting

• The target database connectivity pre-check is interpreted as follows:

Ping test	Telnet test	Mysql connect test	Prompt
Successful	Successful	Successful	Connection successful
Successful	Successful	Failed	The target database connection failed. Please check whether the database service has enabled the corresponding port and whether the authorization is added. Fr Baidu's exit ip is not a fixed one, you need to authorize 10.% of IP address range or authorize % directly, and check the firewall setting
Successful	Failed	Failed	The target database port connection failed. Please check whether the database service is enabled, or whether the corresponding port is enabled, check the firewall setting, and whether to block the port access
Failed	Failed	Failed	There is no network connection between the target ip and the DTS server. Please check whether the ip accessible by the external network is filled in or check the firewall setting

• Pre-check items, failure causes, and troubleshooting:

Pre-check items	Failure causes	Solutions
Check whether the data transmission server can connect to the source database.	The network is blocked, a firewall is set, and mysql does not authorize all ips.	Check whether the filled ip is accessible on the public network, check the firewall configuration, and check mysql account privileges
Check whether the data transmission server can connect to the target database.	The target database network is not connected, mysql authorization problem	Check whether the target data is accessed normally, and check mysql account privileges
Check the version number of the source database.	The mysql version is lower than 5.1.	Upgrade mysql to version 5.1 and above
Check whether the database to be migrated off the target database is available.	The db in the target database does not exist.	db:create database dest_db used to create the target database
Check whether the target database is read-only.	The target database is read-only and not writable.	Set the target database writable: set global read_only=0;
Check whether the account privileges of the source database meet the migration requirements.	The account privileges of the source database do not meet the migration requirements.	Add the privileges of the source database account, see the Privilege Requirementsfor more information
Check whether the account privileges of the target database meet the migration requirements.	The account privileges of the target database do not meet the migration requirements.	Add the privileges of the target database account, see the Privilege Requirements for more information
Check whether the number of connections of the source database and the target database meets the requirements.	The number of remaining connections cannot meet the dts requirements.	Increase the maximum number of database connections: Set global max_connections=1000
Check whether there is a structure object with the same name as the object to be migrated in the target database.	There's a table with the same name as the table to be migrated in the target database.	Delete the table in the target database or select incremental migration
Check whether there are structure objects with the same name in different databases at the source.	There are identical structure objects in different databases at source, e.g., identical table name.	Divide the same structure objects into different dts tasks
Check whether the foreign key parent table on which the migration table depends is migrated.	The table on which the foreign key depends is not migrated.	When selecting database table information, the dependent tables are migrated together
Check whether the table structure of the migrated table exists in the target database.	There's no table structure to migrate in the target database.	Create a table to migrate in the target database, or add structure migration for the migration object
Check the migrated table for unsupported storage engines.	There are unsupported storage engines in the migrated table.	Remove the tables not supporting migration
Check whether SQL_MODE in the source database and the target database is valid.	The sql_mode of the source database and the target database is invalid.	Change to valid sql_mode, and see the supported sql_mode for more information
Check whether the table in the target database is null.	The table in the target database is not null.	Clear the table data of the target database