BosImport tool mainly includes two parts: server and client.

• For small-scale data migrations less than 30TB, client and server can run on the same machine, modify the client.properties file, and then directly start the migration with 'sh start_migrations.sh'.
• For large-scale data migration or to improve the migration speed, multiple servers can be deployed to multiple machines. The client can configure multiple servers through the serverList and select a suitable server for migration according to the load, thus improving the migration efficiency.

Note:

1. A single machine can open multiple server processes, but you need to be aware of the size occupied by the server/logsfolder.
2. The thread number of server process needs to be adjusted to the appropriate number according to the number of CPU cores on the machine, and maxConnectionsPerServer is recommended to set as 1; otherwise, it is easier to fail.

Deploy Server

1. Modify the configuration file server.properties of server, which is mainly to modify the listening port number;
2. If multiple servers are required for concurrent migration, you can copy the server folder to multiple machines for migration in batch;
3. Execute sh start_server.sh to start the server, and determine whether the server is successfully started by checking whether the server process listens to the specified port.

Deploy Client

Modify the file client.properties to configure the client, and the main configuration items include:

1. Migration types: type, migration name: migrationName
2. Number of concurrency used in response to a single request by server: serverThreadNum`
3. Number of files per single request: filesPerRequest
4. Task list: taskList
5. File list: listDelimiter specifies the form of expression for file list: listFormat
6. The information on ak, sk and endpoint, etc. at the source and destination end
7. Migration policies: migrationStrategy 8.Block size: The blockSize defaults to be 5MB, as the number of parts ranges from 1 to 1000 during BOS multi-part uploading; when blockSize=5MB, the single object that can be uploaded is 50GB at maximum, and such parameter needs to be increased when the large file is migrated, and it is better that such number of parts is within 1000.

Start Migration

When the server and client are deployed in a stand-alone manner, execute the script sh start_migration.sh to start the server and client, and the system migrates the data according to the file list configured in the file client.properties; if the server is deployed on multiple machines, the script sh start_server.sh is executed on multiple machines to start the server progress.

Stop Migration

Execute the script to stop the server and client. If the server is deployed on multiple machines, the script sh stop_server.sh is executed on multiple machines to stop the server progress.