Deployment methods
Updated at:2025-11-03
BosImport mainly consists of two components: server and client.
- For small-scale data migrations below 30TB, client and server can run on the same machine. Modify the client.properties file and run
sh start_migration.shdirectly. - For large-scale data migrations or to enhance migration speed, multiple servers can be deployed across different machines. The client sets up these servers through serverList and selects a suitable server for migration based on workload to optimize efficiency.
Notes
- A single machine can run multiple server processes, but attention should be paid to the space occupied by the
server/logsfolder; - Adjust the number of server threads according to the machine's CPU core count. It is recommended to set maxConnectionsPerServer to 1; otherwise, errors may occur more frequently.
Deploy server
- Modify the server configuration file
server.properties, and mainly change the listener port No.; - If concurrent migration by multiple servers is required, you can batch-copy the server folder to multiple machines used for migration.
- Execute
sh start_server.shto start the server. Check whether the server process listens to the specified port to determine whether the server has been started successfully.
Deploy client
Modify the client.properties file to configure the client. Configuration items mainly include the following:
- Migration type:
type, migration name:migrationName - Number of concurrent server responses to single requests:
serverThreadNum - Number of single request files
filesPerRequest - Task list:
taskList - File list:
listDelimiter, specified representation format of the file list:listFormat - AK, SK, endpoint, etc. at source location and destination location
- Migration strategy:
migrationStrategy - Part size
blockSize, default value: 5MB. Because the number of parts in BOS multipart upload ranges from 1 to 10000, a maximum single object for upload is 50GB for blockSize=5MB. For large file migration, this parameter should be increased, with the number of parts within 1,000 being optimal.
Start migration
For single server and client deployment, execute the sh start_migration.sh script to start the server and client. The system migrates data based on the file list configured in the client.properties file. If the server is deployed on multiple machines, execute sh start_server.sh on each machine to start the server process.
Stop migration
Execute the sh stop_migration.sh script to stop the server and client. If the server is deployed on multiple machines, execute sh stop_server.sh on each machine to stop the server process.
