BosImport tool is used to migrate other data stored on the cloud to BOS. The tool is implemented based on C-S structure, and the client end is responsible for reading the file list, task scheduling and statistics on progress, and all parameters/configuration related to migration are implemented on the client end; the migration task is completed on the server end, and the client end notifies the server through RPC request which files are migrated.

The BosImport tool has the following characteristics:

• Support for a variety of migration modes, with good scalability, and support for the migration of different source ends and destination ends, and support AWS S3, Aliyun OOS, Tencent Cloud COS, Qiqiu Cloud, Mi Eco Cloud, BOS, ect;
• Support for real-time migration concurrency/pressure adjustment, "hot upgrade" of server;
• Support for hot start and stop of migration task, and real-time display of migration progress;
• Do not miss any disk during the data transfer, reduce the disk IO, split the large file into parts for transmission during the transmission, configure the block size in real time and utilize the bandwidth resources of the migration machine to maximum efficiency;
• Load balancing, client selects the server without pressure for migration tasks;
• Support the customized migration task list and file number;
• Support QPS of migration tasks and rate-limiting of traffic;
• Support multiple migration policies: 1. FORCE_OVERWRITE: mandatory overwriting of the files at the destination end may destroy destination-end files, which is dangerous and shall be used with caution and used on the premise of determining the requirements; 2. KEEP_DESTITNATION: skip if a file with the same name on the destination end exists; 3. KEEP_NEWER: if a file with the same name exists at the destination end, it is newer than that at the source end, skip the file; otherwise override it; if the file with the "same name" does not exist at the destination end (the file name may be changed during the migration), copy it to the destination end directly according to the migration rules

Download Link

Operating environment

Java 1.7 and above

Category Structure

├── client 
│   ├── client.jar   # The client jar is responsible for accessing the file list, task scheduling and progress statistics, etc. 
│   ├──resources    # Client configuration folder 
│   │   ├── client.properties   # Client migration task configuration file 
│   │   └── client_logback.xml   # Client log configuration file 
│   ├── start_client.sh   # Start the client progress 
│   └── stop_client.sh   # Stop the client progress 
├── properties_examples   # Sample configuration for different sources of migration tasks client.properties 
│     ├── client.properties.bos   # Migrate from BOS to BOS 
│     ├── client.properties.bos-check   # Verify from BOS to BOS 
│     ├── client.properties.cos   # Migrate from COS to BOS 
       ├── client.properties.fds   # Migrate from Mi Eco Cloud to BOS 
│     ├── client.properties.http   # Migrate from HTTP to BOS 
│     ├── client.properties.oss   # Migrate from OOS to BOS 
│     ├── client.properties.qiniu   # Migrate from Qiqiu Cloud to BOS 
│     ├── └── client.properties.s3   # Migrate from AWS S3 to BOS 
├── server 
│     ├── resources   # Server configuration folder 
│     │   ├── server.properties   # Server configuration 
│     ├── └── server_logback.xml   # Server log configuration file 
│     ├── server.jar   # The server.jar is responsible for accessing the data source and migrating to BOS 
      ├── supervise.sh   # Supervisor script maintains to restart the server progress automatically if it exits 
│     ├── start_server.sh   # Start server progress 
│     └── stop_server.sh   # Stop server progress 
├── start_migration.sh   # Start the migration task with one key, including server and client 
└── stop_migration.sh   # Stop the migration task with one key, including server and client