Configure BOSCMD

General configuration

Configuration supports Chinese display

If you need to configure a CMD environment that supports Chinese display, execute the following command:

$ export LANG=en_US.UTF-8

Note: You can first run the locale command to check the language support status of the current environment. If it is not LANG=en_US.UTF-8, you can execute the above command to modify it.

Configure AK/SK, Region and Host information

Before using the BOS CMD tool, it is recommended to configure the Access Key, Secure Key, Region, and Host. You can set AK, SK, Region, and Host information using the -c/--configure option.

1$ ./bcecmd -c  [<conf-path>]
2
3$ Your Access Key ID [None]: Enter Your AK
4
5$ Your Security Token []: Enter Your STS token
6
7$ Your Secret Access Key ID [None]: Enter Your SK
8
9$ Default Region Name [bj]: Enter Your region
10
11$ Default Domain [bj.bcebos.com]: Enter Your host
12
13$ Default Proxy Host [none]: Enter Your proxy host
14
15$ Default breakpoint_file_expiration [7] days:
16
17$ Default use path style [yes]:

Description:

• <conf-path> is a local file directory. When a user has multiple Baidu AI Cloud accounts, specifying <conf-path> allows configuration files for different accounts to be written to different directories. When executing commands, you can specify the directory to select the configuration of the corresponding account.
• If <conf-path> is not specified, after configuration is completed, the CMD will automatically write the configuration file to the current user’s home directory ~/.go-bcecli; if <conf-path> is specified, the configuration file will be automatically written to the specified directory.
• Even without configuring AK/SK, you can still access public buckets via BOS CMD.
• Once the Region is specified, you can omit the Host, as BOS CMD will automatically append Region.bcebos.com as the Host.
• If you manually specify a Host, BOS CMD will use it directly without automatic concatenation.
• You can enter none to set AK/SK/Region, and domain to null. When Region is left empty, the default value is bj. When Domain is set to null, it will use the Region to concatenate by default; if both are null, the default access address is bj.bcebos.com.
• After specifying a proxy host, the CMD client will access the BOS servicer via the proxy.
• When using temporary STS tokens, the token can be set. If a token is set, it takes precedence.
• breakpoint_file_expiration: The validity period of breakpoint files. After expiration, previous breakpoints become invalid.
• use path style specifies whether to use path style when sending BOS requests. The path style is used by default, where the host in the request is a third-level domain name, and the URL path includes the bucket name.

BOS CMD-related configurations

In addition to general settings, the bcecmd -c command also supports enabling automatic domain name switching and setting BOS upload-related attributes, such as whether to use the HTTPS protocol and the number of concurrent multipart uploads, etc.

Set automatic domain name switching

BOS operates across multiple regions. For cross-region operations, the BOS CMD’s automatic domain name switching feature seamlessly switches domain names based on the bucket's region. This feature is enabled by default for ease of use.

If you want all your operation requests to be sent to a specified domain name or Host (domain configured via bcecmd -c), you can disable the "automatic domain name switching" function using the bcecmd -c command. Set to no to disable this function.

1$ ./bcecmd -c [<conf-path>]
2
3$ Default use auto switch domain [yes]:

Note: The value can be yes or no. Setting it to yes enables automatic domain name switching based on the bucket's region, while setting it to no disables it. This feature is enabled by default.

Set the HTTPS protocol for uploads

To ensure the security of user data, it is recommended to use the [HTTPS protocol for uploads](BOS/Typical Practices/HTTPS Transmission Encryption Practice.md). Whether to use HTTPS can be set via the bcecmd -c command. Set it to no to use the HTTP protocol for uploads.

1$ ./bcecmd -c [<conf-path>]
2
3$ Default use https protocol [no]:

Note: The value can be yes or no. By default, uploads use the HTTP protocol.

Set the number of concurrent multipart upload of large files.

When uploading files larger than 32MB using BOS CMD, BOS CMD will automatically perform multipart upload. You can set the number of concurrent single-file multipart upload via the bcecmd -c command.

1$ ./bcecmd -c [<conf-path>]
2
3$ Default multi upload thread num [10]:

Note: The number of concurrent multipart uploads must be an integer of 1 or greater. The default value is 10.

Set the number of files for concurrent upload and download using the BOS CMD sync command.

BOS CMD performs concurrent multi-file operations during upload and download using the sync command. You can set the number of the concurrent sync command using the bcecmd -c command.

1$ ./bcecmd -c [<conf-path>]
2
3$ Default sync processing num [10]:

Note: The concurrency count must be set to an integer that is 1 or higher. By default, it is set to 10.

Note: When the sync command is used, multiple files are processed concurrently. If a file exceeds the multipart upload threshold, it will also be split into multiple parts for concurrent uploads. Consequently, in the worst-case scenario, the total number of coroutines running simultaneously will equal "number of concurrent syncs x number of concurrent single-file multipart uploads." However, increasing the number of concurrent operations does not always lead to better upload efficiency. Therefore, you should configure the number of concurrent syncs and single-file multipart uploads based on your computer's specifications.

Set the size of a single part for multipart upload.

The default single par size for BCE CMD is 10MB. If your network environment is stable (no timeout occurs during the uploading of files with the default part size), the default part size is recommended. When a timeout occurs during upload (when BOS returns the error code RequestTimeout), it is recommended that the size of single part should be reduced to avoid the upload timeout. You can configure the part size using the following commands:

1$ ./bcecmd -c [<conf-path>]
2
3$ Default multi upload part size [10] MB (Must be positive integer and equal or greater than 1) :

Description:

• The part size must be set to an integer no smaller than 1, with a default of 10 (unit: MB). The number of parts for BOS multipart uploads can range from 1 to 10,000. With the default value, the maximum size of a single object for upload is 100 GB. When uploading large files, this parameter should be adjusted, ideally keeping the number of parts to within 1,000.
• Dynamic adjustment: If the part size is set too small (e.g., 1MB) and the file size exceeds 10GB (but is still below the BOS-supported maximum file size of 48.8TB), the number of parts will surpass 10,000. In such cases, the BCE CMD will automatically adjust the part size to reduce the number of parts, ensuring normal file uploads.
• If your network environment is unfavorable, it is recommended that reduce the [number of concurrently uploaded and downloaded files for the sync command] (#Set the number of concurrent upload and downloaded files for the BOS CMD sync command) and the [number of concurrent large-file multipart uploads](#Set the number of concurrent multipart upload of large files.).