Download file
The BOS Python SDK provides a variety of file download APIs, allowing users to download files from BOS in the following ways:
- Simple streaming download
- Download to a local file
- Download as a string
- Resumable download
- Range download
- Obtain download progress
Simple object read
Users can read the object into a stream using the following code:
1response = bos_client.get_object(bucket_name, object_key)
2s = response.data
3# Process object
4...
5# Close stream
6response.data.close()
Directly download an object to a file or string
Users can download an object to the specified file using the following code:
1bos_client.get_object_to_file(bucket_name, object_key, file_name)
Users can refer to the following code to download an object to a string:
1result = bos_client.get_object_as_string(bucket_name, object_key)
2print(result)
Range download
To implement more functions, you can specify the download range by specifying range parameter to achieve more refined acquisition of the object. If the specified download range is 0-100, it will return data from byte 0 to byte 100, including byte 100, with a total of 101 bytes of data, i.e., [0, 1000].
1range = [0,1000]
2 #Return object data in the specified range
3print(bos_client.get_object_as_string(bucket_name, object_key, range = range))
4 #Return object data in the specified range to a file
5bos_client.get_object_to_file(bucket_name, object_key, file_name, range = range)
The range parameter of get_object_as_string and get_object_to_file allows you to specify the range of the returned object. This feature enables users to perform segmented downloads and resumable uploads of files seamlessly.
Other methods
Get storage class of the object
The storage class attributes of an object are divided into STANDARD (standard storage), STANDARD_IA (infrequent access storage), COLD (cold storage), and ARCHIVE (archive storage). The storage class attribute of an object can be obtained through the following code:
1response = bos_client.get_object_meta_data(bucket_name, object_key)
2print(response.metadata.bce_storage_class)
Obtain only ObjectMetadata
The get_object_meta_data method can be used to obtain only the metadata of the object, not the object entity itself. As shown in the following code:
1response = bos_client.get_object_meta_data(bucket_name, object_key)
The parameters available for calling in the get_object_meta_data parsing class are:
| Parameters | Description |
|---|---|
| content_length | Object size |
| e_tag | The HTTP protocol entity tag of object |
| bce_meta | If user_metadata custom meta is specified in PutObject, this item will be returned () |
| storageClass | Storage class of the object |
| bce_restore | Returned when an archive storage object is being restored or has been restored. For archived objects being restored, the value of bce_restore is ongoing-request="true"; for archived objects that have been retrieved, the value of bce_restore is ongoing-request="false", expiry-date="Wed, 07 Nov 2019 00:00:00 GMT". Where, expiry-date indicates the expiration time of the restored object, specified in Greenwich Mean Time (GMT). |
Obtain download progress
The Python SDK provides real-time download progress updates during downloads. Currently, it supports downloading objects to files. To enable this, simply add the progress_callback parameter to the API and supply a progress bar callback function, or utilize the default progress bar callback function provided in the tool class.
The example of the callback function is as follows:
1def percentage(consumed_bytes, total_bytes):
2 """Use progress bar callback function to calculate the current completion percentage
3
4 :param consumed_bytes: The amount of data uploaded/downloaded
5 :param total_bytes: Total amount of data
6 """
7 if total_bytes:
8 rate = int(100 * (float(consumed_bytes) / float(total_bytes)))
9 print('\r{0}% '.format(rate))
10 sys.stdout.flush()
11# progress_callback is an optional parameter used to implement the progress bar function.
12bos_client.get_object_to_file(bucket_name, key, download, progress_callback=percentage)
It is recommended to use the default progress bar callback function (utils.default_progress_callback) in the tool class, which currently supports percentage and progress bar display. The example is as follows:
1# Import the SDK tool class package
2from baidubce import utils
3# progress_callback is an optional parameter used to implement the progress bar function.
4bos_client.get_object_to_file(bucket_name, key, download, progress_callback=utils.default_progress_callback)
- Code example of get_object_to_file
1# Import the SDK tool class package
2from baidubce import utils
3# progress_callback is an optional parameter used to implement the progress bar function.
4bos_client.get_object_to_file(bucket_name, key, download, progress_callback=utils.default_progress_callback)
Support single-link rate limit
Baidu AI Cloud Object Storage (BOS) provides a public network bandwidth limit of 10 Gbit/s per bucket and an intranet bandwidth limit of 50 Gbit/s per bucket. When the upload or download usage reaches the bandwidth limit, the error code RequestRateLimitExceeded will be returned. To ensure normal service usage, BOS supports traffic control during uploads and downloads to prevent high traffic services from affecting other applications.
Example of download class request API
The value range of the rate limit value is 819200~838860800 in bit/s, that is, 100KB/s~100MB/s. The rate limit value must be a number. BOS will limit the rate of this request according to the specified rate limit value. When the rate limit value is not within this range or is illegal, it will return the error code 400.
1traffic_limit_speed = 819200 * 5
2# get obj url by traffic limit
3param = {}
4param[b'x-bce-traffic-limit'] = traffic_limit_speed
5url = bos_client.generate_pre_signed_url(bucket_name, key, timestamp=1649923427,
6 expiration_in_seconds=1000,
7 params=param)
8__logger.debug("[Sample] get object url is %s", url)
9# get object into file
10bos_client.get_object_to_file(bucket_name, key, download, traffic_limit=traffic_limit_speed)
11__logger.debug("[Sample] get object into file, file size:%s", os.path.getsize(download))
