Baidu AI Cloud
中国站

百度智能云

Object Storage

Initialization

Getting Started

  1. Initialize BosClient.

    BosClient is a client interacting with BOS service, and all BOS operations of BOS Python SDK are completed via BosClient. You can refer to [BosClient](#create BosClient). Complete the operation of initializing client.

  2. Create bucket.

    Bucket is a namespace on BOS. It is equivalent to a container of data and can store several data entities (objects). You can complete the operation of creating a bucket by reference to Create Bucket

  3. Upload object.

    BOS is the essential data unit in object, and you can understand object as a file simply. Users can upload object by reference to Upload Object

  4. List all objects in bucket.

    After completing a series of uploads, you can refer to View Object List in Bucket to view all objects under specified bucket.

  5. Get the specified object.

    You can obtain one or more objects by reference to Obtain Object

Confirm Endpoint

Before confirming the Endpoint you configured when using the SDK, read the section of the Developer Guide on BOS Access Domain Name to understand Endpoint-related concepts. Baidu AI Cloud currently has opened access to multi-region support, please refer to Region Selection Description.

Multiple regions are supported currently. The corresponding information is as follows:

Region Access to EndPoint Support protocol
Beijing bj.bcebos.com HTTP,HTTPS
Baoding bd.bcebos.com HTTP,HTTPS
Suzhou su.bcebos.com HTTP,HTTPS
Guangzhou gz.bcebos.com HTTP,HTTPS
Hong Kong hkg.bcebos.com HTTP,HTTPS
Financial cloud Wuhan special region fwh.bcebos.com HTTP,HTTPS
Financial cloud Shanghai special region fsh.bcebos.com HTTP,HTTPS

Get the Key

To use Baidu AI Cloud BOS, you need to have a valid AK (Access Key ID) and SK (Secret Access Key) for signature authentication. AK/SK is assigned to users by the system and is a string to identify users and verify signatures for accessing BOS. You can obtain and understand your AK/SK information through the following steps:

Register Baidu AI Cloud Account

Create AK/SK

Create BOSClient

BosClient is the client of BOS service, providing methods for developers to interact with BOS service.

Access to BOS via AK/SK

BosClient is a Python client of BOS service, which provides a series of methods for callers to interact with BOS service.

1.Before creation of BosClient, first create a configuration file to configure BosClient, and in the following, configuration file is named as bos_sample_conf.py, with specific configuration information as follows:

#!/usr/bin/env python
#coding=utf-8

#Import Python standard log module
import logging

#Import BOS configuration management module and security
from baidubce.bce_client_configuration import BceClientConfiguration
from baidubce.auth.bce_credentials import BceCredentials

#et Host, Access Key ID and Secret Access Key of BosClient
bos_host = "bj.bcebos.com"
access_key_id = "AK"
secret_access_key = "SK"

#Set handle and log level of log file
logger = logging.getLogger('baidubce.http.bce_http_client')
fh = logging.FileHandler("sample.log")
fh.setLevel(logging.DEBUG)

#Set output sequence, structure and contents of log file
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(fh)

#Create BceClientConfiguration
config = BceClientConfiguration(credentials=BceCredentials(access_key_id, secret_access_key), endpoint = bos_host)  

Note: For the log file, Logging has the following levels: DEBUG,INFO , WARNING, ERROR, CRITICAL.

In the above code, ACCESS_KEY_ID and SECRET_ACCESS_KEY correspond to the "Access Key ID" and the "Access Key Secret" in the console, respectively. For more information on how to access it, please see the < Operation Guide Manage ACCESSKEY>.

For the method above, you need to specify service domain name of BOS by themselves, which can be specified by assigning a value to bos_host variable. If not configured, it is not needed to introduce endpoint parameter, and the default is Beijing region http://bos.bj.bcebos.com

2.After completing the configuration above, create a BosClient by reference to the following codes.

#Import BosClient configuration file 
import bos_sample_conf 
		
#Import BOS related interface 
from baidubce import exception 
from baidubce.services import bos 
from baidubce.services.bos import canned_acl 
from baidubce.services.bos.bos_client import BosClient 
	
#Create BOSClient. 
bos_client = BosClient(bos_sample_conf.config) 

Access to BOS via sts

BOS can implement temporary authorized access by a third party through the STS mechanism. STS (Security Token Service) is a temporary authorized service provided by Baidu AI Cloud; for more information, please see Introduction to Use of Baidu AI Cloud STS. With STS, you can issue a third-party user with a custom time-based and privileged access credential. The third-party user can use the access credential to directly call API or SDK of Baidu AI Cloud to access its resources.

To access to BOS via sts, users apply for a set of AK, SK and token via sts-client first, and then configure that set of parameters to BosClient, and users can create a BosClient by reference to the following codes:

1.Obtain a temporary token, and create a configuration rule "sts_sample_conf.py" to configure BosClient.

import logging
from baidubce.bce_client_configuration import BceClientConfiguration
from baidubce.auth.bce_credentials import BceCredentials

sts_host = "STS_HOST"
access_key_id = "AK"
secret_access_key = "SK"

logger = logging.getLogger('baidubce.services.sts.stsclient')
fh = logging.FileHandler("sample.log")
fh.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(fh)

config = BceClientConfiguration(credentials=BceCredentials(access_key_id, secret_access_key), endpoint=sts_host)

Note: Currently, when STS is used to configure client, wherever the endpoint of the corresponding BOS service is, endpoint needs to be configured as http://sts.bj.baidubce.com.

In the codes above, access_key_id corresponds to "Access Key ID"in the console, and secret_access_key corresponds to "Access Key Secret" in the console.

2.Create StsClient, and configure BceClientConfiguration with the temporary token obtained. Create file "sts_sample.py", and apply for a set of AK, S and token via StsClient.

import sts_sample_conf
from baidubce.services.sts.sts_client import StsClient
from baidubce.bce_client_configuration import BceClientConfiguration
from baidubce.auth.bce_credentials import BceCredentials

sts_client = StsClient(sts_sample_conf.config)

duration_seconds = 3600
# you can specify limited permissions with ACL
access_dict = {}
access_dict["service"] = "bce:bos"
access_dict["region"] = "bj"   
access_dict["effect"] = "Allow"
resource = ["*"]
access_dict["resource"] = resource
permission = ["READ"]
access_dict["permission"] = permission

access_control_list = {"accessControlList": [access_dict]}
# Create StsClient
response = sts_client.get_session_token(acl=access_control_list, duration_seconds=duration_seconds)

sts_ak = str(response.access_key_id)
sts_sk = str(response.secret_access_key)
token = response.session_token
bos_host = "BOS_HOST"
#Configure BceClientConfiguration
config = BceClientConfiguration(credentials=BceCredentials(sts_ak, sts_sk), endpoint = bos_host, security_token=token)

3.After completing the configuration above, create a BosClient by reference to the following codes.

import sts_sample 
		
#Import BOS related interface 
from baidubce import exception
from baidubce.services import bos
from baidubce.services.bos import canned_acl
from baidubce.services.bos.bos_client import BosClient
	
#Create a BOSClient
bos_client = BosClient(sts_sample.config)

Configure HTTPS to Access BOS

You can access to BOS service with HTTPS in BOS Python SDK in the following 2 ways:

  • Specify HTTP in endpoint.

    config = bce_client_configuration.BceClientConfiguration(
        credentials = bce_credentials.BceCredentials(
            access_key_id = 'your-ak',
      	    secret_access_key = 'your-sk'
        ),
        endpoint = 'https://bj.bcebos.com'
    )
    client = bos_client.BosClient(config)
  • Set HTTPS protocol by specifying https in protocol

     config = bce_client_configuration.BceClientConfiguration(
         credentials = bce_credentials.BceCredentials(
             access_key_id = 'your-ak',
     	    secret_access_key = 'your-sk'
         ),
         endpoint = 'bj.bcebos.com',
         protocol = baidubce.protocol.HTTPS
     )
     client = bos_client.BosClient(config)

Note: If you specify scheme of endpoint while specifying protocol parameter, endpoint should prevail.

config = bce_client_configuration.BceClientConfiguration(
    credentials = bce_credentials.BceCredentials(
        access_key_id = 'your-ak',
	    secret_access_key = 'your-sk'
    ),
    endpoint = 'http://bj.bcebos.com',
    protocol = baidubce.protocol.HTTPS
)
client = bos_client.BosClient(config) 

Configure custom domain name/alternate domain name to access BOS

Use custom domain name

If you want to use a custom domain name as an endpoint to access BOS, after binding the custom domain name with a bucket in Bos on the console, configure endpoint as a custom domain name and open cname_enabled switch,example cdn-test.cdn.bcebos.com,The configuration code is as follows:

config = bce_client_configuration.BceClientConfiguration(
    credentials = bce_credentials.BceCredentials(
        access_key_id = 'your-ak',
	    secret_access_key = 'your-sk'
    ),
    endpoint = 'http://cdn-test.cdn.bcebos.com',
    cname_enabled = True
)
client = bos_client.BosClient(config) 

Use alternate domain name

On the basis of using official domain name and user-defined domain name as endpoint to access BOS, the SDK further supports the mechanism of using endpoint primary domain name to access BOS again when it fails to access BOS. The usage method is as follows:

config = bce_client_configuration.BceClientConfiguration(
    credentials = bce_credentials.BceCredentials(
        access_key_id = 'your-ak',
	    secret_access_key = 'your-sk'
    ),
    endpoint = 'http://cdn-test.cdn.bcebos.com',
    cname_enabled = True,
    # If using endpoint to access BOS fails, the SDK will automatically retry the backup_endpoint domain name
    backup_endpoint = 'cdn-test.bj.bcebos.com'
)
client = bos_client.BosClient(config)

Configure BosClient

Set Network Parameters

You can set some network parameters:

#Set request timeout period 
bos_sample_conf.config.connection_timeout_in_mills = TIMEOUT 
	    
#Set size of receiving buffer region 
bos_sample_conf.config.recv_buf_size(BUF_SIZE) 
	
#Set size of sending buffer region 
bos_sample_conf.config.send_buf_size(BUF_SIZE) 
	
#Set connection retry strategy 
#Cubic exponential retreat retry 
bos_sample_conf.config.retry_policy = BackOffRetryPolicy() 
#No retry 
bos_sample_conf.config.retry_policy = NoRetryPolicy() 

Parameter description

All parameters that can be specified via bos_client_configuration are as shown in the table below:

Parameter Description
port BOS port number
send_buf_size Send buffer size
recv_buf_size Receive buffer size
connection_timeout_in_mills Request timeout (unit: millisecond)
retry_policy Connect retry strategy, which is cubic exponential retreat by default when initializing Client
Previous
SDK Installation
Next
Bucket Management