Upload files

Updated at：2025-11-03

In BOS, the basic data unit for user operations is an object. An object consists of a key, metadata, and data. The key is the object’s name, the metadata provides a user-defined description as a set of name-value pairs, and the data is the content of the object.

The BOS Java SDK provides a rich set of file upload APIs, and files can be uploaded in the following ways:

Simple upload
Append upload
Multipart upload
Resumable upload

Simple upload

In simple upload scenarios, BOS supports uploading objects in the form of specified files, data streams, binary strings, and character strings. Please refer to the following code or Simple Upload Demo.

                Java
                
            

                public void PutObject(BosClient client, String bucketName, String objectKey, byte[] byte1, String string1){
 // Get specified file
    File file = new File("/path/to/file.zip");
 // Obtain data stream
    InputStream inputStream = new FileInputStream("/path/to/test.zip");
	
 // Upload object as a file
    PutObjectResponse putObjectFromFileResponse = client.putObject(bucketName, objectKey, file);
 // Upload an object in the form of a data stream
    PutObjectResponse putObjectResponseFromInputStream = client.putObject(bucketName, objectKey, inputStream);
 // Upload object as binary string
    PutObjectResponse putObjectResponseFromByte = client.putObject(bucketName, objectKey, byte1);
 // Upload object in string form
    PutObjectResponse putObjectResponseFromString = client.putObject(bucketName, objectKey, string1);
 // Create an empty directory. The objectKey must end with a forward slash, such as test/
 // The BOS console will, by default, display objects ending with a forward slash (/) as file directories
    PutObjectResponse putObjectResponseFromString = client.putObject(bucketName, objectKey, "");
	
 // Print ETag
    System.out.println(putObjectFromFileResponse.getETag());
}
            

Files are uploaded to BOS as objects. The PutObject function supports uploading objects with a size of up to 5 GB. Upon successful processing of a PutObject request, BOS returns the object's ETag in the response header as its unique identifier.

Set file meta information

Object metadata refers to the attributes of files provided by users when uploading to BOS. It is mainly divided into two categories: standard HTTP attribute settings (HTTP headers) and user-defined metadata.

Set Http header of object

The BOS Java SDK essentially interacts with the backend HTTP API, allowing users to customize the HTTP headers of the object when uploading files. The following describes some commonly used HTTP headers:

Name	Description	Default value
Content-MD5	File data verification: After setting, BOS will enable file content MD5 verification, compare the MD5 you provide with the MD5 of the file, and throw an error if they are inconsistent	None
Content-Type	File MIME: This defines the file type and web page encoding, determining how the browser reads the file. If unspecified, BOS generates it based on the file's extension. If the file lacks an extension, a default value will be applied.	application/octet-stream
Content-Disposition	Indicate how the MIME user agent displays the attached file, whether to open or download it, and the file name	None
Content-Length	The length of the uploaded file. If it exceeds the length of the stream/file, it will be truncated; if it is insufficient, it will be the actual value	Stream/file duration
Expires	Cache expiration time	None
Cache-Control	Specify the caching behavior of the web page when the object is downloaded	None
x-bce-content-crc32	The CRC value (cyclic redundancy check code) of uploaded object	None

Reference code is as follows:

                Java
                
            

                //Initialize the upload input stream
ObjectMetadata meta = new ObjectMetadata();
 // Set ContentLength size
meta.setContentLength(1000);
 // Set ContentType
meta.setContentType("application/json");
 //Set cache-control
meta.setCacheControl("no-cache");
 // Set x-bce-content-crc32. The file’s CRC32 check code must be calculated by the user themselves. The format example is as follows
meta.setxBceCrc("0x74E947D0");
client.putObject(bucketName, objectKey, content, meta);
            

User-defined meta information

BOS supports user-defined metadata for describing objects. Example usage is shown in the following code:

                Java
                
                // Set the value of custom metadata name to my-data
meta.addUserMetadata("name", "my-data");
    
 // Upload Object
client.putObject(bucketName, objectKey, content, meta);

Prompt:

In the above code, the user has customized a metadata with the name "name" and value "my-data".

When users download this object, this metadata can also be obtained.

An object may have multiple similar parameters, but the total size of all user meta must not exceed 2KB.

Set storage class when uploading an object

BOS supports standard storage, infrequent access storage, and cold storage. Uploading an object and storing it as an infrequent access storage class is achieved by specifying the StorageClass. The parameters corresponding to the three storage classes are as follows:

Storage class	Parameters
Standard storage	STANDARD
Infrequent access storage	STANDARD_IA
Cold storage	COLD

Taking infrequent access storage as an example, the code is as follows:

                Java
                
            

                public void putObjectStorageClass(){
    PutObjectRequest request = new PutObjectRequest(bucketName, key, file);
    request.withStorageClass(BosClient.STORAGE_CLASS_STANDARD_IA);
    client.putObject(request);
}
            

Append upload

Objects created using the simple upload method described above are all of a standard type and do not support append writes. This limitation can be inconvenient in scenarios where frequent data overwriting occurs, such as log files, video surveillance, and live video streaming.

To address this, Baidu AI Cloud Object Storage (BOS) specifically supports the AppendObject method, which allows files to be uploaded in an append-write fashion. Objects created through the AppendObject operation are categorized as Appendable Objects, enabling data to be appended to them. The size limit for AppendObject files is 0–5 GB.

The sample code for uploading via the AppendObject method is as follows, or refer to Append Upload Demo.

                Java
                
            

                public void AppendObject(BosClient client, String bucketName, String objectKey, byte[] byte1, String string1) {
 // Get specified file
        File file = new File("/path/to/file.zip");
 // Obtain data stream
        InputStream inputStream = new FileInputStream("/path/to/test.zip");
 // Upload object as a file
        AppendObjectResponse appendObjectFromFileResponse = client.appendObject(bucketName, objectKey, file);
 // Upload an object in the form of a data stream
        AppendObjectResponse appendObjectResponseFromInputStream = client.appendObject(bucketName, objectKey, inputStream);
 // Upload object as binary string
        AppendObjectResponse appendObjectResponseFromByte = client.appendObject(bucketName, objectKey, byte1);
 // Upload object in string form
        AppendObjectResponse appendObjectResponseFromString = client.appendObject(bucketName, objectKey, string1);
 // Print ETag
        System.out.println(appendObjectFromFileResponse.getETag());
 //Print NextAppendOffset
        System.out.println(appendObjectFromFileResponse.getNextAppendOffset());
 // Print ContentMd5
        System.out.println(appendObjectFromFileResponse.getContentMd5());
 //Example of append upload, need to add the position of the next append write in the request
        Long nextAppendOffset = appendObjectFromFileResponse.getNextAppendOffset();
        AppendObjectRequest appendObjectFromFileRequest = new AppendObjectRequest(bucketName,objectKey,file);
        appendObjectFromFileRequest.setOffset(nextAppendOffset);
        AppendObjectResponse appendObjectFromFileResponse = client.appendObject(appendObjectFromFileRequest);
    }
            

Note:

For append upload, the position for appending must be specified; otherwise, the already written data will be overwritten.

Concurrent operations are not supported. If there is indeed a need for multi-client writing, you need to maintain the append position by yourself to avoid conflicts

Multipart upload

Besides uploading files to BOS using simple upload and append upload modes, BOS also supports another option—Multipart Upload. Users can use Multipart Upload mode in various scenarios, including but not limited to the ones below:

When resumable uploads are required.
When uploading files larger than 5GB.
When the connection to the BOS server is frequently interrupted due to unstable network conditions.
Enable streaming file uploads.
The file size cannot be determined before uploading.

The following will introduce the implementation of Multipart Upload step by step. Suppose there is a file with the local path /path/to/file.zip. Since the file is large, it will be transmitted to BOS in parts. Or refer to Multipart Upload Demo

Initialize multipart upload

Use initiateMultipartUpload method to initialize a multipart upload event:

                Java
                
            

                // Initiate Multipart Upload
InitiateMultipartUploadRequest initiateMultipartUploadRequest =
        new InitiateMultipartUploadRequest(bucketName, objectKey);
InitiateMultipartUploadResponse initiateMultipartUploadResponse =
        client.initiateMultipartUpload(initiateMultipartUploadRequest);
    
 // Print UploadId
System.out.println("UploadId: " + initiateMultipartUploadResponse.getUploadId());
            

The return result of initiateMultipartUpload contains UploadId, which is the unique identifier for distinguishing multipart upload events, and we will use it in subsequent operations.

Initialization for uploading infrequent access storage class objects

Initialize a multipart upload event for infrequent access storage:

                Java
                
            

                public void putMultiUploadStorageClass(){
    InitiateMultipartUploadRequest iniReq = new InitiateMultipartUploadRequest(bucketName, key);
    iniReq.withStorageClass(BosClient.STORAGE_CLASS_STANDARD_IA);
    client.initiateMultipartUpload(iniReq);
}
            

Initialization for uploading a cold storage class object

Initialize a multipart upload event for infrequent access storage:

                Java
                
            

                public void putMultiUploadStorageClass(){
    InitiateMultipartUploadRequest iniReq = new InitiateMultipartUploadRequest(bucketName, key);
    iniReq.withStorageClass(BosClient.STORAGE_CLASS_COLD);
    client.initiateMultipartUpload(iniReq);
}
            

Upload parts

The file is then uploaded in multiple parts.

                Java
                
            

                // Set each part to 5MB
final long partSize = 1024 * 1024 * 5L;
    
File partFile = new File("/path/to/file.zip");
    
 // Calculate the count of parts
int partCount = (int) (partFile.length() / partSize);
if (partFile.length() % partSize != 0){
    partCount++;
}
    
 // Create a list to save the ETag and PartNumber of each uploaded part
List<PartETag> partETags = new ArrayList<PartETag>();
    
for(int i = 0; i < partCount; i++){
 // Get file stream
    FileInputStream fis = new FileInputStream(partFile);
    
 //Jump to the beginning of each part
    long skipBytes = partSize * i;
    fis.skip(skipBytes);
    
 // Calculate the size of each part
    long size = partSize < partFile.length() - skipBytes ?
            partSize : partFile.length() - skipBytes;
    
 // Create UploadPartRequest to upload parts
    UploadPartRequest uploadPartRequest = new UploadPartRequest();
    uploadPartRequest.setBucketName(bucketName);
    uploadPartRequest.setKey(objectKey);
    uploadPartRequest.setUploadId(initiateMultipartUploadResponse.getUploadId());
    uploadPartRequest.setInputStream(fis);
    uploadPartRequest.setPartSize(size);
    uploadPartRequest.setPartNumber(i + 1);
    UploadPartResponse uploadPartResponse = client.uploadPart(uploadPartRequest);
    
 // Save the returned PartETag to the List.
    partETags.add(uploadPartResponse.getPartETag());
   
 // Close the file
    fis.close();
}
            

The core of the above code is to call the UploadPart method to upload each part, but the following points should be noted:

The UploadPart method requires that the size of each part, except the last one, must be greater than or equal to 100KB. However, the Upload Part interface does not immediately validate the Part size; validation occurs only during completing multipart upload.
To ensure no errors during network transmission, it is recommended to use the Content-MD5 value returned by BOS for each part after UploadPart to verify the correctness of the uploaded part data. When all part data is combined into one Object, it no longer contains the MD5 value.
The part number must be within the range of 1 to 10,000. If this limit is exceeded, BOS will return an InvalidArgument error code.
For each uploaded part, the stream must be positioned at the beginning of the respective part.
After each Part upload, the return result of BOS will include a PartETag object, which is a combination of the uploaded block's ETag and block number (PartNumber). It will be used in subsequent steps to complete the multipart upload, so it must be saved. Generally speaking, these PartETag objects will be saved in a List.

Complete multipart upload

Complete the multipart upload as shown in the following code:

                Java
                
                CompleteMultipartUploadRequest completeMultipartUploadRequest =
        new CompleteMultipartUploadRequest(bucketName, objectKey, initiateMultipartUploadResponse.getUploadId(), partETags);
    
 // Complete multipart upload
CompleteMultipartUploadResponse completeMultipartUploadResponse =
        client.completeMultipartUpload(completeMultipartUploadRequest);
    
 // Print Object's ETag
System.out.println(completeMultipartUploadResponse.getETag());

The partETags in the above code is the list of partETag saved in the second step. After BOS receives the part list submitted by the user, it will verify the validity of each data Part one by one. Once all data parts are validated, BOS will assemble the data parts into a complete Object.

Cancel multipart upload event

Users can cancel multipart uploads by using the abortMultipartUpload method.

                Java
                
                AbortMultipartUploadRequest abortMultipartUploadRequest =
        new AbortMultipartUploadRequest(bucketName, objectKey, uploadId);
    
 // Cancel multipart upload
client.abortMultipartUpload(abortMultipartUploadRequest);

Retrieve unfinished multipart upload event

Users can obtain the unfinished multipart upload events in the bucket by the listMultipartUploads method.

                Java
                
            

                ListMultipartUploadsRequest listMultipartUploadsRequest =
    new ListMultipartUploadsRequest(bucketName);
 // Retrieve all upload events within the bucket
ListMultipartUploadsResponse listing = client.listMultipartUploads(listMultipartUploadsRequest);
    
 // Traverse all upload events
for (MultipartUploadSummary multipartUpload : listing.getMultipartUploads()) {
    System.out.println("Key: " + multipartUpload.getKey() + " UploadId: " + multipartUpload.getUploadId());
}
            

Note:

By default, if the number of multipart upload events in a bucket surpasses 1,000, only 1,000 records will be returned. In such cases, the IsTruncated value in the response will be True, and the NextKeyMarker will indicate the starting point for the next query.

To retrieve more multipart upload events, you can utilize the KeyMarker parameter for batch reading.

Get all uploaded part information

Users can obtain all uploaded parts in an upload event by the listParts method.

                Java
                
                ListPartsRequest listPartsRequest = new ListPartsRequest(bucketName, objectKey, uploadId);
    
 // Retrieve all uploaded part information
ListPartsResponse partListing = client.listParts(listPartsRequest);
    
 // Traverse all parts
for (PartSummary part : partListing.getParts()) {
    System.out.println("PartNumber: " + part.getPartNumber() + " ETag: " + part.getETag());
}

If you need to view the storage class of an object, use the following code:

                Java
                
                public void listPartsStorageClass(){
    ListResponse listPartsResponse = client.listParts(bucketName, key, uploadId);
    String storageClass = listPartsResponse.getStorageClass();
}

Note:

By default, if the number of multipart upload events in a bucket surpasses 1,000, only 1,000 records will be returned. In such cases, the IsTruncated value in the response will be True, and the NextPartNumberMarker will indicate the starting point for the next query.

To fetch additional multipart upload events, you can utilize the PartNumberMarker parameter for batch reading.

Encapsulate multipart upload

In the Java SDK, Bos provides users with the putSuperObjectFromFile API, which encapsulates the three methods involved in multipart upload: initiateMultipartUpload, UploadPart and completeMultipartUpload. Users only need to call this API to complete the multipart upload.

                Java
                
                File file = new File("/path/to/file.zip");
PutSuperObjectRequest request = new PutSuperObjectRequest(bucketName, objectKey, file);
bosClient.putSuperObjectFromFile(request);

The parameters of PutSuperObjectRequest include:

Parameters	Description
chunkSize	Part size: The default is 5 MB, and it cannot be less than 100 KB
nThreads	Number of threads in the thread pool for multipart upload: The default is equal to the number of CPU cores
isSuperObjectUploadCanced	Cancel multipart upload or not
File	Upload files

If uploading a large file takes too long and the user wishes to end the multipart upload, they can call the cancel() method in PutSuperObjectRequest to set isSuperObjectUploadCanceled to true, thereby canceling the multipart upload operation.

Resumable upload

When users upload large files to BOS, if the network is unstable or the program crashes, the entire upload will fail, and the parts that have been uploaded before the failure will also be invalid. Users have to start over. This not only wastes resources but also often fails to complete the upload after multiple retries in an unstable network environment. Based on the above scenarios, BOS provides the capability of resumable upload:

In a generally stable network, it is recommended to use the three-step upload method, dividing the object into 5 MB blocks, refer to [Multipart Upload](#Multipart upload).
If your network condition is very poor, it is recommended to use the appendObject method for resumable upload, appending small data (256 KB) each time, refer to [Append Upload](#Append upload).

Tips

Resumable upload is an encapsulation and enhancement of multipart upload, implemented using multipart upload;

For large files or poor network environments, it is recommended to use multipart upload;

Fetch and upload

The following code retrieves resources from a specified URL and stores them in a designated bucket. This process requires the requester to have write permissions for the bucket. Only one object can be fetched at a time, and the user can specify a custom object name. For details, refer to the FetchObject API.

                Java
                
                String fetchurl = "bj.bcebos.com/sourbucket/sourceobject";
FetchObjectRequest req = new FetchObjectRequest(bucketname, objectname, fetchurl);
req.setReferer("www.referer.com");
bosClient.fetchObject(req);

The parameters of FetchObjectRequest include:

Parameters	Description
source	Source address of the fetched file
mode	Fetch mode: Supports two modes, asynchronous fetch (async) and synchronous fetch (sync), and the optional values are MODE_SYNC and MODE_ASYNC
storageClass	Storage class: Supports standard storage (STANDARD), infrequent access storage (STANDARD_IA), cold storage (COLD), and archive storage (ARCHIVE). The default storage class is STANDARD
callbackAddress	Fetch result callback address. If the asynchronous fetch mode is selected, results will be pushed here upon completion
referer	Referer header to be passed through during fetch at origin server
userAgent	User-Agent header to be passed through during fetch at origin server

Obtain upload progress

Supports providing real-time upload progress updates during the upload process. It currently supports four APIs: PutObject, AppendObject, UploadPart, and PutSuperObjectFromFile. The progress upload API accepts optional custom data.

The Java SDK defines upload progress callback APIs that allow users to define actions needed during upload, such as updating the API.

                Java
                
                public interface ProgressCallback<T> {
    void onProgress(long currentSize, long totalSize, T data);
}

PutObject example code:

                Java
                
            

                BosProgressCallback<Object> callback = new BosProgressCallback<Object>() {
    @Override
    public void onProgress(long currentSize, long totalSize, Object data) {
 // Print upload progress
        System.out.println("put " + currentSize + "/" + totalSize);
    }
};
 // If you need to use custom data in onProgress
callback.setData(obj);
PutObjectRequest request = new PutObjectRequest(bucketName, objectKey, file);
request.setProgressCallback(callback);
client.putObject(request);
            

AppendObject example code:

                Java
                
            

                BosProgressCallback<Object> callback = new BosProgressCallback<Object>() {
    @Override
    public void onProgress(long currentSize, long totalSize, Object data) {
        System.out.println("put " + currentSize + "/" + totalSize);
    }
};
 // If you need to use custom data in onProgress
callback.setData(obj);
AppendObjectRequest request = new AppendObjectRequest(bucketName, objectKey, file);
request.setProgressCallback(callback);
AppendObjectResponse response = client.appendObject(request);
Long nextAppendOffset = response.getNextAppendOffset();
            

UploadPart example code:

                Java
                
            

                BosProgressCallback<Object> callback = new BosProgressCallback<Object>() {
    @Override
    public void onProgress(long currentSize, long totalSize, Object data) {
        System.out.println("put " + currentSize + "/" + totalSize);
    }
};
 // If you need to use custom data in onProgress
callback.setData(obj);
UploadPartRequest request = new UploadPartRequest();
request.setBucketName(bucketName);
request.setKey(objectKey);
request.setUploadId(id);
request.setInputStream(inputStream);
request.setPartSize(size);
request.setPartNumber(i);
request.setProgressCallback(callback);
client.uploadPart(request);
            

PutSuperObjectFromFile example code:

                Java
                
            

                BosProgressCallback<Object> callback = new BosProgressCallback<Object>() {
    @Override
    public void onProgress(long currentSize, long totalSize, Object data) {
        System.out.println("put " + currentSize + "/" + totalSize);
    }
};
 // If you need to use custom data in onProgress
callback.setData(obj);
PutSuperObjectRequest request = new PutSuperObjectRequest(bucketName, objectKey, file);
request.setProgressCallback(callback);
client.putSuperObjectFromFile(request);
            

Single-link rate limit

Baidu AI Cloud Object Storage (BOS) applies bandwidth limits per bucket. When the user's upload or download bandwidth usage reaches the threshold, the error code RequestRateLimitExceeded will be triggered.

To ensure normal service usage, BOS implements traffic control during upload and download processes, ensuring that large traffic services do not impact other applications.

The upload API supports setting the specified rate limit value. The value range of the rate limit value is 819200~838860800 in bit/s, that is, 100KB/s~100MB/s. The rate limit value must be a number. BOS will limit the rate of this request according to the specified rate limit value. When the rate limit value is not within this range or is illegal, it will return the error code 400.

PutObject example code:

                Java
                
                PutObjectRequest request = new PutObjectRequest(bucketName, objectKey, file);
 // Specify the upload speed limit as 819,200 bit/s, i.e., 100 KB/s
request.setTrafficLimitBitPS(819200);
client.putObject(request);

AppendObject example code:

                Java
                
            

                AppendObjectRequest request = new AppendObjectRequest(bucketName, objectKey, file);
 // Specify the upload speed limit as 819,200 bit/s, i.e., 100 KB/s
request.setTrafficLimitBitPS(819200);
AppendObjectResponse response = client.appendObject(request);
Long nextAppendOffset = response.getNextAppendOffset();
            

PutSuperObjectFromFile example code:

                Java
                
                PutSuperObjectRequest request = new PutSuperObjectRequest(bucketName, objectKey, file);
 // Specify the upload speed limit as 819,200 bit/s, i.e., 100 KB/s
request.setTrafficLimitBitPS(819200);
client.putSuperObjectFromFile(request);

Upload callback

The API is used by BOS to provide synchronous callbacks to the application server when uploading file (object). The application server refers to the server corresponding to the callback address provided by the user. For specific descriptions, please refer to Upload Callback. The usage of the Java SDK server upload callback is as follows:

PutObject example code:

                Java
                
            

                PutObjectRequest putObjectRequest = new PutObjectRequest("bucketName","objectKey", file);
String callBackUrl = "[\""+"https://localhost:8080/callback" +"\"]";
String enCodeUrl = "";
try {
    enCodeUrl = Base64.getEncoder().encodeToString(callBackUrl.getBytes(DEFAULT_ENCODING));
}catch (Exception e){
    e.printStackTrace();
}
putObjectRequest.setxBceProcess("callback/callback,u_" + enCodeUrl);
PutObjectResponse putObjectResponse = client.putObject(putObjectRequest);
System.out.println(putObjectResponse.getCallback().getResult());
            

Download file

Restore archived storage class files