List files in the storage space

The BOS SDK allows users to list objects in the following two ways:

• Simple listing
• Advanced listing via parameters

In addition, users can also simulate folders while listing files.

Simple listing

After users complete a series of uploads, they may need to view all objects in a specified bucket, which can be achieved through the following code:

1response = bos_client.list_objects(bucket_name)
2for object in response.contents:
3    print(object.key)

Note: 1. By default, if the number of objects in the bucket is greater than 1,000, only 1,000 objects will be returned, and the is_truncated value in the return result will be True, and next_marker will be returned as the starting point for the next reading. 2. To increase the number of returned objects, the marker parameter can be used for reading in batches.

Users can also list all the objects in the current bucket at once.

1for object in bos_client.list_all_objects(bucket_name):
2    print(object.key)

Advanced listing via parameters

Other optional parameters of the list_objects method include:

Note:

1. If there is an object named with the prefix, when querying only with prefix, all returned keys will still include the object named with the Prefix. For details, see [Recursively List All Files in a Directory](#Simulate folder function).
2. If there is an object named with the prefix, when querying with the combination of prefix and delimiter, there will be Null in all returned keys, and the key names do not contain the prefix. For details, see [View Files and Subdirectories in a Directory](#Simulate folder function).

The following are several examples to illustrate the method of listing via parameters:

Specify the maximum number of returned items

1    max_keys = 500
2# Specify the maximum number of returned items as 500
3    response = bos_client.list_objects(bucket_name, max_keys = max_keys)
4    for obj in response.contents:
5        print(obj.key)

Return objects with a specified prefix

1    prefix = "test"
2# Specify to return objects with the prefix “test"
3    response = bos_client.list_objects(bucket_name, prefix = prefix)
4    for obj in response.contents:
5        print(obj.key)

Return from after a specified object

1    marker = "object"
2# Users can define to exclude a certain object and start returning from after it
3    response = bos_client.list_objects(bucket_name, marker = marker)
4    for obj in response.contents:
5        print(obj.key)

Pagination to get all objects

1    isTruncated = True
2# Users can set a maximum of 500 records per page
3    max_keys = 500
4    marker = None
5    while isTruncated:
6        response = bos_client.list_objects(bucket_name, max_keys = max_keys, marker=marker)
7        for obj in response.contents:
8        	print(obj.key)
9        isTruncated = response.is_truncated
10        marker = getattr(response,'next_marker',None)

Results of paginated restoration of all objects after a specific object

1    # Users can set a maximum of 500 records per page and start getting from after a specific object
2    max_keys = 500
3    marker = "object"
4    isTruncated = True
5    while isTruncated:
6    	response = bos_client.list_objects(bucket_name, max_keys = max_keys, marker=marker)
7        for obj in response.contents:
8        	print(obj.key)
9        isTruncated = response.is_truncated
10        marker = getattr(response,'next_marker',None)

Results of paginated restoration of all objects after a specific object

1    # Users can set pagination to get objects with a specified prefix, with a maximum of 500 records per page
2    max_keys = 500
3    prefix = "object"
4    isTruncated = True
5    while isTruncated:
6    	response = bos_client.list_objects(bucket_name, prefix = prefix)
7    	for obj in response.contents:
8    	    print(obj.key)
9    	isTruncated = response.is_truncated
10        marker = getattr(response,'next_marker',None)

The parameters available for calling in the resolution class returned by the list_objects method are as follows:

The list_all_objects method returns a generator for contents and is not limited by the maximum of 1,000 results returned at a time; it will return all results.

Simulate folder function

In BOS storage, there is no concept of folders. All elements are stored as objects. However, when using data, BOS users often need to manage files in folders. Therefore, BOS provides the ability to create simulated folders, which essentially involves creating an object with a size of 0. This object can be uploaded and downloaded, but the console will display objects ending with “/” as folders.

By combining the delimiter and prefix parameters, users can simulate folder functionalities. The combined effect of delimiter and prefix is as follows:

If prefix is set to a folder name, files starting with the prefix can be listed, including all files and subfolders (directory) recursively listed under the folder. File names are displayed in contents. If delimiter is set to /, the return values only list the files and subfolders (directory) under the folder. The names of subfolders (directory) under the folder are returned in the CommonPrefixes. The files and folders recursively listed under the subfolders are not displayed.

For example, if there are 5 files in the bucket: bos.jpg, fun/, fun/test.jpg, fun/movie/001.avi, and fun/movie/007.avi, the "/" character can be used as a folder delimiter.

The following are several application methods:

List all files in the bucket

When users need to get all files under a bucket, they can refer to [Pagination to Get All Objects](#Advanced listing via parameters)

Recursively list all files in a directory

You can obtain all files in the a directory by setting the Prefix parameter:

1    prefix = "fun/"
2    print("Objects:")
3# Recursively list all files under the fun directory
4    response = bos_client.list_objects(bucket_name, prefix = prefix)
5    for obj in response.contents:
6        print(obj.key)

Output:

1    Objects:
2    fun/
3    fun/movie/001.avi
4    fun/movie/007.avi
5    fun/test.jpg

View files and subdirectories in a directory

With the combination of Prefix and Delimiter, it can list files and subdirectories under the directory:

1    # "/" is the delimiter for folders
2    delimiter = "/"
3    prefix = "fun/"
4# List all files and folders under the “fun” directory
5    response = bos_client.list_objects(bucket_name, prefix = prefix, delimiter = delimiter)
6    print("Objects:")
7    for obj in response.contents:
8        print(obj.key)
9        
10# Traverse all CommonPrefix
11    print("CommonPrefixs:")
12    for obj in response.common_prefixes:
13        print(obj.prefix)

Output:

1    Objects:
2    fun/
3    fun/test.jpg
4    CommonPrefixs:
5    fun/movie/

In the returned results, the list under Objects shows the files under the fun directory. The list in CommonPrefixs shows all subfolders under the fun directory. It can be seen that the two files fun/movie/001.avi and fun/movie/007.avi are not listed because they belong to the movie directory under the fun folder.

List storage properties of objects in a bucket

In addition to viewing all objects in a specified bucket, users can also check the storage class of objects. The implementation code is as follows:

1    response = bos_client.list_objects(bucket_name)
2    for obj in response.contents:
3        print('object:{}, storage_class:{}'.format(obj.key, obj.storage_class))