Convert Normal Bucket to Hierarchical Namespace Bucket

Overview

The BOS metadata conversion service allows you to seamlessly convert a standard BOS bucket into a hierarchical namespace (HNS) bucket without needing additional tools. HNS buckets support most features of standard flat buckets, are compatible with most HDFS file system APIs, and offer atomic directory operations like rename and delete. They also enhance the performance of list and head requests, making them ideal for data lake computing scenarios in the big data and AI domains.

Prerequisites

1. Currently, the metadata conversion service and HNS buckets are available only in Beijing, Suzhou, Guangzhou, and Baoding regions.
2. To activate the BOS metadata conversion feature, you must submit a ticket to request access.

Usage restrictions

1. Conversion is not available for buckets that have the trash or versioning feature enabled, or that include archived objects, symlinks, or object tags.
2. A flat bucket converted via the metadata conversion process cannot revert to a non-hierarchical namespace; the change is irreversible. Only the metadata organization format changes during conversion—no data is physically moved.
3. The time required to complete a metadata conversion task depends on the number of objects in the bucket and backend resource availability. Currently, no guaranteed time frame can be provided.

Risk statement

• Once the conversion task begins and until it is completed, avoid incremental writes, overwrites, or deletions of existing data to prevent data loss. If archived storage class files are encountered during conversion, the task will fail and automatically abort.
• Ensure there are no conflicting files and folders with the same name in the BOS bucket to be converted (e.g., both a file named dir and a folder named dir/). Such conflicts arise during conversion due to import order. The service resolves conflicts through renaming; for more information, refer to the conversion rules.
• Check the conversion rules to verify that no files with special reserved names exist in the BOS bucket, and ensure such files are not written during the conversion process. Otherwise, files with identical names may overwrite one another, leading to data loss.
• Before initiating bucket metadata conversion, back up your data using methods like replication. Once the metadata conversion is complete, verify the integrity of the bucket's data again.

Conversion rules

• For files ending with / before conversion (e.g., a/b/): If the file is empty (size = 0), a directory with the same name (e.g., a/b) will be created directly after conversion. If the file is non-empty, it will be renamed with the special suffix .slash followed by the conversion timestamp. For instance, the file a/b/ will become a/b.slash.1716487553. You can use the ListObjects API to check the newly converted filename.

• HNS buckets do not allow files and directories with the same name to coexist. Hence, if the bucket being converted contains both a file a/b and a directory a/b/, a conflict will occur during the conversion process (determined by their processing order). The standard resolution prioritizes creating and retaining directories.

  • Scenario I: When creating the directory a/b, if a file named a/b already exists, the file will be renamed with a special suffix .file followed by the conversion timestamp. For example, a/b becomes a/b.file.1716487553. If the new filename already exists, a numeric suffix is appended for a single retry (e.g., a/b.file.1716487553.1).
  • Scenario II: When creating the file a/b, if a directory named a/b already exists, a new file will be created with the special suffix .file followed by the conversion timestamp (e.g., a/b.file.1716487553). If this new name conflicts, up to three retries will be made with numeric suffixes appended (e.g., a/b.file.1716487553.1).