百度智能云

All Product Document

          Object Storage

          Lifecycle Management

          Basic Introduction

          Files stored on BOS usually have lifecycle processes such as new construction, archiving and deletion. When a file is newly created, it is usually read and accessed frequently, then the number of reads decreases and gradually cools down, and finally it is deleted. If users manually maintain the lifecycle of data, it will take time and effort. However, if it is not maintained, the data will always be stored in standard storage, which will incur high costs. BOS provides lifecycle management to help users automate data lifecycle management and realize the automatic process of data creation, archiving and deletion, thus saving labor and money costs.

          Lifecycle management supports the following functions:

          • Custom time to storage type, from standard storage to infrequency storage, to cold storage, to archive storage
          • Delete data that is no longer needed
          • Clear expired three-step upload data

          From the perspective of scenarios, lifecycle management supports two modes:

          • Data are archived or deleted automatically after reaching a certain life. For example, it is defined that all data that has been created for more than 30 days will be transferred to infrequency storage.
          • Specify the time to perform the operation. Such as filing all documents at a specific time.

          Operation Method

          You can use lifecycle function in both the console and the API:

          1.To manage the lifecycle through the console, please refer to Manage Lifecycle.
          2.To manage lifecycle through API, refer to interface Putbucketlifecycle, GetLifeCycle, DeleteLifeCycle. The following focuses on the rule file syntax when managing the lifecycle through API.

          Rule Definition

          The lifecycle management service provided by BOS is implemented by defining rules and setting actions for each rule.

          Note:

          • BOS Lifecycle Rules will take effect within one day after they are set.
          • After the rules take effect, BOS processes the eligible objects accordingly, but it takes some time (usually a few hours), so you may not see the effect immediately after the rules are set.
          • The time calculated in the rules (namely the "age" of object) is based on the creation time of object, instead of the creation/modification time of the lifecycle rules.
          • BOS only saves the last modification time, namely the last-modified time; if the meta is not updated or overrides files, the last-modified time is the creation time. So, the "creation time" in the lifecycle is actually the last-modified time.

          Rule Definition follows the following principles:

          • Rules are based on bucket definitions;
          • Each bucket can have up to 1000 rule entries;

          The following is an example of a lifecycle rule that is attached to a bucket named samplebucket and contains 3 rule entries:

          • Delete all files under Prefix1 after September 7, 2016;
          • Automatically transfer files created under Prefix2 for more than 30 days to infrequency storage;
          • Automatically clear the three-step uploads that have not been completed for more than 5 days under Prefix3.

            { "rule": [ { "id": "sample-rule-delete-prefix", "status": "enabled", "resource": [ "samplebucket/prefix1/*" ], "condition": { "time": { "dateGreaterThan": "2016-09-07T00:00:00Z" } }, "action": { "name": "Deleteobject" } }, { "id": "sample-rule-transition-prefix", "status": "enabled", "resource": [ "samplebucket/prefix2/*" ], "condition": { "time": { "dateGreaterThan": "$(lastModified)+P30D" } }, "action": { "name": "Transition", "storageClass": "STANDARD_IA" } }, { "id": "sample-rule-abort-multiupload-prefix", "status": "enabled", "resource": [ "samplebucket/prefix3/*" ], "condition": { "time": { "dateGreaterThan": "$(lastModified)+P5D" } }, "action": { "name": "AbortMultipartUpload" } } ], }

          Each rule consists of id, status, resource, condition, and action, which are explained in the following table.

          Rule item Description Is it necessary to fill in Comments
          id Identifier of the rule. Optional Rule id in the same bucket must be unique and cannot be repeated. If the user does not fill in, the system will automatically generate one for the user.
          status Status of the rule. Required The value can be enabled or disabled. When the rule is disabled, the rule does not take effect.
          resource For which resources the rule takes effect. Required For example: Effective for object prefixed with prefix/in samplebucket: samplebucket/prefix/*; effective for all objects in samplebucket: samplebucket/*
          condition Conditions that the rule depends on. Required Currently only time is supported.
          +time Time restricted conditions. Required Implemented by the defined dateGreaterThan.
          ++dateGreaterThan Describe the time relationship. Required Support absolute time date and relative time days. The format of absolute time date is yyyy-mm-ddThh: mm: ssZ, eg. 2016-09-07T00: 00: 00Z. The absolute time is UTC time and must end with 0:00:00 (UTC 0 o'clock); the description of relative time days follows ISO8601, and the minimum time granularity supported is day, such as: $ (lastModified) + P7D represents the time of object 7 days after last-modified.`
          action Action to perform on resource. Required -
          +name Name of operation performed. Required The values are Transition,Deleteobject and AbortMultipartUpload.
          +storageClass Storage type of object Optional When the action is Transition, it can be set. The value is STANDARD_IA, which means the change from standard type to infrequency storage type, the value is COLD, which means the change from standard type to cold storage type, and the value is ARCHIVE, which means the change from standard type to archive storage type.

          Action

          BOS supports three operations: Transition, Deleteobject and AbortMultipartUpload. Details of the three operations are as follows:

          Operation Description Comments
          Transition Convert resource to the storage type. -Files must be stored in standard storage for more than 7 days before they can be converted to infrequency storage. Because the 7 days after the file is created are usually frequent access periods, converting to low frequencies may increase costs. Files stored in infrequency storage for more than 30 days will be converted to cold storage, because there may be infrequency access requirements in infrequency storage. If converted to cold storage, the cost may be increased
          -the last-modified of files will not change when converted to infrequency through the lifecycle.
          Deleteobject Delete the resource. Delete the resource according to the set rules. infrequency storage has a minimum storage time of 30 days, so if users create rules to delete a infrequency file that is stored for less than 30 days, it will incur extra expenses.
          AbortMultipartUpload Clear the three-step uploads that have not been completed for a period of time. You need to set the time (days) when the chunked upload is discarded.

          DateGreaterThan

          BOS supports dateGreaterThan settings in the form of absolute time (date) and relative time (days). The following matters should be noted:

          DateGreaterThan entry Description Support action Comments
          days The operation is performed after the data reaches lastModified+days. Three operations are supported: transition, expiration and abortMultipartUpload.
          -For files that use days for lifecycle management, the operation time is: File's Last-modified plus days, rounded to the next UTC midnight. For example, the last update time of an object is 1: 00 a.m. on April 12, 2014 in UTC, the number of days defined by the matching rule is 3 days, and the operation time is 0: 00 a.m. on April 16, 2014 in UTC.
          -If the last-modified of the file is later modified, such as meta update and data overwrite, the execution time of the operation will also change accordingly.
          date Perform the operation on the specified date. Support both transition and expiry operations.
          -If date is specified, the date must be UTC midnight and conform to ISO8601 format like 2014-01-01T00:00:00.000Z. The
          -date rule will only work for files with a Last-modified time earlier than or equal to date.
          -date rule executes at the current time>=date.
          -When date conflicts with "standard storage for 7 days", "standard storage for 7 days" will be given priority.

          Note:

          • Each rule can contain only one action.
          • Each resource can have at most one rule for each action, for example, bucket/prefix1/* can't have two transition rules, but can have one transition rule plus one expansion rule.
          • If the scope of action defined by prefix in the rule has an inclusion relationship, the action defined by the minimum scope of action shall prevail for the included prefix. For example, rule 1 defines prefix as the 10-day delete of the object of p1/p1/, rule 2 defines prefix as the 15-day delete of the object of p1/p1/p1/, rule 1 contains the prefix of rule 2, performs the 15-day delete for the object with the prefix of p1/p1/p1/, and performs the 10-day delete action for the object other than p1/p1/p1/in p1/p1/.
          • If date is used as Condition, for example, the file with "folder/" prefix will turn to infrequency after 30 days and be deleted on August 17, 2016.The system will calculate the time when each file performs the corresponding operation according to the rules and compare the time. If the file performs transition and expansion on the same day, it is deleted directly. If the file deletion time is earlier than the infrequency conversion time, it is also deleted directly.
          Previous
          Set Bucket Tag
          Next
          Basic Operations of Object