Lifecycle Management
Basic Introduction
Files stored on BOS usually have lifecycle processes such as new construction, archiving and deletion. When a file is newly created, it is usually read and accessed frequently, then the number of reads decreases and gradually cools down, and finally it is deleted. If users manually maintain the lifecycle of data, it will take time and effort. However, if it is not maintained, the data will always be stored in standard storage, which will incur high costs. BOS provides lifecycle management to help users automate data lifecycle management and realize the automatic process of data creation, archiving and deletion, thus saving labor and money costs.
Lifecycle management supports the following functions:
- Custom time to storage type, from standard storage to infrequency storage, to cold storage, to archive storage
- Delete data that is no longer needed
- Clear expired three-step upload data
From the perspective of scenarios, lifecycle management supports two modes:
- Data are archived or deleted automatically after reaching a certain life. For example, it is defined that all data that has been created for more than 30 days will be transferred to infrequency storage.
- Specify the time to perform the operation. Such as filing all documents at a specific time.
Operation Method
You can use lifecycle function in both the console and the API:
1.To manage the lifecycle through the console, please refer to Manage Lifecycle.
2.To manage lifecycle through API, refer to interface Putbucketlifecycle, GetLifeCycle, DeleteLifeCycle. The following focuses on the rule file syntax when managing the lifecycle through API.
Rule Definition
The lifecycle management service provided by BOS is implemented by defining rules and setting actions for each rule.
Note:
- BOS Lifecycle Rules will take effect within one day after they are set.
- After the rules take effect, BOS processes the eligible objects accordingly, but it takes some time (usually a few hours), so you may not see the effect immediately after the rules are set.
- The time calculated in the rules (namely the "age" of object) is based on the creation time of object, instead of the creation/modification time of the lifecycle rules.
- BOS only saves the last modification time, namely the last-modified time; if the meta is not updated or overrides files, the last-modified time is the creation time. So, the "creation time" in the lifecycle is actually the last-modified time.
Rule Definition follows the following principles:
- Rules are based on bucket definitions;
- Each bucket can have up to 1000 rule entries;
The following is an example of a lifecycle rule that is attached to a bucket named samplebucket and contains 3 rule entries:
- Delete all files under Prefix1 after September 7, 2016;
- Automatically transfer files created under Prefix2 for more than 30 days to infrequency storage;
-
Automatically clear the three-step uploads that have not been completed for more than 5 days under Prefix3.
{ "rule": [ { "id": "sample-rule-delete-prefix", "status": "enabled", "resource": [ "samplebucket/prefix1/*" ], "condition": { "time": { "dateGreaterThan": "2016-09-07T00:00:00Z" } }, "action": { "name": "Deleteobject" } }, { "id": "sample-rule-transition-prefix", "status": "enabled", "resource": [ "samplebucket/prefix2/*" ], "condition": { "time": { "dateGreaterThan": "$(lastModified)+P30D" } }, "action": { "name": "Transition", "storageClass": "STANDARD_IA" } }, { "id": "sample-rule-abort-multiupload-prefix", "status": "enabled", "resource": [ "samplebucket/prefix3/*" ], "condition": { "time": { "dateGreaterThan": "$(lastModified)+P5D" } }, "action": { "name": "AbortMultipartUpload" } } ], }
Each rule consists of id, status, resource, condition, and action, which are explained in the following table.
Rule item | Description | Is it necessary to fill in | Comments |
---|---|---|---|
id | Identifier of the rule. | Optional | Rule id in the same bucket must be unique and cannot be repeated. If the user does not fill in, the system will automatically generate one for the user. |
status | Status of the rule. | Required | The value can be enabled or disabled . When the rule is disabled , the rule does not take effect. |
resource | For which resources the rule takes effect. | Required | For example: Effective for object prefixed with prefix/in samplebucket: samplebucket/prefix/* ; effective for all objects in samplebucket: samplebucket/* |
condition | Conditions that the rule depends on. | Required | Currently only time is supported. |
+time | Time restricted conditions. | Required | Implemented by the defined dateGreaterThan . |
++dateGreaterThan | Describe the time relationship. | Required | Support absolute time date and relative time days. The format of absolute time date is yyyy-mm-ddThh: mm: ssZ, eg. 2016-09-07T00: 00: 00Z. The absolute time is UTC time and must end with 0:00:00 (UTC 0 o'clock); the description of relative time days follows ISO8601, and the minimum time granularity supported is day, such as: $ (lastModified) + P7D represents the time of object 7 days after last-modified.` |
action | Action to perform on resource. | Required | - |
+name | Name of operation performed. | Required | The values are Transition ,Deleteobject and AbortMultipartUpload . |
+storageClass | Storage type of object | Optional | When the action is Transition , it can be set. The value is STANDARD_IA , which means the change from standard type to infrequency storage type, the value is COLD , which means the change from standard type to cold storage type, and the value is ARCHIVE , which means the change from standard type to archive storage type. |
Action
BOS supports three operations: Transition, Deleteobject and AbortMultipartUpload. Details of the three operations are as follows:
Operation | Description | Comments |
---|---|---|
Transition | Convert resource to the storage type. | -Files must be stored in standard storage for more than 7 days before they can be converted to infrequency storage. Because the 7 days after the file is created are usually frequent access periods, converting to low frequencies may increase costs. Files stored in infrequency storage for more than 30 days will be converted to cold storage, because there may be infrequency access requirements in infrequency storage. If converted to cold storage, the cost may be increased -the last-modified of files will not change when converted to infrequency through the lifecycle. |
Deleteobject | Delete the resource. | Delete the resource according to the set rules. infrequency storage has a minimum storage time of 30 days, so if users create rules to delete a infrequency file that is stored for less than 30 days, it will incur extra expenses. |
AbortMultipartUpload | Clear the three-step uploads that have not been completed for a period of time. | You need to set the time (days) when the chunked upload is discarded. |
DateGreaterThan
BOS supports dateGreaterThan settings in the form of absolute time (date) and relative time (days). The following matters should be noted:
DateGreaterThan entry | Description | Support action | Comments |
---|---|---|---|
days | The operation is performed after the data reaches lastModified+days. | Three operations are supported: transition, expiration and abortMultipartUpload. | -For files that use days for lifecycle management, the operation time is: File's Last-modified plus days, rounded to the next UTC midnight. For example, the last update time of an object is 1: 00 a.m. on April 12, 2014 in UTC, the number of days defined by the matching rule is 3 days, and the operation time is 0: 00 a.m. on April 16, 2014 in UTC. -If the last-modified of the file is later modified, such as meta update and data overwrite, the execution time of the operation will also change accordingly. |
date | Perform the operation on the specified date. | Support both transition and expiry operations. | -If date is specified, the date must be UTC midnight and conform to ISO8601 format like 2014-01-01T00:00:00.000Z. The -date rule will only work for files with a Last-modified time earlier than or equal to date. -date rule executes at the current time>=date. -When date conflicts with "standard storage for 7 days", "standard storage for 7 days" will be given priority. |
Note:
- Each rule can contain only one action.
- Each resource can have at most one rule for each action, for example, bucket/prefix1/* can't have two transition rules, but can have one transition rule plus one expansion rule.
- If the scope of action defined by prefix in the rule has an inclusion relationship, the action defined by the minimum scope of action shall prevail for the included prefix. For example, rule 1 defines prefix as the 10-day delete of the object of p1/p1/, rule 2 defines prefix as the 15-day delete of the object of p1/p1/p1/, rule 1 contains the prefix of rule 2, performs the 15-day delete for the object with the prefix of p1/p1/p1/, and performs the 10-day delete action for the object other than p1/p1/p1/in p1/p1/.
- If date is used as Condition, for example, the file with "folder/" prefix will turn to infrequency after 30 days and be deleted on August 17, 2016.The system will calculate the time when each file performs the corresponding operation according to the rules and compare the time. If the file performs transition and expansion on the same day, it is deleted directly. If the file deletion time is earlier than the infrequency conversion time, it is also deleted directly.