Set access logs
Overview
If users need to track access requests for BOS, they can enable the access log functionality for the bucket. Each access log records detailed information about an individual access request, including the requester, bucket name, request time, and request operation, among other details. The log function is useful for statistical analysis and security audits. Once enabled, access logs for the bucket will automatically be written to a user-specified bucket as log files, following a predefined naming convention, on an hourly basis.
Note: Storing access logs in the target bucket is a "best-effort" process. Typically, it takes 2 to 3 hours for logs to be stored in the bucket after an access request occurs. However, network conditions and other factors may lead to some logs being delayed for a longer period before storage.
Operation types
You can set up access logs through the console and API, with the specific methods as follows:
- To set up access logs via the console, please refer to Setting Up Access Logs.
-
Setting up access logs via API:
Log naming rules
The naming rule for the object that stores access log records is <TargetPrefix><SourceBucketName>-YYYY-mm-DD-HH-MM-SS-UniqueString, for example: mylog/mybucket-2016-05-17-10-00-00-00000. Where:
- TargetPrefix: The prefix specified by the user, i.e., mylog/ in the example;
- SourceBucketName: The name of the source bucket, i.e., mybucket in the example;
- YYYY, mm, DD, HH, MM, and SS: Represent the year, month, day, hour, minute, and second corresponding to the minimum timestamp of the access request recorded in the log, respectively;
- UniqueString: A string generated by the BOS system to uniquely identify a log file (e.g., "00000" in the example).
Log format
| Output fields | Examples | Description |
|---|---|---|
| Time | 2016-09-05T09:02:03+08:00 | Time when the request was initiated |
| Bucket Owner | e13b12d0131b4c8bae959df4969387b8 | Owner ID of the bucket |
| Bucket | samplebucket | Name of the bucket accessed by the request |
| Key | dir/sample.jpg | Name of the object accessed by the request |
| Reserve | - | Reserved fields |
| Remote IP | 111.30.20.12 | IP address of the requester |
| Requester | e13b12d0131b4c8bae959df4969387b8 | ID of the requester |
| Request ID | a94dbe12-717e-4e72-bcf2-0aa4a3100360 | UUID for uniquely identifying the request (same as the x-bce-request-id in the Header) |
| Operation | GetObject | API corresponding to the request |
| Request Host | bj.bcebos.com | Host header in the HTTP request |
| Request URI | "GET/v1/tables.xml" | URI in the HTTP request (must be enclosed in double quotes) |
| HTTP Status | 200 | HTTP status code returned by BOS |
| Error Code | InvalidURI | Error code returned by BOS |
| Bytes Send | 2314 | Traffic downloaded by the user from BOS |
| Object Size | 1024 | Object size (unit: Bytes) |
| Total Time(ms) | 11 | Total time from starting to receive the request to returning the response (unit: milliseconds) |
| Server Cost Time (ms) | 10 | The total time taken from when all request data is fully received to when the response to the user's request begins (measured in milliseconds). Currently, this field is represented as “-”. |
| Referer | <https://cloud.baidu.com/product/bos.html> |
Referer header in the HTTP request |
| User Agent | "curl/7.33.0" | User Agent header in the HTTP request |
| Storage Class | STANDARD | Storage class of the object |
| Multiple Delete | "abc%0Adef" | Information of objects deleted by “Multiple Delete” (with “%0A” as the delimiter for multiple objects) |
| vip | 10.0.0.100 | VIP requested by the user |
| rename_src | "test" | Source object for “rename” operation |
| ak | 12342556 | User’s AK |
| source | cdn | CDN identifier for CDN back-to-origin requests, “-” indicates the request is not a CDN back-to-origin request |
| mirror_type | 302 | "302" indicates a redirected mirror back-to-origin request. "fetch" refers to a mirror back-to-origin request for image fetching. "-" signifies that the request is not a mirror back-to-origin request. |
| copy_src | STANDARD | Source storage class for “copy” operation |
Log sample:
2016-09-05T09:02:03+08:00 e13b12d0131b4c8bae959df4969387b8 bos-logging - - 127.0.0.1 e13b12d0131b4c8bae959df4969387b8 98a20094-2002-4950-899d-9347244c71da GetBucket bj.bcebos.com "/bos-logging?delimiter=%2F&maxKeys=1000&prefix=" 200 success 2335 - 10 - "-" "bce-sdk-python/0.8.9/2.7.3 (default, Jun 24 2016, 15:08:42) [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)]/linux2" - - - - 123 cdn 302 STANDARD
Note:
- Log fields are separated by spaces.
- In BOS logs, certain fields may include spaces. Such fields (e.g., "Referrer") are enclosed in quotation marks when the log is generated.
- The "Reserve" field is a reserved field represented by "-" in the log.
- BOS generates log files for bucket access on an hourly basis. However, this does not guarantee that all requests within a specific hour will be recorded in the log file for that hour; some requests might appear in the log file for the previous or following hour.
- The log files created by BOS occupy storage space in the target bucket. While the log access function itself is free, the storage space consumed by the logs will incur regular charges. Users can manage log files just like they do with regular objects.
- After log files are generated in the target bucket, BOS will not proactively clean up these log files. The target bucket for storing logs must be in the same region as the source bucket, and logs can also be delivered to the source bucket itself.