Search analysis log QueryLogRecord
Description
Users can search or analyze data in a specified logstore by submitting a query. Only one logstore can be queried at a time.
Query statements support three formats, such as:
match search statement: it is required to enable the full-text index or field-level index and search the log contents according to the conditions, for example:match method:GET and status >= 400SQL analysis statement: execute SQL statements, for example:select * limit 10match search statement| SQL statement: it is required to enable the full-text index or field-level index and execute SQL statements on the result set that meets the search conditions, with the search statement separated from the SQL statement by a vertical line, for example:match method:GET and status >= 400 | select host, count(*) group by host
Query related restrictions are as follows:
- The maximum number of query concurrency supported by each account is 15
- Limit the size of the result set returned by search statement to not more than 1000 records
- Limit the size of the result set returned by analysis statement to not more than 100000 records
For the part related to search, enable index; the search syntax may refer to Search Syntax
Analysis statements may not contain from clauses, and syntax details may refer to SQL Syntax
Request
- Request syntax
1GET /v1/logstore/{logStoreName}/logrecord?project={project} HTTP/1.1
2Host: <Endpoint>
3Authorization: <Authorization String>
- Request headers
No additional headers are required beyond the standard request headers.
- Request parameters
| Parameter name | Types | Required or not | Parameter location | Description |
|---|---|---|---|---|
| project | String | No | Query | Project name |
| logStoreName | String | Yes | Path | Logstore name |
| logStreamName | String | No | Query | Logstream name, composed of at most 192 characters which are only limited to: a-z, A-Z, 0-9, “_”, “-”, “.” , “/” |
| query | String | Yes | Query | Input Query statement |
| startDateTime | DateTime | Yes | Query | Start datetime, including start time |
| endDateTime | DateTime | Yes | Query | End datetime, excluding end time |
| marker | string | No | Query | This parameter is valid only when the query parameter is a search statement, specifying the starting line of the query. The default value is an empty string. |
| limit | Int | No | Query | This parameter is valid only when the query parameter is a search statement, defining the maximum number of logs returned per request. Minimum: 0, Maximum: 1000, Default: 100. |
| sort | string | No | Query | This parameter is valid only when the query parameter is a search statement, determining the sorting method for logs. "desc": logs are returned in descending order based on the log timestamp; "asc": logs are returned in ascending order based on the log timestamp; Default: "desc.\ |
Note: The datetime is in UTC ISO8601 format, for example: 2020-01-10T13:23:34Z; the ranges of startDateTime and endDateTime restrict the logrecord timestamp
Response
- Response headers
No additional headers are required beyond the standard response headers.
- Response parameters
| Parameter name | Types | Required or not | Description |
|---|---|---|---|
| nextMarker | string | No | When query is a search statement, a nextMarker field is returned and the next log is marked if additional log data are available, or the nextMarker field is not returned or an empty string is returned in case of no next page |
| resultSet | Object | Yes | Return the result set meeting the conditions |
| datasetScanInfo | Object | Yes | Scan the statistic information of raw datasets |
Therein, the resultset represents the result set obtained by this query, structured as follows:
| Parameter name | Types | Required or not | Description |
|---|---|---|---|
| queryType | string | Yes | Returned statement type, match: search statement, sql: include analysis statement |
| columns | List<string> | Yes | Included field name list |
| columnTypes | List<string> | Yes | Included field name lists, corresponding to columns one by one. |
| rows | List<List<obj>> | Yes | Each row corresponds to the column name in columns |
| tags | List<Map<string, string>> | No | Tags in the log data, corresponding to rows one by one |
datasetScanInfo represents the statistic information of raw dataset scanning, structured as follows:
| Parameter name | Types | Required or not | Description |
|---|---|---|---|
| statistics | Object | Yes | Partial statistic information of raw dataset scanning |
statistics represents the statistic information of the raw datasets scanned during this query:
| Parameter name | Types | Required or not | Description |
|---|---|---|---|
| executionTimeInMs | Int | Yes | Time spent by this query, unit: millisecond |
| scanCount | Int | Yes | Number of data scanned during this query |
| scanBytes | Int | Yes | Number of data bytes scanned during this query |
Error code
Apart from standard error codes, the following codes may be returned:
| Error code | Error message | Description | HTTP status code |
|---|---|---|---|
| QueryParseError | [msg] | msg indicates the specific error content | 400 |
| InvalidParameter | Request param invalid: [param] | param indicates the error parameter | 400 |
| LogStoreStopped | LogStore stopped | LogStore write operations blocked due to debt | 403 |
| LogStoreNotFound | LogStore not found | LogStore not found | 404 |
| ExecutionTimeout | [msg] | msg indicates the specific error content | 408 |
| LogStoreNotReady | LogStore not ready | LogStore in initialization | 409 |
Example
Search statement
- Request example
1GET /v1/logstore/demo/logrecord?project=default&logStreamName=my-stream&marker=CNCetPuLMhD__4vWtPuLMg&startDateTime=2019-09-01T22:00:00Z&endDateTime=2019-09-01T23:00:00Z&query=match * HTTP/1.1
2Host: bls-log.bj.baidubce.com
3Authorization: bce-auth-v1/18717522d39411e9b721df098b0b908c/2019-09-10T07:00:20Z/1800/content-type;host;x-bce-date;x-bce-request-id/6a7cb6c9ac7ec156c805e55e7d0bcfc443b47feee97cf099c1c0d93a0b4c8304
- Response example
1HTTP/1.1 200 OK
2Date: Thu, 12 Jun 2014 09:26:46 GMT
3Content-Type: application/json; charset=utf-8
4X-Bce-Request-Id:47e0ef1a-9bf2-11e1-9279-0100e8cf109a
5{
6 "resultSet": {
7 "queryType": "match",
8 "columns": [
9 "@timestamp",
10 "@seq",
11 "@stream",
12 "@raw"
13 ],
14 "columnTypes": [
15 "int",
16 "int",
17 "string",
18 "string"
19 ],
20 "rows": [
21 [
22 1721198514000,
23 28200116569702400,
24 "wang_test",
25 "{\"@raw\": \"hello test1004\", \"level\": \"info\"}"
26 ]
27 ]
28 },
29 "datasetScanInfo": {
30 "statistics": {
31 "executionTimeInMs": 23,
32 "scanCount": 23,
33 "scanBytes": 3159
34 }
35 }
36}
Analysis statement
- Request example
1GET /v1/logstore/demo/logrecord?project=default&startDateTime=2019-09-01T22:00:00Z&endDateTime=2019-09-01T23:00:00Z&query=select level, count(*) group by level HTTP/1.1
2Host: bls-log.bj.baidubce.com
3Authorization: bce-auth-v1/18717522d39411e9b721df098b0b908c/2019-09-10T07:00:20Z/1800/content-type;host;x-bce-date;x-bce-request-id/6a7cb6c9ac7ec156c805e55e7d0bcfc443b47feee97cf099c1c0d93a0b4c8304
- Response example
1HTTP/1.1 200 OK
2Date: Thu, 12 Jun 2014 09:26:46 GMT
3Content-Type: application/json; charset=utf-8
4X-Bce-Request-Id:47e0ef1a-9bf2-11e1-9279-0100e8cf109a
5{
6 "resultSet": {
7 "queryType": "sql",
8 "columns": [
9 "level",
10 "count(*)"
11 ],
12 "columnTypes": [
13 "string",
14 "int"
15 ],
16 "rows": [
17 [
18 "DEBUG",
19 32
20 ],
21 [
22 "INFO",
23 1
24 ]
25 ]
26 },
27 "datasetScanInfo": {
28 "statistics": {
29 "executionTimeInMs": 26,
30 "scanCount": 47,
31 "scanBytes": 979
32 }
33 }
34}
