LogRecord Operations

Push LogRecord

Support bulk pushing of log records to the BLS platform. The log record format can be either TEXT or JSON. For TEXT log records, logs remain unprocessed; for JSON log records, JSON fields are automatically identified (only first-level fields are currently supported, nested fields are not yet supported).

To upload both specific fields extracted from processing and the raw log text, you can use JSON format, including the raw log text as part of JSON (@raw as the key and the raw log text as the value). When processing @raw, BLS will interpret its content as the raw log text.

Use the following code to bulk push JSON log records to the specified log stream within the specified LogStore.

1// Push JSON logrecord
2pushLogRecordRequest := PushLogRecordRequest{
3    Project:       "default",
4    LogStoreName:  "test,
5    LogStreamName: "json-logStream",
6    LogType:       "JSON",
7    LogRecords: []api.LogRecord{
8        {
9            Message:   "{\"@raw\": \"raw text\", \"level\": \"info\"}",
10            Timestamp: time.Now().UnixMilli(),
11        },
12    },
13}
14 // Specify the logrecord type to JSON, and push the logrecord to json-logstream in logstore demo
15 // If this logstream is unavailable, create a logstream automatically
16err = BLS_CLIENT.PushLogRecordV2(pushLogRecordRequest)
17if err != nil {
18  fmt.Println("Push logRecords failed: ", err)
19} else {
20  fmt.Println("Push logRecords success")
21}

Prompt:

• Detailed parameter configuration and restrictions may refer to BLS API document Push Logrecord

View the specified logrecord

With following codes, view the logrecords in the specified logstream, and obtain the recent logrecord contents or the time range of use for filtering.

1pullLogRecordRequest := PullLogRecordRequest{
2    Project:       "default",
3    LogStoreName:  "test",
4    StartDateTime: time.Now().Add(-10 * time.Minute).UTC().Format("2006-01-02T15:04:05Z"),
5    EndDateTime:   time.Now().UTC().Format("2006-01-02T15:04:05Z"),
6}
7res, err := BLS_CLIENT.PullLogRecordV2(pullLogRecordRequest)
8if err != nil {
9  fmt.Println("Pull logRecord failed: ", err)
10} else {
11  fmt.Println("LogRecords result: ", res)
12}

Prompt:

• Detailed parameter configuration and restrictions may refer to BLS API document Pull Logrecord

Query the specified logrecord

Users can search or analyze data in a specified logstore by submitting a query. Only one logstore can be queried at a time.

Query statements support three formats, such as:

• match search statement: it is required to enable the full-text index or field-level index and search the log contents according to the conditions, for example:match method:GET and status >= 400
• SQL statement: execute SQL statements, for example: select * limit 10
• match search statement| SQL statement: it is required to enable the full-text index or field-level index and execute SQL statements on the result set that meets the search conditions, with the search statement separated from the SQL statement by a vertical line, for example: match method:GET and status >= 400 | select host, count(*) group by host

Query related restrictions are as follows:

• The maximum number of query concurrency supported by each account is 15
• Limit the size of the returned result set to a maximum of 1 MB or 1,000 records.

For search syntax, please refer to Search Syntax

SQL statements may not contain from clauses, and syntax details may refer to SQL Syntax

Use the following code to query log records that satisfy specified conditions within the desired LogStore.

1queryLogRecordRequest := QueryLogRecordRequest{
2    Project:       "default",
3    LogStoreName:  "test",
4    Query:         "match *",
5    StartDateTime: time.Now().Add(-20 * time.Second).UTC().Format("2006-01-02T15:04:05Z"),
6    EndDateTime:   time.Now().UTC().Format("2006-01-02T15:04:05Z"),
7}
8res, err := BLS_CLIENT.QueryLogRecordV2(queryLogRecordRequest)
9if err != nil {
10  fmt.Println("Query logRecord failed: ", err)
11} else {
12  fmt.Println("LogRecords result: ", res)
13}

Prompt:

• Detailed parameter configuration and restrictions may refer to BLS API document QueryLogRecord