Search Syntax
Search mode
After enabling full-text search or configuring indexes for fields, you can perform full-text searches or search by specifying field names and their content.
The search syntax keyword is match, with the search statement written after the match keyword and separated from the match keyword by spaces. Format:
match search statement
Supported query types:
- Numerical query: Specify fields and query conditions. Logs matching the specified field conditions will be returned as results. For example,
match status=200 and time_cost in [100 200)means querying logs where status is equal to 200 and time_cost is greater than or equal to 100 but less than 200. - Keyword query: Specify fields and keywords, and logs containing the keyword in the specified fields will be returned as results. If no field is specified, the query will be performed on the original log text. If the keyword to be queried contains search syntax operators or spaces, you can enclose the keyword in double quotes ("") to indicate that the content within the quotes should be treated as multiple keywords for the query. For example,
match method:GET and msg:"service not avaliable"means querying logs where the method is GET and msg contains the keywords service, not, and available, which is equivalent to queryingmatch method:GET and msg:service and msg:"not" and msg:avaliable. -
Phrase query: To query a specific combination of keywords, enclose the keywords in single quotes ('') as a phrase, indicating that the matching query is conducted in the order of the keywords within the single quotes. If no field is specified, the query will be performed on the original log text. For example,
match msg:'service not avaliable'means querying logs where msg contains the phrase service not available, and no other keywords precede the phrase's components; service is temporarily not available and not service available will not match the query.Limitations
- Phrase queries do not support fuzzy search. Fuzzy query keywords (* ?) in phrase queries are treated as character constants.
- Pagination: Baidu Log Service performs a phrase query with each pagination, with a maximum of 10,000 logs per query. During pagination (20 logs per page), you may encounter a certain page displaying fewer than 20 logs, but the page can still be scrolled down. For example, the total number of logs is 20,000. When you perform a phrase query, only 18 logs are returned and the page down function is available. At this time, only 18 of the top 10,000 logs meet the phrase query conditions. You can perform pagination, and Baidu Log Service will automatically perform the second phrase query in the last 10,000 logs, and return the matching logs.
-
Fuzzy query: Add fuzzy query keywords (* and ?) in the middle or at the end of keywords (where * matches any number of characters and ? matches only a single character). Baidu Log Service will search all logs to find entries that meet the specified conditions for you. For example, match http_user_agent:andr* indicates searching a log where the http_user_agent field contains terms beginning with andr within all logs.
Limitations:
- Queries must specify a prefix; the symbols * and ? cannot appear at the start of a query term.
- The more specific the search term, the more accurate the query results.
Query type corresponding to the field
- For text fields, you can perform simple queries, phrase searches, and fuzzy searches.
- For fields of type float or long, numerical ranges can be specified for queries.
For example, the query statement is match method:GET and status >= 500, means querying logs where the method field value equals to GET and status field value is ≥500.
Analysis of error causes during log search
Currently, log search requires corresponding index configurations to query the corresponding bar charts and log content; otherwise, an error will prompt that the index is not configured
- For example, match GET full-text search requires configuring full-text index
- For example, match method:GET field search requires configuring the corresponding field-level index
- For example, to query match * all logs, it currently requires configuring either a full-text index or a field-level index
Operator
| Operator | Description |
|---|---|
| and | The "and" operator: The format query1 and query2 represents the intersection of results from query1 and query2. If no operator exists between terms, the default operator is "and.\ |
| or | The "or" operator: The format query1 or query2 represents the union of results from query1 and query2. |
| not | The "not" operator: The format query1 not query2 signifies results that match query1 but exclude query2, equivalent to query1 - query2. If query1 is omitted and only not query2 is provided, it selects logs that do not contain query2. |
| () | Parentheses group one or more subqueries into a single query condition, prioritizing the conditions within the parentheses. |
| : | This is used for key-value pair queries. The format key:term represents a term query on the key's index. If the key or value has spaces, colons (:), etc., the entire key or value must be enclosed in double quotes (""). Any term within the quotes will be queried as is, without being treated as a syntax keyword. |
| * | A fuzzy query keyword used to substitute zero or more characters. For example, andr\* will return all terms starting with andr. |
| ? | A fuzzy query keyword used to replace a single character. For instance, andr?id will return all terms starting with andr, ending with id, and having one character in between. |
| " " | Enclosing a syntax keyword in " " converts it into an ordinary character. In field queries, all terms within the quotes are treated as a single unit. For example, match msg:"service not available" treats the keyword "not" as a regular character, searching for logs where the msg field contains "service," "not," and "available." Symbols *,? within double quotes represent a literal fuzzy query. |
| ' ' | This indicates a phrase query. Contents within the quotes are treated as a keyword phrase for the query, matching based on the specified keyword order in the quotes. Fuzzy query keywords (* ?) within quotes are treated as constants. |
| \ | Escape characters, used for escaping: ' " *? > = < ( )[ ] \n \r etc., the escaped symbols represent the symbols themselves. For example, if the field value of key1 contains double quotes like abc"def, you can add an escape character to search key1:abc\"def.When searching with double quotes, only escape "*?\ n \r \t, unescaped * and ? When performing a fuzzy search with single quotes, only escape '*?\ n \r \t, unescaped*and? Treated as a constant |
| > | Query logs where a field is greater than a specified value, e.g., status > 500. |
| >= | Query logs where a field is greater than or equal to a specified value, e.g., status >= 500. |
| < | Query logs where a field is less than a specified value, e.g., status < 400. |
| <= | Query logs where a field is less than or equal to a specified value, e.g., status <= 400. |
| = | Query logs where a field is equal to a specified value, e.g., status = 404. |
| in | To query logs where a field is within a specific range, use square brackets for closed intervals and parentheses for open intervals, separating two numbers with spaces inside the brackets. For example, status in \[500 600\] or status in \[200 400). |
| is null | Query logs where a field value is null or the field is missing. For example, error is null. |
| is not null | Query logs where a field value is not null. For example, error is not null. |
| is true | Query logs where a Boolean field value is true, e.g., success is true. |
| is false | Query logs where a Boolean field value is false, e.g., success is false. |
| is not true | Query logs where a Boolean field value is not true. Note: If the field content is null or the field does not exist in the log, the default field value is null, satisfying the condition of not being true. For example, success is not true. To query logs where the success field's value is not true, use success is not null and success is not true. |
| is not false | Query logs where a Boolean field value is not false. Note: If the field content is null or the field does not exist in the log, the default field value is null, satisfying the condition of not being false. For example, success is not false. |
| | | The pipe operator is used to separate search syntax from SQL syntax, indicating additional computations to perform based on the search. For example: match query1 \ |
