yasuki_core.search.parse_search.tokenize_query#

tokenize_query(query)[source]#

Tokenize search query into components.

Handles quoted strings, field:value pairs, and boolean operators.

Parameters:
querystr

Raw search query

Returns:
tokenslist of str

Query tokens

Examples

>>> tokenize_query('name:Doji type:personality')
['name:Doji', 'type:personality']

>>> tokenize_query('"Doji Hoturi" force>3')
['"Doji Hoturi"', 'force>3']