- Docs
- Guides
- Build with Enigma
- Python SDK
Customize query generation
Pick the LLM provider, write your own provider or strategy, control the query shape cache, and read the token usage of each call.
The LLM providers snippet shows only what changes from the quickstart programs. Run the snippets in this guide explains how to run it.
LLM providers
Section titled “LLM providers”TemplateStrategy and GenerativeStrategy call Anthropic’s API by default, with anthropic_api_key= or the ANTHROPIC_API_KEY environment variable; their model= argument applies only to that default. Pass llm_provider= to use another provider:
from enigma_api_client.strategies import ( AnthropicProvider, BedrockProvider, GenerativeStrategy, OpenRouterProvider, TemplateStrategy,)
# Anthropic's API (ANTHROPIC_API_KEY)TemplateStrategy(llm_provider=AnthropicProvider(model="claude-haiku-4-5-20251001"))
# OpenRouter (OPENROUTER_API_KEY). provider_routing is sent as OpenRouter's "provider"# object and reasoning as its "reasoning" object; omit either to keep OpenRouter's default.GenerativeStrategy(llm_provider=OpenRouterProvider( model="anthropic/claude-haiku-4.5", provider_routing={"order": ["anthropic"], "allow_fallbacks": False}, reasoning={"enabled": False},))
# Claude on Amazon Bedrock (pip install "enigma-api-client[bedrock]"), signed with your AWS credentials.# The default model is a US inference profile; pass model= outside US regions.TemplateStrategy(llm_provider=BedrockProvider(region="us-east-1"))Write your own LLM provider
Section titled “Write your own LLM provider”To use another LLM, implement the LlmProvider protocol. Import LlmProvider, Completion, and ProviderHTTPError from enigma_api_client.strategies.
- Implement
async def complete(system, user, max_tokens) -> Completion. - Return
Completion(text, truncated), wheretruncatedis true when the model hit its output limit. - Pass the call’s token counts to
Completionasinput_tokens=,output_tokens=,cache_read_tokens=, andcache_write_tokens=, so that Token usage includes them. Leave cache reads and writes out ofinput_tokens; they have their own fields. - If your provider reports which host served the call, pass it as
served_by=. - If your provider reports what it billed for the call, pass the amount in USD as a float,
cost_usd=. - Raise these exceptions for failures:
| Raise | Reported as |
|---|---|
ProviderHTTPError(status_code, message) with 408, 429, or 5xx |
TRANSIENT_PROVIDER_ERROR |
ProviderHTTPError with any other HTTP status |
PROVIDER_ERROR |
ProviderHTTPError with a value that isn’t an HTTP status |
TRANSIENT_PROVIDER_ERROR |
ValueError, for a missing key or bad configuration |
GENERATE_QUERY_CALLBACK_NOT_CONFIGURED |
RuntimeError, for any other provider failure |
TRANSIENT_PROVIDER_ERROR |
Any other exception propagates unchanged.
Query shape cache
Section titled “Query shape cache”TemplateStrategy and GenerativeStrategy keep an in-memory cache of query shapes learned from successful generations:
- A shape is a generated query with its names and places left as blanks.
- Once a shape has been confirmed across several different entities, a matching question is answered from the cache without an LLM call.
- The cache is on by default and is shared by every caller of the same strategy instance at the same tier.
- To turn it off, pass
shape_cache=False, or setENIGMA_SHAPE_CACHE_ENABLED=0before you create the strategy.
Token usage
Section titled “Token usage”After client.search(prompt=...) or generate_query returns or raises, enigma_api_client.strategies.GENERATION_USAGE.get() returns that call’s LLM token counts as an LlmUsage:
| Field | Holds |
|---|---|
completions |
The number of LLM completions |
input_tokens, cache_read_tokens, cache_write_tokens, output_tokens |
The token counts |
unmetered_completions |
The number of completions whose provider left out the input or the output token count, so the sums miss their tokens |
served_by |
The host each completion named as having served it, in call order, for providers that report one, such as OpenRouterProvider |
cost_usd |
The amount in USD billed for the completions whose provider reported one, such as OpenRouterProvider; completions from other providers add nothing |
- Read it in the same task that awaited the call.
asyncio.gatherruns each call in a task of its own, so read the usage inside the coroutine you pass to it. ServiceStrategyreports zero usage, andAgentStrategyreports its run’s token totals and nocost_usd.
Write your own strategy
Section titled “Write your own strategy”To write your own strategy:
- Subclass
QueryStrategy. - Implement
generate_queryandaclose. - At the start of
generate_query, setGENERATION_USAGEto a freshLlmUsage(), so Token usage reports this call, not an earlier one.
This strategy treats the whole question as a brand name:
import asyncioimport os
from enigma_api_client import DEFAULT_TIER, Client, ClientConfig, QueryStrategyfrom enigma_api_client.strategies import GENERATION_USAGE, LlmUsagefrom enigma_api_client.strategies.base import GenerateQueryResponse
QUERY = """query Brand($name: String!) { search(searchInput: { name: $name, entityType: BRAND, conditions: { limit: 1 } }) { ... on Brand { names(first: 1) { edges { node { name } } } } }}"""
class BrandNameStrategy(QueryStrategy): async def generate_query(self, prompt, *, tier=DEFAULT_TIER, prior_attempt=None): GENERATION_USAGE.set(LlmUsage()) return GenerateQueryResponse(QUERY, {"name": prompt})
async def aclose(self): pass
async def main(): config = ClientConfig(enigma_api_key=os.environ["ENIGMA_API_KEY"], query_strategy=BrandNameStrategy()) async with Client(config) as client: response = await client.search(prompt="Chipotle") print(response.data["search"][0]["names"]["edges"][0]["node"]["name"]) # CHIPOTLE MEXICAN GRILL
asyncio.run(main())