Retrieval and ask

search asks two retrievers at once and merges what they find.

Each ranked list is first filtered, and chunks are collapsed to messages (the best chunk stands for the message, and the other chunks that matched are kept as matched_chunks). A message then scores the sum of weight / (rrf_k + rank) over the lists it appears in: Reciprocal Rank Fusion. Only ranks are used, so a BM25 score and a cosine similarity never have to be compared. Every hit carries its score, a snippet and why: which retrievers found it.

Settings, under [search]:

Reading the query

No model is involved. The query is read into words, "phrases", -exclusions and operators, and every word is quoted before it reaches SQLite, so nothing a person types can be FTS5 syntax.

A value that cannot be read (priority:9, category:nope) is reported as a warning and filters nothing. Read state and the triage result live in raw.db, not in the index, so those filters are applied to the candidates there. The operation also takes the same filters as fields, plus a thread. A query with no words, such as is:unread from:alex, lists the newest matching messages.

When the vector side is unavailable

Search never fails because of it. If there is no embedder, no vector store, no vectors yet, the vectors were built by another model (so they would mean nothing), the call times out or it raises, the keyword results come back with degraded: true and a degraded_reason. A stale index is treated like a missing one rather than mixed in.

Search does not mark mail read

A result is a reference. Opening a message is what changes read state, and that is a separate, user-initiated operation.

ask(question, filters) retrieves, builds a context, and asks the model to answer in claims. Each claim has the excerpt it rests on and the exact words it quotes.

1. A question has its stop words removed for the keyword side, which then matches messages containing at least half of what is left; the vector side gets the question as written.
2. The context builder spends ai.ask_context_tokens (3,000 by default) in tiers, across all hits in rank order before going deeper into any: a header line and the stored summary for each hit; the matched chunks for the first ask_chunk_hits; the neighbouring chunks and the summaries of the rest of the thread for the first ask_neighbor_hits.
3. Near-identical chunks are sent once, and words a neighbouring chunk already shows are cut. The total is counted again at the end and trimmed, so the budget is a limit and not a target.
4. The mail goes to the model as UntrustedText, one block per message, behind the usual data rules (see AI safety).
5. Core keeps a claim only if its excerpt was in the context and its quote really is in that chunk's stored text (whitespace may differ, nothing else may). Claims that fail are dropped and counted. If any were dropped, confidence is low: a model that invented something is not trusted to rate itself. If none are left, the answer is insufficient_evidence, and the text says so instead of guessing.

The result has answer, citations (message, chunk ordinal, the quote as it stands in the message, and its offsets), confidence, insufficient_evidence, dropped_claims, degraded and the token figures: evidence_tokens and full_body_tokens, the cost of sending every hit whole, so the saving is visible on every call. When nothing is found there is no model call at all.

Mail from more than one account is never put in one prompt, because an account decides which workspace, and so which models, may see it. A question that finds mail in several accounts is refused with a request to name one. ask does not mark anything read.

ask is off over MCP by default. It would let an agent read mail by asking questions, so the user turns it on, like any operation (see Operations).

uv run python -m oaken.evals retrieval builds a seeded synthetic mailbox of about a thousand messages with planted facts and questions about them, ingests it through the real pipeline and scores each retriever. Evaluation results shows the latest run. The corpus has four kinds of answerable question and one kind that the mail does not answer, chosen so that keyword search and vector search each fail on one of them. That is why hybrid wins by a wide margin offline, and why the numbers say little about real mail.

Offline, the embedder and the reader are stand-ins: a small concept embedder and an extractive reader that quotes the sentence closest to the question. The run is deterministic, and a smaller run is a test, which asserts that hybrid is at least as good as each single retriever. A run with a real embedder and model (--live) is manual and is not part of CI.

No reranking model, no cross-account search, no search inside attachments, and no query rewriting unless llm_rewrite is on.