Meadowlark IQ
Cited, or refused.
Plain-English questions about the documents that govern, answered from the clause that governs, cited beneath the answer, or refused with the reason.
Status
- State
- Live
- Address
- meadowlarkiq.com
- Measured
- 99.9% correct on 1,503 templated, answerable questions, 19 September 2026
One answer, one refusal
Two results from the evaluation run of 19 September 2026 over the synthetic corpus; the answer is as returned, formatting removed, with [2] as its citation marker.
Answered
- Question
- What is the individual deductible for Heritage Advisors?
- Answer
- The individual (in-network) calendar-year deductible for members of the Heritage Advisors group is $4,600 [2].
- Source
- The run records that the controlling document was cited but not which clause. This is the line in it that states the figure, from its Schedule of Benefits (out-of-network column omitted):
Benefit In-Network — You Pay Calendar Year Deductible — Individual / Family $4,600 / $11,500
Refused
- Question
- What is the individual deductible for Driftwood Hospitality Group?
- Why
- IQ calls this an ambiguous record. The question names a group with two plans in effect, a PPO and an HDHP, so it matches two records, and IQ stops before retrieving anything rather than answer from one plan.
- IQ renders
- This question matches 2 different plan records, which carry different figures. Did you mean — pick a plan type to narrow to one record: PPO HDHP
Synthetic data · evaluation run of 19 September 2026
What it is
Meadowlark IQ answers plain-English questions about the documents that govern, with the clause the answer came from cited beneath it. When the documents do not support an answer, it says so and names the reason.
It is not a chat assistant. It does not summarize, draft or advise. It looks things up in documents that govern, and it would rather say the document does not say than fill the gap. That asymmetry is deliberate, and it is the property the whole design protects.
What it sits on top of
What's inside
- Ask
- A question is pinned to one record from the identifiers it contains, retrieved from that record only, and answered from cited clauses. Every figure in the answer must appear in a cited clause.
- Refuse, with the reason
- No evidence, an ambiguous record, conflicting authority, or a document not in force on the date asked. Each refusal says which, and what would get an answer.
- Read
- Every document whole, with the cited section highlighted in place, so a claim can be checked without leaving the question.
- Measure
- An evaluation harness runs the production path on a golden set of questions with known answers and reports accuracy, refusal accuracy and citation rate, stamped with the run that produced them.
What it refuses to do
- It does not summarize, draft or advise.
- It does not blend two documents into one answer.
- It does not return a figure that does not appear in the cited clause.
- It does not answer at all when it cannot show where the answer came from. The refusal is a first-class result, not an error.
What has been measured
Evaluation run, 19 September 2026: 1,503 answerable questions and 60 that should be refused. 99.9% of the answerable questions were answered correctly, each citing the controlling document; 57 of the 60 were refused end to end, and the other three said in words that no figure is stated. The questions are filled in from templates using the synthetic corpus's own records, and each answerable one asks for a single figure, an individual deductible or out-of-pocket maximum. The run shows IQ finding the right record and figure among near-duplicates, not how it handles questions a person wrote.
Model comparison, 7 August 2026: five models, 125 questions each. Every cheaper model refused more often, which a reader sees; one also stated two wrong figures, which a reader would act on, and the rest stated none.
Built and secured
Built
- The model is a setting, not a dependency. Every model in the stack is open-weight; the reranker already runs as a service the operator hosts, and the rest can move the same way. Swapping the model mostly moves cost and refusal rate; of the five compared, one also moved correctness.
- Measured on 7 August 2026: 36 cents per thousand questions on the production model, under two cents on a four-billion-parameter one.
- One TypeScript service and one web application served from the same process. One store for the corpus and its indexes; a separate, isolated store that holds nothing but sessions.
- 345 server tests and 56 web tests, counted on 1 October 2026. Lint, typecheck, tests, migrations and smoke checks run in CI on every push to main.
Secured
- Tokens are matched by their SHA-256 hash, and sessions are stored only as the hash of the cookie, so a copy of the session store signs no one in. Neither secret is logged. Sessions are re-verified on every request.
- Sign-in is throttled to five attempts per fifteen minutes per client. A failed attempt records only the client address and the reason, never what was typed.
- Admin is a per-token flag. If no token carries it, the admin routes fail closed.
- Nothing about the visitor leaves the service. Only clause text and the question go to the models. Every model call is logged with its tokens, latency, cost and actor, and exported for monitoring.
Because the pipeline does the deciding, the generation model does not have to be expensive. That is the whole reason IQ can afford to refuse.