INILLUCENT
An embedded retrieval engine that filters inside the walk, takes the exact plan when the exact plan is faster, and can decline to answer.
A Rust library that runs inside your own process. It holds 185,078 chunks of a graded corpus in 1.86 GB, answers an unfiltered search in 0.70 ms, and returns every row a filter admits rather than as many as a scan happened to collect.
- 185,078chunks
- 38,847 documents, 768 dimensions
- 0.70ms p50
- unfiltered search, in process
- 1.86GB
- resident, serving the whole corpus
- 0processes
- no server, no socket, no port
The stack it replaces is also the stack it is graded against
PostgreSQL, the pgvector extension, and llama.cpp serving an embedding model over HTTP. That combination is the sensible default. Four things about it are structural rather than a matter of tuning, and each one is a place a purpose-built engine wins.
The filter runs after the search, not during it
pgvector evaluates a WHERE clause once the index scan has already chosen its candidates. A plain HNSW scan produces only ef_search of them, so a search restricted to a minority source can be left with almost none. Iterative scan fixes that by running the scan again until enough rows pass, and it costs latency: a filtered search that took a few milliseconds takes tens.
An exhaustive scan is often right, and it will not choose one
When a filter admits 7,000 chunks of 186,000, comparing the query against all 7,000 is both exactly correct and faster than walking a graph over the whole corpus. Inillucent counts what the filter admits, compares that against a measured crossover, and takes the exact plan. Narrow filters become the case where accuracy is perfect rather than the case where it collapses.
The embedding model is a separate process on a socket
llama.cpp runs the model in its own program. Every query pays a process boundary and an HTTP hop before any searching happens, and the deployment has two things to keep alive instead of one. Inillucent runs nomic-embed-text-v1.5 through the ONNX runtime in the same process, at full precision.
You are paying for durability you are not using
A retrieval index is derived data, rebuilt from the source documents. Write-ahead logging, multiversion concurrency control, a cost-based planner and a wire protocol are all cost with no return on this workload. Once the index fits in memory — and 186,000 chunks fit in 1.86 GB — everything PostgreSQL does to survive a power cut is overhead.
product-overview.md, README.md
It walks through the nodes it will not admit
A node that fails the predicate is still expanded, so the traversal can pass through it to reach the region behind it — but it is never admitted to the results. The walk therefore continues until it has collected enough passing chunks, at the cost of a longer walk rather than a repeated scan.
Pick a predicate. The plan below is the one the engine chooses for it, and the numbers are the ones it was graded on.
With no predicate there is nothing to be exact about: 0.9252 is recall@10 against exhaustive cosine over the whole corpus, which is what the graph gives up for its speed.
inillucent-scorecard.md — Filtered vector search, per source
Seventeen comparisons, each with its uncertainty attached
Every family declares one primary measurement and only that one is judged; the rest are diagnostics, because nDCG, success@1, success@10 and reciprocal rank all move together when one behaviour changes, and counting each of them turns one result into four. Each comparison is decided by a 95% paired bootstrap interval and a paired randomization test against a practical threshold declared before the run.
15 better, 1 equivalent, 1 inconclusive, 0 worse. Correctness gates: all pass.
delta, oriented so positive is better whatever the metric’s own direction
Latency, reported apart because it is not paired
- no predicate0.704 msagainst1.519 ms
- source = slack0.864 msagainst1.393 ms
inillucent-scorecard.md — every primary comparison, with its uncertainty
It can say no
A question nothing in the corpus answers is the failure that does not announce itself. Ten confident-looking passages about nothing look exactly like ten good ones, and an agent writes a paragraph out of them. Measuring it needs an absolute notion of confidence, and per-list normalization destroys one by construction: it maps the best hit of every list to exactly 1.0, whether the list is good or hopeless.
So the engine stopped asking one number to do two jobs. Every hit carries a score, from whichever fusion ranks best, and a confidence, always computed against absolute bounds — cosine over normalized vectors is bounded by one, and BM25 by the query's own idf mass at saturation. The abstention threshold is set on confidence; the ranking is decided by score.
confident answer rate on 200 questions with no answer
Given a question nothing in the corpus answers, the baseline returns a confident top result every single time. Inillucent does it on one query in two hundred.
Three questions this corpus answers and three it does not. The instrument does not know which is which until it looks.
Nothing asked yet. The instrument does not know which of these the corpus holds until it looks.
The unanswerable questions are built the way the graded family builds them: distinctive words from documents in sources the corpus draws from disjoint pools, so nothing can be relevant. The readouts are illustrative of the behaviour the 0.0050 rate measures — the rate itself is the measurement.
inillucent-scorecard.md — Abstention on questions nothing answers
Meaning finds the passage. Words find the identifier.
A user who types PROJ-1932 or parse_headers wants that identifier, not passages about vaguely similar ones. Meaning-based search is bad at this and word-based search is good at it, so the engine does both and fuses the results.
PostgreSQL full-text search has two properties BM25 lacks, and on a corpus this size both matter. to_tsquery joins query terms with &, so a chunk missing one word never appears at all; and ts_rank_cd is cover-density ranking, so a chunk whose terms sit close together outranks one that mentions the same words in different paragraphs. Scoring any term with BM25 finds far more of the right chunks and puts them lower.
Inillucent keeps the recall and takes both properties as gradients rather than gates.
- lexical_coverage
- 3.0
- Scales a score by the share of the query's idf mass the chunk holds, raised to this exponent.
- lexical_proximity
- 1.0
- Scales it by matched terms over the smallest window holding one of each, blended by this weight.
- lexical_phrase
- 0.75
- Scales it by whether the matched terms arrived in the query's own order inside that window.
- lexical_tier
- off
- Rank by how many query terms a chunk holds first and score second — the ordering & gives PostgreSQL.
- lexical_prefix
- off
- Let a query term match the terms it prefixes, the way :* does.
lexical_phrase is the one ts_rank_cd has no answer to. Cover density asks how tightly the terms sit; it does not ask whether they came in the order the question asked them in, and "offer eligibility rules" and "rules for eligibility of an offer" have the same window width and are not the same answer.
Every one of them is measured rather than assumed, and every one has an off switch that restores plain BM25. Two are off by default because the measurement said so.
inillucent-scorecard.md — Lexical retrieval; README.md — the lexical settings
It is running on a real mailbox today
Nikaya answers questions over a Gmail corpus on this machine. It ran PostgreSQL full-text search over a weighted tsvector and pgvector HNSW over halfvec(768); it now runs Inillucent, over 598,560 chunks across 64,172 email and 2,287 attachment documents. Both engines were given identical vectors, so nothing below is attributable to the embedder.
- 57% → 0%natural-language questions that retrieved nothing at all
17 of 30 real questions returned zero rows through websearch_to_tsquery. "car registration renewal" returned 0; "registration renewal" returned 829. The gate was the failure, not the corpus.
- 3,167 MBof index deleted from a 5,849 MB database
chunk_embedding at 2,736 MB and chunk_search_vector_idx at 431 MB both go away, replaced by one in-process index.
- 18×faster on a novel semantic query
80.6 ms p50 to 4.41 ms, and the cold-cache cliff — a p95 of 275 ms — disappears rather than shrinking.
| query | pgvector p50 | pgvector p95 | Inillucent p50 | Inillucent p95 |
|---|---|---|---|---|
| semantic, novel query, cold page cache | 80.60 ms | 275.20 ms | 4.41 ms | 10.29 ms |
| semantic, warm | 33.70 ms | 66.80 ms | 4.41 ms | 10.29 ms |
| exact top-100 over the whole corpus | 330.00 ms | — | 109.11 ms | 165.82 ms |
Inillucent has one latency figure rather than two because the index is resident in the process. The pgvector figures split because a 1,486 MB HNSW index was being served out of a 128 MB shared_buffers; twelve earlier probes measured a novel query as high as 405 ms.
What it costs to hold 598,560 chunks
- resident, one index over 598,560 chunks
- 3.80 GB
- on disk
- 2.40 GB
- graph
- 19,788,020 edges, 5 layers
- lexical index
- 1,696,619 terms, 32,436,680 postings
- build, single-threaded
- 9 min 27 s
- reopen a saved index
- 26.6 s
One place it is slower: the lexical branch, 11.54 ms p50 against PostgreSQL's 2.5 ms. That is not a defect being hidden — PostgreSQL's branch is fast on those queries because on 17 of the 30 it is returning nothing at all. Inillucent returns 100 rows on every one of them, and rescores the leaders for proximity and phrase order on top.
the notebook TDD "Nikaya on inillucent", measured on the real corpus
And a file SQLite opens, reads, and keeps writing to
A second, separate engine in the same repository targets SQLite's file format and its observable behaviour. It links no database engine and no SQL parser — a test walks every crate manifest and fails on a dependency that would break the rule. SQLite appears in exactly one form: a pinned 3.53.4 build, compiled from the official amalgamation, run as a child process and compared against as a black-box oracle.
A row reaches pass only when a recorded run passed every test it cites, on both platforms. The manifest carries one row per capability owed, including the ones nothing has been written for yet — that is the denominator on purpose, because a capability with no row cannot be reported as owed. The 7 that are missing are all in "optional surfaces this release does not implement".
- every join form, including RIGHT and FULL
- ordinary and recursive CTEs
- window functions, all three frame units, all four EXCLUDE forms
- subqueries in every position, compound selects, views
- STRICT tables and generated columns, VIRTUAL and STORED
- transactions, savepoints, and a rollback journal that survives a power cut at every cut point
- a costed planner that reorders joins on what ANALYZE measured
- foreign keys, ATTACH, and multi-database commit
- WAL and concurrent connection semantics
- FTS5, R-Tree, the JSON built-ins, PRAGMAs, the C ABI and a CLI
CREATE TABLE people(id INTEGER PRIMARY KEY, name TEXT UNIQUE,
score REAL CHECK (score >= 0));
INSERT INTO people(name, score) VALUES('ada', 9.5) RETURNING id;
BEGIN; UPDATE people SET score = score + 1;
SAVEPOINT s; DELETE FROM people; ROLLBACK TO s; COMMIT;
WITH RECURSIVE n(i) AS (SELECT 1 UNION ALL SELECT i + 1 FROM n WHERE i < 10)
SELECT sum(i) FROM n;
SELECT name, rank() OVER (PARTITION BY team ORDER BY score DESC) FROM people;
EXPLAIN QUERY PLAN SELECT * FROM people WHERE score > 5;This is a compatibility claim and nothing else. The relational engine's performance qualification is open: measured against the same pinned SQLite over 30 paired rounds, it currently runs at 0.240× at 5,000 rows and 0.192× at 100,000, against a release bound of 1.50×. Read that as: correct, evidenced, and not yet fast. It is on the board, and this page will carry the number whatever it turns out to be.
compat/compat-report.md and compat/release/scorecard.md
How a measurement becomes a verdict
The card used to count measurements won, with anything above 1e-4 a win. All three parts of that were wrong in the same direction: 1e-4 is a hundredth of what one query in ninety changing its mind moves a mean by, so noise was being counted; every row got a vote, so four metrics turned one behaviour into four wins; and rows returned was scored higher-is-better, so fifty irrelevant chunks beat ten useful ones.
One primary metric per family
Everything else is a diagnostic: printed, argued about, never voted on.
Paired statistics, seeded
A 95% paired bootstrap interval and a paired randomization test over the per-query scores. Both are reported because they answer different questions — the interval says how large the difference is, the p-value says whether it could be noise.
A practical threshold declared before the run
0.01 on the ranking measures, five per cent on latency. With enough queries every difference eventually becomes detectable, including differences far too small to matter.
Four verdicts, not three
better when the interval clears both zero and the threshold, equivalent when the whole interval sits inside it, worse in the other direction, and inconclusive when the run cannot tell. A run that cannot separate two engines says so.
Completeness is a gate, not a score
Returning thirty rows where fifty exist is a defect. It is just never a relevance win.
Every run leaves its evidence behind
An aggregate card can be read but not interrogated. Beside the card, every run writes a manifest — commit and dirty flag, corpus file and size, model, device, every query seed, every ranking setting, host, thresholds — and one JSONL line per engine per query holding the ranking, each hit's relevance grade, the component scores, the latency and the metrics that query contributed. That is what makes the intervals recomputable without repaying the run, and lets a miss be looked at rather than guessed at.
runs/<unix time>-<commit>/
manifest.json
per-query.jsonlWhat keeps the comparison fair
- Both engines read byte-identical vectors. Every chunk is embedded once and both are loaded with the same numbers; every query is embedded once and handed to both. A score difference cannot come from the embedding model.
- The baseline gets the right setting for each row rather than one setting for both: iterative scan on for a filtered search, off for an unfiltered one, ef_search 400 and 100, scan_mem_multiplier 4 — the one most easily missed, and missing it produces a baseline that looks tuned and is not.
- Anything that is a ranking policy rather than a retrieval capability is given to both engines.
- The corpus is assembled from public data by the repository itself, and every figure on the card can be reproduced by anyone with the repository, an internet connection and a few hours.
- The one equivalent is a source where both engines reach recall 1.000 within the filter and neither can do better. "Both engines are at the ceiling" is reported separately from "we cannot tell", because those are not the same statement.
- The one inconclusive is confluence, where Inillucent leads 0.9960 to 0.9760 and the interval runs 0.0000 to 0.0440 on 25 queries. The run declines to call that a win.
- Every comparison is against the better of two pgvector configurations, never the extension defaults. Beating a misconfiguration proves nothing.
inillucent-scorecard.md and README.md — How it is graded
What it does not do
Stated here rather than discovered later. None of these is a defect being managed; they are the shape of the thing.
- It holds the index in memory
- That suits a corpus that fits in memory, and 186,000 chunks fit in 1.86 GB. A corpus far larger than available memory needs a different design.
- Adding content rebuilds the graph
- A full build over 186,829 chunks is 175 seconds on one core, and 9 minutes 27 seconds over Nikaya's 598,560. The insert loop is single-threaded and every distance computation in it is independent, so the published parallel-HNSW approach applies directly and has not been taken yet.
- The relational engine is correct, not fast
- Compatibility is evidenced on two platforms. The performance qualification is a loss today and is open work.
- One corpus and one embedding model
- The harness works against any corpus; these figures describe this one. Where the correct answer is a document's own title, titles share vocabulary with the text beneath them, which flatters word-based search — equally for both engines, so the comparison holds, but the absolute figures are optimistic.
- The lexical baseline is measured with AND
- That is what to_tsquery does by default. Joining the terms with OR would return more rows and has not been measured, so read the identifier figures as the default rather than the ceiling.
- Only searching was measured
- Ingest, update and delete paths are not on this card.
product-overview.md — What these numbers do not cover
A library, not a deployment
One dependency in a Cargo manifest. An index is a directory of four files; reopening one takes 5.3 seconds against 175 to build it, so a worker, a command-line tool or a request handler can hold a real index without paying for a build. There is nothing to install, no port to configure, no process to supervise, and nothing between your application and its index.
let mut index = Index::new(IndexConfig {
dims: 768, quantized: true, ..Default::default()
});
index.add(chunks, &vectors); // one vector per chunk
index.commit(); // graph, lexical index, int8 codes
let compiled = index.compile(&Filter::source("slack"));
let hits = index.hybrid_search(
"how does the release process work", &q, &compiled, 10, None);Compression is free; shortening the embedding is not
| configuration | bytes per embedding | accuracy |
|---|---|---|
| full, uncompressed | 3,072 | 0.995 |
| one byte per number | 772 | 0.995 |
| shortened to 512 numbers | 516 | 0.770 |
| shortened to 256 numbers | 260 | 0.635 |
| shortened to 64 numbers | 68 | 0.345 |
A quarter of the memory at identical accuracy. Shortening the embedding is expensive on this corpus and is not recommended, which is worth knowing because nomic-embed-text-v1.5 advertises the capability and the cost is not obvious until it is measured.