Build with
Inillucent

Embedded means there is no database server to install or keep running. Your program opens a file, sends SQL to a library in the same process, and gets typed rows back. You can use the same engine from a terminal with inillucent-shell.

The SQL dialect follows SQLite 3.53.4. If you have used SQLite, most statements will look familiar. If SQL is new to you, read the next four chapters and type each example in order.

The examples in these chapters build one database, one chapter after another. Every result shown was printed by the engine when the examples ran in that order.

• Tables are stored in B+trees. A reader never sees half of a transaction. Every change goes to a write ahead log before it reaches the file.
• A VECTOR(N) column and an inillucent_hnsw index give you nearest neighbor search in ordinary SQL.
• The inillucent_search table combines keyword search and vector search. It reports a confidence value, so your application can decide to return no answer.
• A feature the engine has not built fails with the status unsupported. You get an error, never an approximate answer.

The install commands are in the first example below. Each script downloads the archive for your machine, checks its SHA-256 against the published SHA256SUMS file, and puts the programs on your PATH. Neither script needs administrator rights, and neither writes outside your home folder.

macOS gets one universal archive that runs on Apple silicon and on Intel. Linux gets archives for x86-64 and ARM. The downloads page also has a signed macOS installer package and .deb and .rpm packages. Homebrew, npm, pip, Go, Composer and cargo can install the programs too. https://inillucent.com/install.md lists every way to install.

You get four programs. inillucent is the command line, with 35 commands. inillucent-shell is a shell that works like sqlite3. inillucent-mcp serves a database to an AI agent, with 30 of the 35 commands as MCP tools. inillucent-migrate builds a database from a legacy retrieval index.

The .rdb suffix is a convention. Any file name works, and the path can be relative or absolute. Keep the file on a local disk. Several processes can open the same file. PRAGMA locking_mode is normal by default, which lets processes take turns on the file. PRAGMA locking_mode = exclusive keeps the file for one process.

A line that starts with a dot is a shell command. The shell handles it and does not send it to the SQL engine. Every other line is SQL. While you learn, turn on .headers on and .mode table, because column names make a result easier to check.

The shell answers 63 of SQLite's 65 dot commands. The two it does not have are .expert and .session. A SQLite .dump of a database with an AUTOINCREMENT table or an FTS5 table loads into inillucent-shell, and the FTS5 table arrives with its rows and answers MATCH in the same session. The shell prints the rows of a statement as the engine produces them in the list, csv, tabs, quote, ascii, tcl, c, jatom, line, insert, json, jobject, html and count modes. The modes that size each column to the widest value, which are column, table, box, markdown, qbox, psql and split, hold the whole result. inillucent query keeps the rows it prints, up to --limit, and counts the rest, so total is exact and memory stays small. --limit 0 holds the result. A statement that fails partway prints the rows produced before the error, then the error. .dump and inillucent dump work on a table name that has characters of several bytes.

The -bail option stops a script at the first error. Use -bail in scripts. Without -bail, a script that fails at a CREATE TABLE keeps going, and each later error points at the wrong cause.

• .help lists the shell commands.
• .tables lists the tables and views in the open database.
• .schema prints the CREATE statement stored for each object.
• .mode sets the output format for every result after it.
• .parameter sets values that later statements can bind.
• .quit closes the database and leaves the shell.

NULL means missing or unknown. NULL is different from 0 and from an empty string. Test for NULL with IS NULL and IS NOT NULL. The comparison NULL = NULL gives NULL, and a WHERE clause treats NULL as false.

Ordinary tables use SQLite type affinity. Affinity converts a value to the column type when the conversion loses nothing. For example, the text 42 becomes the integer 42 in an INTEGER column. A STRICT table applies affinity first, then rejects any value that is still the wrong type.

• INTEGER holds a signed whole number of up to 64 bits.
• REAL holds a 64 bit floating point number.
• TEXT holds a string of UTF-8 bytes.
• BLOB holds bytes exactly as you gave them.
• VECTOR(N) holds exactly N 32 bit floating point numbers, stored little endian.
• ANY, in a STRICT table, accepts every type of value.

A column declared INTEGER PRIMARY KEY is the row's key. When an INSERT leaves the key out, the engine picks the next number. AUTOINCREMENT stops the engine from reusing the key of a deleted row. WITHOUT ROWID stores the table by its declared primary key. Use WITHOUT ROWID when the natural key is not an integer.

Foreign keys are checked only when PRAGMA foreign_keys is ON. The default is OFF, as in SQLite. A foreign key check runs at once, or at the commit when the key is declared DEFERRABLE INITIALLY DEFERRED. ON UPDATE and ON DELETE accept CASCADE, SET NULL, SET DEFAULT, RESTRICT and NO ACTION.

• NOT NULL requires a value in every row.
• UNIQUE stops two rows from having the same values in the listed columns.
• CHECK requires its expression to be true or NULL.
• DEFAULT gives the value to use when an INSERT leaves the column out.
• STRICT rejects a value that cannot be converted to the declared type.
• Generated columns, composite keys and named constraints use SQLite syntax.
• A row can have 2,000 integer columns on a 32 KiB page when the values are small. Once the integers of a row would take more than half a page, each is written in the fewest bytes that hold it, which fits about 3,200 columns of small numbers. A row of integers that need eight bytes each holds about 1,925 columns. The first such wide row moves the file to format 4.

The WHERE clause decides which rows UPDATE and DELETE change. With no WHERE clause, they change every row in the table. Before you run a new UPDATE or DELETE, run a SELECT with the same WHERE clause and check the rows it returns.

ON CONFLICT decides what happens when an INSERT would repeat a unique value. DO NOTHING skips the new row. DO UPDATE changes the row that is already there. INSERT OR IGNORE and INSERT OR REPLACE work as they do in SQLite. The ON CONFLICT target can be a column list, a partial unique index (its WHERE must match the index) or an expression such as ON CONFLICT(lower(name)). DO UPDATE runs the CHECK, NOT NULL and STRICT checks and fires the UPDATE triggers of the table.

DELETE and UPDATE also accept ORDER BY and LIMIT. A loop of DELETE ... LIMIT 1000 removes many rows in small transactions. SQLite accepts this form only when it is compiled with SQLITE_ENABLE_UPDATE_DELETE_LIMIT.

INSERT ... SELECT reads all of its source before it writes a row. Source rows past 32 MiB go to a temporary file as they arrive and are read back in order, so a copy does not need memory for every row. If a row fails, every earlier row of the statement is put back.

A SELECT describes the answer you want. It does not say how to find the answer. The query planner decides whether to scan the table, search an index, or sort in a temporary table. Chapter 11 shows how to see the plan the planner picked.

LIKE matches text patterns. In a LIKE pattern, % matches any run of characters and _ matches one character. GLOB uses * and ? the way a Unix shell does. COLLATE NOCASE compares and sorts ASCII letters without regard to case. COLLATE RTRIM ignores spaces at the end of a string.

• DISTINCT removes duplicate rows from the result.
• LIMIT n OFFSET m skips m rows and returns the next n. LIMIT and OFFSET take any expression that reads no row, such as LIMIT 1 + 1 or a bound parameter. A value that is not an integer, such as 2.7 or NULL, is refused with datatype mismatch, as SQLite refuses it.
• CASE picks a value based on conditions.
• CAST converts a value to the type you name.
• IN tests a value against a list. BETWEEN tests a value against a range.
• A parameter written ?1 or $1 binds the first value, ?2 or $2 the second. $N binds by its number, as PostgreSQL does, whatever order the placeholders appear in.
• IS and IS NOT compare two values and treat two NULLs as equal.

INNER JOIN returns the pairs of rows that match. LEFT JOIN also keeps the rows of the left table that have no match. RIGHT JOIN keeps the unmatched rows of the right table. FULL OUTER JOIN keeps the unmatched rows of both. CROSS JOIN returns every possible pair, so use CROSS JOIN only on small tables.

A subquery can appear in WHERE, in FROM, or in place of a value. A correlated subquery refers to the current row of the outer query. WITH RECURSIVE runs a query again on its own output until the query adds no new rows. Use WITH RECURSIVE to walk a tree of rows or to generate a sequence.

A row value such as (a, b) on the left of IN takes a list of values or a subquery, as in (a, b) IN (SELECT x, y FROM s). A window function inside a subquery in FROM, a CTE or a view runs, so a rank computed in an inner query can be filtered in an outer one.

When a join or a correlated subquery compares a column that has no index, the engine reads the other table once into a hash table and looks up each outer row in it. This applies to a correlated EXISTS, NOT EXISTS or scalar subquery, and to a join whose ON clause has conditions beside the equality, such as ON a.k = b.k AND a.id < b.id, or an expression on one side of the equality, such as ON a.k = b.k + 1. The hash join tests the rest of the condition for each pair it finds, so LEFT, RIGHT and FULL joins keep the rows that nothing matched. EXPLAIN QUERY PLAN shows the automatic index. PRAGMA automatic_index = OFF turns the hash table off, and the engine then scans the table for each outer row. A subquery inside UPDATE, DELETE or INSERT is always scanned. A correlated subquery whose one table is a view or a derived table that reads nothing from the outer row, such as a view with a GROUP BY, runs the query of the view once and looks up each outer row in the result. A comparison against an expression of the outer row, such as agg.s = o.v * 5, runs the query of the view for each outer row.

GROUP BY makes one result row for each group. HAVING filters the groups after the aggregates are computed. The aggregate functions include count, sum, total, avg, min, max, group_concat, string_agg, median, the percentile functions and the JSON group functions.

UNION combines two results and removes duplicate rows. UNION ALL keeps the duplicates. INTERSECT keeps the rows found in both results. EXCEPT keeps the rows of the first result that are not in the second.

A window function needs an OVER clause. OVER accepts PARTITION BY, ORDER BY, the ROWS, RANGE and GROUPS frames, and EXCLUDE. The eleven window functions are row_number, rank, dense_rank, percent_rank, cume_dist, ntile, lag, lead, first_value, last_value and nth_value. Each one gives the same answers as SQLite 3.53.4. A window function computes one partition at a time, so memory holds the largest partition and not the table. A partition larger than 32 MiB is computed a run of peer groups at a time when every call is row_number, rank, dense_rank or an aggregate over the default frame. A window with another call, such as ntile or lead, or a frame with following rows, holds one partition whole. DISTINCT, UNION, INTERSECT and EXCEPT keep their rows in a hash table while the keys and rows fit in 32 MiB, and sort them in runs in a temporary file past that. The rows and their order are the same either way. A window query with a LIMIT stops reading once the limit is met.

A savepoint marks a point inside a transaction. ROLLBACK TO undoes the changes made after the savepoint and keeps the transaction open. RELEASE removes the savepoint and keeps its changes. ROLLBACK TO also ends every savepoint opened after its target, so ROLLBACK TO s0 leaves only the savepoints opened before s0.

ROLLBACK restores everything the transaction changed, including after DROP TABLE and ALTER TABLE. With foreign keys on, BEGIN; DROP TABLE p; ROLLBACK brings p back with its rows. An INSERT, UPDATE or DELETE followed by ALTER TABLE ADD COLUMN or DROP COLUMN in one transaction is undone by ROLLBACK as well. A rolled back DROP of an FTS5 table or of a table with a vector index leaves the table usable, a rolled back DROP TABLE keeps the AUTOINCREMENT counter, and schema_version goes back too.

One writer works at a time. A second writer waits for up to PRAGMA busy_timeout milliseconds, 5000 by default, and then fails with busy. A reader in another process does not wait for a write transaction: it reads the last committed state, and never sees half of a transaction. Opening a file another process is writing waits too; set INILLUCENT_BUSY_TIMEOUT in milliseconds, or pass the wait to the open, to change how long. PRAGMA synchronous sets how often the engine waits for the disk: OFF, NORMAL or FULL. FULL is the default.

BEGIN and BEGIN DEFERRED take no lock, as in SQLite. The first statement inside the transaction takes the lock it needs, so a read transaction reads beside another process that has a write transaction open. BEGIN IMMEDIATE and BEGIN EXCLUSIVE take the write lock at once. A transaction that reads first and then writes fails with busy when another process committed after its first read. Roll it back and run it again, or start it with BEGIN IMMEDIATE.

A process that waits for a busy lock tries again every 2 or 3 milliseconds once it has waited 15 milliseconds. BEGIN IMMEDIATE from many processes at once is fairer, because a process that just committed no longer takes the lock again ahead of a process that has waited for seconds.

A connection can always end its transaction. ROLLBACK, a COMMIT of a transaction that wrote nothing, and pragmas that set only the connection, such as busy_timeout, take no lock, so they do not fail with busy while another connection writes the file. A transaction that has read and written nothing goes on after another connection runs VACUUM.

A write transaction does not need memory for every row it changes. The transaction keeps each changed row as it was before, so that a rollback can put it back. Those before images stay in memory until the transaction holds 16 MiB of them. Past that, the older images go to a temporary file next to the database, which only ROLLBACK and ROLLBACK TO read. A commit discards the file unread, and the operating system deletes it when the connection closes it. A transaction can therefore change more rows than fit in memory, and PRAGMA cache_size is not what limits it. VACUUM reads an ordinary table a page of rows at a time. CREATE INDEX and REINDEX hold at most 32 MiB of index entries in memory, write sorted runs to a temporary file past that, and merge the runs into the index. A UNIQUE index is checked before any page is written, so a failure leaves nothing behind. The shell command .import reads a block of the file at a time and inserts at most 20,000 rows in one statement.

• Deferred foreign keys are checked at COMMIT.
• A statement that fails keeps none of its rows, including one whose trigger broke a foreign key. Under OR FAIL it keeps the rows it wrote before the failure, as in SQLite.
• Full text indexes and vector indexes commit and roll back with the rows they index.
• A transaction that writes two attached files commits both files or neither. A super journal records which way it went.
• When a database opens, recovery replays every complete record in the log. Running recovery twice gives the same result as running it once.

CREATE INDEX builds a B+tree over one or more columns. Partial indexes, indexes on expressions and indexes on WITHOUT ROWID tables all work. EXPLAIN QUERY PLAN shows how a query will run: a table scan, an index search, a vector index search, or a sort in a temporary B-tree. A WHERE that joins conditions with OR uses an index for each condition when every condition has one, and the plan shows MULTI-INDEX OR.

An indexed value or a primary key value can be up to 1,048,576 bytes long. A key longer than one eighth of the page size, 4,096 bytes at the default page size, is stored on pages of its own like any other large value. The index leaf keeps the first 512 bytes of the key, so a search compares that prefix and reads the rest only when the prefix is equal. A longer key fails with a message that names the limit. Before release 2.5.0 the limit was about 32 KB.

CREATE TRIGGER supports BEFORE, AFTER and INSTEAD OF, FOR EACH ROW, WHEN, the OLD and NEW rows, and RAISE. ALTER TABLE can rename a table, rename a column, add a column, drop a column, and, as in SQLite 3.53, change a column with SET NOT NULL or DROP NOT NULL and add or drop a CHECK constraint. A rename rewrites the foreign keys, views and triggers that name the table or column, and fails with SQLite message error in view v when a dependent view or trigger would break. A rename rewrites only what it renames: RENAME TO leaves a CTE that has the table name alone, and RENAME COLUMN leaves an ORDER BY that names a result alias alone, so a view keeps its row order. RENAME of a table with a vector index carries the index, and RENAME of an FTS5 or inillucent_search table renames its shadow tables. A column added with ADD COLUMN c AS (c) fails with generated column loop on c, and the expression of a generated column added this way is checked as CREATE TABLE checks one. ADD COLUMN with a literal default takes the same time whatever the number of rows, because it writes the catalog row and no leaf. A leaf written before the column existed reads the default for the column, and is rewritten with the column the first time a write lands in it. A CHECK constraint on the new column still runs over the existing rows. A table with no rows, a statement inside a transaction and a default the engine has to evaluate rewrite the rows. The first ADD COLUMN on a table that has rows moves the file to format 4, and a release that reads only format 3 refuses the file by name. On a connection with PRAGMA defensive on, which the shell sets, a shadow table cannot be dropped, altered or given a trigger.

An INSERT, UPDATE or DELETE on a view is refused unless the view has an INSTEAD OF trigger for that statement. With the trigger, the statement runs the trigger body, and the trigger is still there after the file is closed and opened again. INSERT with ON CONFLICT on a view fails with cannot UPSERT a view.

• CREATE VIEW stores a SELECT under a name.
• ANALYZE writes statistics to sqlite_stat1 for the query planner.
• REINDEX rebuilds indexes, for example after a collation changes.
• DROP TABLE, DROP VIEW, DROP INDEX and DROP TRIGGER remove objects. DROP VIEW also drops the triggers of the view.
• A trigger body can use subqueries. RETURNING is refused inside a trigger body.

Common scalar functions include abs, coalesce, hex, instr, length, lower, upper, printf, random, replace, round, substr, trim, typeof, unicode and zeroblob. The math functions include the trigonometric functions, logarithms, pow, floor, ceil and sqrt.

The date and time functions are date, time, datetime, julianday, unixepoch, strftime and timediff. Modifiers such as start of month and +1 day work as they do in SQLite.

The five SQLite library functions Inillucent does not have are fts3_tokenizer, fts5, fts5_get_locale, fts5_insttoken and fts5_locale. load_extension() is present and always refuses, because the engine has no way to load a C extension.

An application registers its own scalar functions and collations through the driver. A statement can call an application function. A schema cannot: a DEFAULT, a CHECK, a generated column or a view that names an application function is refused.

• Values and text: changes, char, coalesce, concat, concat_ws, format, glob, hex, ifnull, iif, instr, last_insert_rowid, length, like, likelihood, likely, lower, ltrim, nullif, octet_length, printf, quote, random, randomblob, replace, rtrim, sqlite_source_id, sqlite_version, substr, substring, total_changes, trim, typeof, unhex, unicode, unlikely, upper and zeroblob.
• Math: abs, acos, acosh, asin, asinh, atan, atan2, atanh, ceil, ceiling, cos, cosh, degrees, exp, floor, ln, log, log10, log2, mod, pi, pow, power, radians, round, sign, sin, sinh, sqrt, tan, tanh and trunc.
• Date and time: date, datetime, julianday, strftime, time, timediff and unixepoch.
• Aggregates: avg, count, group_concat, json_group_array, json_group_object, jsonb_group_array, jsonb_group_object, max, median, min, percentile, percentile_cont, percentile_disc, string_agg, sum and total.
• Windows: cume_dist, dense_rank, first_value, lag, last_value, lead, nth_value, ntile, percent_rank, rank and row_number. Each one needs an OVER clause.
• Vectors: binary_quantize, cosine_distance, hamming_distance, inner_product, jaccard_distance, l1_distance, l2_distance, l2_normalize, subvector, vector_add, vector_concat, vector_dims, vector_distance_cos, vector_distance_l2, vector_dot, vector_mul, vector_norm and vector_sub.
• JSON and JSONB: the next chapter lists them.

A JSON path starts with $. An object key follows a dot, and an array position goes in brackets, as in $.tags[0]. A path that finds nothing returns NULL. A malformed path or a malformed document returns an error.

The -> operator returns the JSON found at a path. The ->> operator returns it as an SQL value.

json_each and json_tree are table valued functions: you use them in FROM, like a table. json_each returns one row for each element of an array or object. json_tree returns one row for every element at every level of the document.

The other functions include json, json_array, json_object, json_extract, json_set, json_insert, json_replace, json_patch, json_remove, json_type, json_valid, json_pretty, json_quote, json_error_position and json_array_length, with a jsonb form of each function that returns a document.

Create an FTS5 virtual table with the text columns you want to search. The FTS5 table can store its own copy of the text, or read the text from another table. Prefix, phrase, boolean and column queries use the SQLite FTS5 syntax.

The index is stored in the same file as your tables and is part of the same transaction. A row inserted by a transaction that rolls back never appears in a search. Maintenance commands such as rebuild use the FTS5 form: an INSERT into the column that has the table's own name. Each commit writes its postings as a segment of its own, and small segments are merged into larger ones, so a load that commits each statement does not rewrite the list of every word for every row. A deleted row is hidden by a marker in a newer segment until a merge drops it. INSERT INTO t(t) VALUES('optimize') merges everything into one run. The first write to an FTS5 index that an earlier build wrote marks it as layout 3, and a build that does not read segments refuses it by name.

• The tokenizers are ascii, unicode61, porter and trigram. trigram matches any run of three or more characters, takes the options case_sensitive and remove_diacritics, and lets an index narrow LIKE and GLOB on its columns. Any other tokenizer name is refused by name.
• The query language is read by a parser written from SQLite's grammar, so a NOT, a nested column filter and a query SQLite refuses behave as they do in SQLite. bm25(), highlight() and snippet() count a phrase in a row only where the part of the query holding it matched that row.
• FTS5, R-Tree and geopoly tables work in temporary and attached databases as well as in main.
• The index stores every word position. The options detail and columnsize are refused.
• highlight(), snippet() and bm25() work as they do in SQLite.
• snippet() on an FTS3 or FTS4 table chooses its fragments the way SQLite does. It picks up to four fragments that cover the most phrases and joins them with the ellipsis text.
• FTS3 and FTS4 tables also work. The fts4aux and fts3tokenize modules are not built.
• An FTS5 table can take its content from a view, as in content='v', and a scan of any external content table reads the rows of its owner, so it answers before the first rebuild.
• FTS3 and FTS4 MATCH read the default query language of SQLite. x NOT y searches for the three words x, NOT and y, and AND is a word. NEAR and NEAR/n work, and a bare NEAR is NEAR/10. A NEAR with no term on one side is the error malformed MATCH expression. The operators bind as SQLite binds them: NEAR tightest, then OR, then two words side by side, so x y OR z is x AND (y OR z). A term written with a leading - applies to the whole query, so x -y NEAR/3 w is (x NEAR/3 w) NOT y. Parentheses only separate words.

The first column is an id. Each dimension then has a minimum column and a maximum column. A table in two dimensions has the columns id, min_x, max_x, min_y and max_y. A table can have one to five dimensions.

Two rectangles overlap when, on every axis, each one starts before the other one ends. The query below finds the stored rectangles that overlap the square from 8 to 12 on both axes.

The geopoly module and its functions, for polygons, are also built in.

An embedding is a list of numbers that stands for the meaning of a text. Texts with similar meanings get vectors that point in similar directions. HNSW is an index that finds the nearest vectors without comparing the query with every row.

A vector is stored as a BLOB of 32 bit floating point numbers in little endian order. In application code, bind the vector as a byte array parameter. The examples below write the bytes as hexadecimal literals so you can see them.

vector_distance_cos returns 0 for two vectors that point the same way and 1 for two vectors at right angles. vector_distance_l2 returns the straight line distance. vector_dot returns the dot product.

The planner uses an HNSW index for a query that orders by a distance function and then has LIMIT k. An index measures cosine distance unless you create it WITH (metric = 'l2'). When the query's distance function does not match the metric of the index, the planner scans the table instead.

An inillucent_hnsw index walks the HNSW graph by default. This is approximate mode: it is fast on a large table and can miss a true neighbor. CREATE INDEX ... WITH (mode = 'exact') makes the index compare the query with every stored vector instead, so the answer is always the true nearest rows. Below about 2,048 rows the index compares every vector in either mode, because that is faster than the walk. How many true neighbors the walk finds depends on your vectors, so compare its answers with an exact index on your own data before you rely on it, and raise ef_search if it misses too many. An index created by inillucent 1.0.29 or earlier is exact. Drop it and create it again to make it approximate.

Most applications compute their own vectors. Inillucent can also compute them with the SQL function embed(TEXT). One command, shown in the last example, downloads ONNX Runtime and the nomic-embed-text-v1.5 model on Windows, macOS or Linux. The download is about 620 MB, and every file is checked against a digest pinned in the build. The release archives include embed(TEXT). A build without it answers with the status unsupported.

Loading the model takes 650 to 800 ms. After that, one embedding takes 12 to 36 ms, and the loaded model holds about 1.9 GB of memory. A residency profile decides how long the model stays loaded. The resident profile keeps the model for the life of the process. The idle profile, the default, unloads the model 5 minutes after the last call. A third profile unloads the model after every call. On a graphics card, unloading leaves about 500 MB on the card until the process exits. That is the CUDA context, which ONNX Runtime cannot release.

• A VECTOR(N) column refuses a value with the wrong number of bytes.
• CREATE INDEX fills the index from the rows already in the table, and later writes keep the index current.
• A query can combine a WHERE filter with nearest neighbor ordering.
• An HNSW index covers one column. An index over two columns is refused.
• The mode is set when the index is created. WHERE mode = 'approximate' in a query fails with no such column: mode.
• embed(TEXT) returns the 3,072 bytes that a VECTOR(768) column holds. When its argument is the same for every row, embed(TEXT) runs once for the whole statement.
• embed(TEXT) can go anywhere an expression can go in a statement your program sends: a VALUES row, UPDATE ... SET, RETURNING or INSERT ... SELECT.
• embed(TEXT) is refused inside a trigger, a view, a generated column, a DEFAULT and a CHECK, with the message embed may only be used from top-level SQL. A database file must not be able to make the program that opens it load a model on every insert. A trigger can copy a vector that your own statement computed.

Declare the text columns you want to search. With no dims option, the table does keyword search only. With dims = N, the table also takes a column named vector, in the same byte format as VECTOR(N), and a query can search by meaning as well.

A virtual table is a table whose storage a module handles instead of the ordinary table code. It is not held in memory. An inillucent_search table keeps its rows, their vectors and its index in five ordinary tables inside the database file, named <table>_config, _content, _delta, _gen and _state. You insert a document once, and it is there after a restart. A search reads the stored rows and embeds only the question.

The table keeps its own copy of the text and the vector. It cannot read them from another table: the FTS5 option content = 'table' is refused. When your own table has to stay the record, triggers can copy each write into the search table, and chapter 18 shows the other way, where the search table holds the chunks.

MATCH gives the words to search for, and k sets how many results to return. A query that also sets vector to the embedding of the question runs both searches and combines their two lists into one. That is hybrid search.

Each result has four hidden columns. rank orders the results. score() is the number the order comes from. confidence() is calibrated against how strong the evidence is, so an application can give no answer when it is low. origin() says whether keyword search, vector search or both found the result. The k constraint sets how many hits the search collects, 10 by default, and the largest k is 1,000,000. A larger k is refused with inillucent_search: k cannot be more than 1000000. A vector of the wrong width is refused with a message that says it is not a vector of the declared width. A result that only keyword search found has a low confidence even when it is right, so check origin() before you decide that the table cannot answer.

The table takes the options in Table options below. metric is cosine by default, and l2 is the other choice. metric is fixed when the index is built, because a cosine index normalizes every vector it stores. fusion and vector_weight choose how the two lists are combined, and rerank_depth sets how many rows a reranker scores. tokenize chooses how text becomes terms, and Tokenizer packs covers text that is not English.

To use this table to find passages for a language model, read chapter 18, Retrieval Augmented Generation (RAG). It explains chunking, embedding, fusion and reranking one step at a time, with what a measured study found about each.

• Filters are applied during the search.
• For a small set of candidate rows, the planner picks an exact search.
• Keyword scores use BM25, with extra weight for matching more of the query words and for words that appear close together.
• Inserts, updates, deletes, compaction and rebuilds follow COMMIT and ROLLBACK and survive a reopen.
• Each commit writes its rows to a new segment of the index. Segments are merged later, and the merge work one commit does has a limit.
• An inillucent_search table can be joined to ordinary tables by rowid, or by a facet column.
• INSERT INTO ... SELECT fills an inillucent_search table from an ordinary table in one statement.
• A facet column is stored as text and has text affinity, so email_id = 3 and email_id = '3' select the same rows, in a search and in a plain SELECT, UPDATE or DELETE. Release 2.0.1 and earlier matched nothing for the integer outside a search.
• A search applies =, IN, >, >=, <, <= and BETWEEN on a facet column inside the search, so it returns k rows that pass them. Any other test on a facet, such as !=, LIKE or a test inside an OR, runs on the k rows the search returned and can leave fewer than k. Release 2.6.0 and earlier applied only = inside the search.

A language model has never read your email, your wiki or your support tickets. RAG gives it that text at the moment of the question. The application searches its own text, takes the best few passages, and puts them in the model's input next to the question. The model answers from those passages.

The answer can only be as good as the passages. If the search misses the passage that holds the answer, the model says it does not know, or it makes something up. So most of the work in RAG is the search, and the search is the part Inillucent does.

RAG has two phases. The prepare phase runs once for each document, when it is added. The answer phase runs for every question. The figure shows both, and the parts below take one step at a time.

The numbers in this chapter come from a study that searched 67,369 emails with 119 graded questions. The research page describes how it was run, every result, and why each feature was built the way it was.

Inillucent answers all 67 of the pragmas that SQLite 3.53.4 lists in pragma_list.

PRAGMA table_info, table_xinfo, table_list, index_list, index_xinfo and database_list describe the schema. PRAGMA page_size, page_count and freelist_count describe the file.

PRAGMA integrity_check, quick_check and foreign_key_check check the database. PRAGMA wal_checkpoint copies the write ahead log into the database file. PRAGMA cache_size, synchronous, busy_timeout, foreign_keys and locking_mode configure the connection. PRAGMA trusted_schema is off by default. Each pragma that answers a value names its result column after the pragma.

PRAGMA integrity_check inside a transaction does not report the pages of an index that REINDEX replaced, or of a dropped table, before the commit gives them back. An INSERT refused for a duplicate key leaves no pages of a long value behind, so integrity_check does not report a page that is never used.

PRAGMA auto_vacuum reads the mode of the file: 0 for NONE, 1 for FULL and 2 for INCREMENTAL. NONE is the default. In NONE mode a page that a delete frees stays in the file and later writes reuse it, so the file does not become shorter. In FULL mode every commit moves the pages at the end of the file into free pages nearer the start and lowers the page count, and the file is cut at the next checkpoint. In INCREMENTAL mode the free pages stay in the file until PRAGMA incremental_vacuum(N) runs. It gives back up to N free pages when the statement commits, and every free page when N is left out. In NONE and FULL mode PRAGMA incremental_vacuum changes nothing.

PRAGMA auto_vacuum = FULL or INCREMENTAL takes effect at once on a file that has no table. On a file in NONE mode that has tables, the new mode applies after VACUUM rebuilds the file. In FULL and INCREMENTAL mode the root page of every table and index is kept at the start of the file. A DELETE frees the out of line values of the rows it deletes in the same statement. A page that cannot be moved stays where it is and the step stops, so PRAGMA freelist_count can read more than 0, and VACUUM rebuilds the file without that limit.

VACUUM aux and VACUUM temp act on the database they name and leave main alone. VACUUM aux INTO writes a rebuilt copy of the attached database aux. VACUUM aux without INTO packs each table and index of the attached file into new trees in the same file and cuts the file after the commit, so the file gets shorter. It does not rename a new file over the old one, so another process that has the file attached keeps working. VACUUM INTO takes a bound parameter for its file name. SQLite writes an empty database for VACUUM temp INTO, and Inillucent writes a copy of the temporary tables.

PRAGMA max_page_count reads the limit on the number of pages in one open file, and PRAGMA max_page_count = N sets the limit when N is a positive integer. A value that is 0, negative or text is ignored. The limit applies to each attached file separately and is not stored in the file. A statement that needs a page past the limit fails with the message database or disk is full and the status full. The statement is undone and the transaction stays open.

PRAGMA automatic_index = OFF turns off the hash table that a join or a correlated subquery builds over a column that has no index. Chapter 08 describes both cases.

A PRAGMA that returns rows can also be used as a table in FROM, as in SELECT name FROM pragma_table_info('books').

Put the schema name in front of an object name, as in archive.books. DETACH closes an attached database when no statement is using it. PRAGMA database_list shows the names and paths that are open. DETACH of a database that was attached before another one works when the open transaction has written the later database. The write is committed, as in SQLite.

ATTACH takes any expression as its file name, such as ATTACH ?1 AS x, ATTACH 'a' || 'b' AS x or ATTACH (SELECT path FROM p) AS x. The engine evaluates the expression when the statement runs, with the bound parameters of the statement, and reads the value as text. A NULL names a private temporary database. The --root limit confines the evaluated name the same way it confines a literal one.

A transaction that writes two database files commits through a super journal, so both files get the change or neither does. Temporary objects live in a database that belongs to one connection and has no file. They disappear when the connection closes.

Each attached or temporary database keeps its own foreign keys, triggers, AUTOINCREMENT counters and schema changes. A foreign key in an attached or temporary table looks up its parent in the same database. A trigger in an attached database that names a table without a database reads and writes that same database. Each database has its own sqlite_sequence. DROP TRIGGER, DROP VIEW and ALTER TABLE RENAME of an attached file inside a transaction are kept when the file is opened again. DROP TRIGGER of a temporary trigger on a main table removes the trigger.

ATTACH ... KEY opens an encrypted database with its own key, and KEY '' opens a plaintext one. Without a KEY clause, an attached database is opened with the key of the connection. Chapter 26 covers encryption.

Values keep their types: Null, Integer, Real, Text or Blob. query and query_all hold every row of the result until they return. In Rust, query_each hands each row to a RowSink as the engine produces it, so a result too large to hold can be read. If a statement fails partway, the Rust driver returns the rows produced before the error in Error::partial, and inillucent query and the MCP tools return them in the rows member of the failure. The C library returns them once from inillucent_error_take_rows, and the Python, Node, PHP and Go packages put them on their error as partial. A sort, an aggregate, UNION, INTERSECT and EXCEPT build their whole answer first, so they return no rows before an error. Each result has an exact total, the rows up to the limit you asked for, and a flag that says whether the limit left rows out.

Unsupported is its own status. The error names the construct the engine has not built, so your application can tell a missing feature from a mistake in the SQL. The capability table answers the same question before you write the statement. The table has 57 rows: 43 supported, 2 partial and 12 refused. Chapter 25 shows how to read the capability table.

A Database is used from one thread. SharedDatabase lets many threads in one process share one database. The threads take turns, and one statement runs at a time. Several processes can also open the same file.

• A Database owns one file and creates connections.
• Each connection of one Database has a transaction of its own. The first connection runs on the engine the open made. Every later connection runs on an engine of its own over the same file, opened when the connection runs its first statement, and that engine takes the file locks the way a second process does. A BEGIN on one connection is not joined by another, the other connection reads only what is committed, and a second writer waits for the busy timeout and then fails with busy. A function, a collation, an authorizer, a pragma, a temporary table or an ATTACH set on one connection is not set on another. Opening that engine costs what opening a second Database costs, so a program keeps its connections instead of opening one for each statement. The C library frees the engine of a connection when the last handle on the connection is freed.
• In the Python binding, every call on a Database takes one lock that the Database owns, so a second thread waits for the first call to finish. Connection.cancel() does not take the lock.
• A program that called embed() holds ONNX Runtime sessions. inillucent_release_models() in the C library (ABI 1.5.0) drops the loaded embedder and reranker sessions and can be called when nothing is loaded. The Node and Python packages call it when the process exits. A binding for another language calls it once from its own exit hook.
• The C library copies every value you pass in before the call returns.
• Read only mode is enforced by the driver, which refuses any statement that is not a query. The file itself is still open for writing.
• A file this process may not write opens read only. Queries work, and a statement that writes fails with the status readonly.
• When the disk is full, the statement that met it fails with the status full, and the same connection writes again once space is free.
• Connection::transaction runs a batch of statements, passes the changed row counts to a check you write, and commits only when the check succeeds. Connection::begin gives you a Transaction to run one statement at a time. Dropping a Transaction without a commit rolls it back.
• Every C handle has a function that frees it. A pointer the library returns stays valid until you free the handle it came from.
• The reference Python binding uses ctypes from the Python standard library.
• An open waits up to five seconds for a file another process is writing. OpenOptions::busy_timeout in Rust, inillucent_open_with_timeout in C and busy_timeout_ms in Python set a longer wait.

All eight libraries call the same C library and offer the same API, so a program written in one language reads the same in the next. Values keep their types, NULL is never an empty string, and each result has an exact total next to the rows the limit returned.

The tabs below show the same program in each language. The program creates a person table, inserts two rows, reads them back by column name, and updates one row. Each output shown is what that program printed.

Each library is tested with the same conformance suite that the Rust driver runs. A library passes when its answers match the engine. The libraries are in the inillucent-clients repository, and each one has a README with its full reference.

Android and iOS apps run the engine inside the app. The inillucent repository has an Android library for Kotlin and Java in packages/android, built as an AAR for arm64-v8a, armeabi-v7a and x86_64, and a Swift package for iOS and macOS in packages/swift, built over an XCFramework. Both link the C library, both run the conformance suite, the Android library on an emulator and the Swift package on the iOS simulator, and both are built from source. Neither is on Maven Central or a Swift package index yet. Chapter 27 shows both.

A web page runs the engine as WebAssembly, through the inillucent-web package on npm, in memory or kept in the browser's origin private file system. Chapter 28 shows it, and the explorer runs it on a film database in this site.

A migration never writes to the source, so the original database stays as it was. The new file is built under another name and renamed into place when it is complete, so an application never opens a half written file. The command refuses a destination that already exists.

A path is read as a SQLite file. A postgres:// or mysql:// URL is a running server. The server is read inside one snapshot with repeatable read isolation, so every table and the schema come from the same instant. A value with no matching type here is copied as the text the server prints for it, so no precision is lost.

Inillucent stores data in its own file format, so it cannot open a SQLite file directly. Import the SQLite file instead. The import recreates the tables, rows, indexes, views, triggers and constraints in the new file. FTS5 tables are rebuilt, an FTS5 table with external content is rebuilt from the table it reads, columns declared UNINDEXED are kept, and R-Tree tables are carried with their rows. A table of 1,000 or more columns is copied, up to about 1,900 numbers per row, which is what fits one 32 KiB page. A table with a wider row fails with the status unsupported and the message a row of N columns does not fit one page.

A connection URL holds a password, and other users of the machine can see a program argument. You can leave the URL out and set INILLUCENT_SOURCE_URL instead. You can also write a single dash in place of the URL and type the URL on standard input.

The --kind index option, or the separate inillucent-migrate program, builds a database from a legacy retrieval index folder.

A page is one fixed size block of the file. The buffer pool is the memory that holds recently used pages. The write ahead log, or WAL, records each change before the change reaches the database file. A checkpoint copies logged changes into the database file, so old log segments can be removed.

The log is a series of segment files next to the database, such as app.rdb-wal.0000000001. Each log record has a crc32c checksum. Recovery runs on every open and stops at the first record whose checksum does not match, so a record that was only partly written is ignored. A connection that can write and opened a file whose log held a transaction with no commit record folds the log when it closes, so the next open recovers nothing.

The file keeps two copies of its meta page, the page that records where everything is. A rollback journal keeps the earlier contents until a checkpoint is on disk, so a crash in the middle of a checkpoint is undone when the file next opens. An older copy of the database file put back beside the log segments of the newer file also opens. When a committed bulk build names a root page that the file does not hold, or holds from before the checkpoint, the open stops before it writes anything, renames the segments with .foreign added to their names, and opens the file as the copy was. The command line says so, and --output json carries log_segments_set_aside. A log that belongs to the file is replayed as before. A checkpoint makes the data pages durable with a sync of their own before it writes the meta record, so a power loss during the checkpoint cannot leave the new meta record beside a damaged page. A value too large for one page is stored in a run of pages of its own.

One writer works at a time. Writers that commit at the same moment share one disk flush, which is called group commit. A reader never sees half of a transaction. Several processes can use one file under PRAGMA locking_mode = normal, the default. Before each statement a connection checks whether another process committed. On Windows, while no other process has the log open, the connection keeps the length of the log itself and the check costs nothing; another process that opens the log waits until the connection has stopped using that length. Set INILLUCENT_LOG_OPLOCK=0 to turn this off. SharedDatabase lets many threads in one process share one database, one statement at a time. When the log moves to a new segment, every connection to the file sees the commits made before the move, and so does a file opened with ATTACH. On Windows, a reader notices a rollback journal that another process put back, because the process that restores a journal opens the log segments first and holds them until it is done.

• Every file created by release 2.5.0 or later is file format 3. Releases 0.1.5 to 2.4.2 refuse such a file by name with the status unsupported and exit code 3. A file made by an earlier release opens with no migration and stays at format 2 until it stores a key out of line, until VACUUM rebuilds it, or until PRAGMA auto_vacuum is set to FULL or INCREMENTAL. A file never returns to a lower format.
• A file moves to format 4 the first time ALTER TABLE ADD COLUMN with a literal default is run on a table that has rows, or a row of more than about 1,900 integers is stored. The log then holds a FormatRaised record. A release that reads only format 3 refuses the file by name with the status unsupported and exit code 3, and refuses a log that holds the record. A file that never gains such a column or such a row stays at the format it had.
• Releases before 2.1.0 stored ALTER TABLE ADD COLUMN by writing the new column after the table constraints. Release 2.9.0 and later read such a table by moving its columns in front of its constraints, and write the corrected text back at the next checkpoint. A table, index or trigger whose stored text still cannot be read stops the open with a message that names it.
• Each table and each index takes at least one 32 KiB page, so 3,000 tables with one index each take about 197 MB. SQLite, at 4 KiB pages, takes about 25 MB for the same tables.
• The page size is 32 KiB. PRAGMA page_size on a new file does not change it. A database made at a page size other than 32 KiB keeps that page size when it is opened again, and VACUUM rebuilds it at the same size.
• The buffer pool has 4,096 frames of 32 KiB. PRAGMA cache_size reports -131072, which means 128 MiB.
• PRAGMA synchronous accepts OFF, NORMAL and FULL. FULL is the default.
• PRAGMA busy_timeout sets how long a writer waits for another writer. The default is 5000 ms. An open waits as long as INILLUCENT_BUSY_TIMEOUT says, and the database then starts with that value.
• All six journal modes work, and DELETE is the default. After PRAGMA journal_mode = wal, the file reopens in WAL mode.
• A transaction can change more pages than the buffer pool holds, in every journal mode. The pages the pool cannot keep go to a temporary spill file that belongs to the connection, until the commit.
• Crash tests stop a commit at each step and check that recovery leaves a correct file. They cover power loss in the DELETE, TRUNCATE and PERSIST journal modes, and I/O errors, a full disk and short writes in DELETE mode.
• Storm tests run four writer processes and two reader processes on one file, kill them at random moments, and check every acknowledged transaction byte for byte afterwards, with a small page cache, the search table and an encrypted file. Readers check inside every transaction they open that no transaction is ever seen in part.

Of the other 14 cases, 6 are vector search features SQLite does not have, so SQLite has no answer to compare with. 2 are DELETE and UPDATE with ORDER BY and LIMIT, which Inillucent runs and the pinned SQLite build refuses. 6 print something different from SQLite.

Two of those 6 come from the 32 KiB page size: the page_size and page_count PRAGMAs, and one line of .recover. Two report on a SQLite library that is not in the program: .vfslist and .stats. EXPLAIN prints the operators of this engine, where SQLite prints its bytecode. .limit reports a trigger_depth of 1000, and the two SQLite reference builds disagree on that value.

None of the 416 cases is refused, but some other constructs are. inillucent capabilities lists 11 refused constructs, and SQLite answers most of them. An example is a full text MATCH under an OR whose pattern reads a column of another table. Six of the 11 fail with exit code 3 and the status unsupported. Five fail with exit code 1 and the status syntax: load_extension, a compound query ordered by an expression, writing to a view, RETURNING inside a trigger, and changing a row of sqlite_schema. inillucent query and the MCP tool inillucent_query report syntax for a statement that does not parse, such as SELEC 1.

Inillucent does not open a SQLite file directly: import the file instead. It does not run two statements at the same time, and it does not run two writers at the same time. It has no interface for loading a C extension. cancel stops a running statement at the next point where the executor checks for a cancel request, so a sort that has started finishes first. In Rust, Database::cancel_handle() gives a handle another thread can cancel with.

Speed: in the graded run of 11 October 2026, at 100,000 rows on Windows with release 2.5.4, the relational engine was 499% faster than SQLite 3.53.4 overall and used 77% less processor time, with 16% more peak memory. SQLite ran within 2% of its idle reference in each of the three runs. Every family is faster than SQLite: reads by key 3,304%, ranges 643%, scans and aggregates 1,020%, joins 394%, writes 312%, transactions 132%, JSON, FTS5 and R-Tree 127%, statement preparation 96% and index builds 33%. The benchmark times 116 workloads, the engine plan of 34 through the engine’s API and through the Connection API and 48 common and edge case workloads through the Connection API, and 110 of them are faster than SQLite. Of the six that are not, a range DELETE is 212% slower because folding the log into the database file lands on it in the benchmark’s fixed order; on its own it is level with SQLite. Compiling SELECT 1 on every call through the engine’s own API is 79% slower; through the Connection API, which keeps compiled plans, it is 66% faster. A join over an index range and an autocommit update are within 6% of SQLite in both APIs, where the same index probes and one disk sync per commit decide the time.

Through the Connection API that every driver calls, on 48 common and edge case workloads, release 2.5.4 is faster than SQLite on 47 of them: 297% faster on the 28 training workloads and 266% faster on the 20 held out workloads, using 66% less processor time. On 3 October 2026 release 2.1.1 was 18% and 49% slower on the same workloads and used 115% more processor time. Earlier runs are on the performance history page of the repository, and the hill climbing lab report tells how each number moved.

Called the way most programs call it, inillucent is faster than SQLite. Measured on 8 October 2026 on 25 workloads (one inillucent process per command, a SQL script piped into inillucent-shell, the Python driver and the npm package) against the same use of SQLite, release 2.4.2 is 74% faster overall, where release 2.1.5 was 216% slower: the npm package is 418% faster than node:sqlite, the Python driver 135% faster than the sqlite3 module in Python, scripts piped into the shell 49% faster, and the command line 6% faster than the sqlite3 shell. All 25 workloads are faster than SQLite, and seven are 200% faster or more, Python lookups by key among them at 427%. The command line is close to SQLite because starting a program costs both engines the same and is most of a short command. The performance page has every workload.

• RIGHT JOIN and FULL OUTER JOIN work.
• Recursive CTEs and correlated subqueries work.
• STRICT tables, generated columns, CHECK, UNIQUE and foreign keys work.
• BEFORE, AFTER and INSTEAD OF triggers work.
• All four forms of ALTER TABLE work.
• Partial indexes, indexes on expressions and indexes on WITHOUT ROWID tables work.
• VACUUM and VACUUM INTO both rebuild the file. PRAGMA auto_vacuum = FULL and INCREMENTAL give pages back without a rebuild.
• PRAGMA max_page_count is enforced.
• FTS5, R-Tree, JSON, json_each, json_tree, generate_series, vector columns, HNSW indexes and inillucent_search work.
• ATTACH, DETACH, temporary objects, savepoints and commits across two files work.
• Encryption at rest works for the database file, the log and the journals, with PRAGMA rekey and ATTACH ... KEY.
• Inillucent answers 172 of the 177 function names in the pinned SQLite library, all 67 pragmas in SQLite's pragma_list, and 63 of SQLite's 65 dot commands.
• The shell and the driver report unsupported separately from syntax, constraint and I/O errors.

The key is a passphrase, or 32 raw bytes written as x' followed by 64 hex digits and '. A passphrase is turned into a key with PBKDF2-HMAC-SHA256 at 600,000 iterations, which costs about 200 ms each time the database is opened. A raw key has no such cost.

The command line, the shell and the MCP server read the key from a file named with --key-file, or from the INILLUCENT_KEY environment variable. None of them takes the key as a word on the command line, because other users of the machine can read a command line. The Rust driver has OpenOptions::key, the C library has inillucent_open_with_key, and the client libraries have a key option.

inillucent encrypt writes an encrypted copy of a plaintext database, and inillucent decrypt writes a plaintext copy of an encrypted one. PRAGMA rekey, or inillucent rekey, changes the key by rewriting one header in each file, so it takes the same time whatever the size of the database. PRAGMA encryption answers xchacha20-poly1305, or none for a plaintext database. It answers for the database the qualifier names, so PRAGMA aux.encryption reports the attached database aux and PRAGMA main.encryption reports the main one. It cannot be set: giving PRAGMA encryption a value fails and says so.

A wrong key, no key for an encrypted database, and a key for a plaintext database all fail with the status corrupt, which is what SQLite and SQLCipher answer for a file they cannot read as a database. An encrypted file is 1.6% larger at the default page size.

Encryption does not stop somebody who can write the file from putting back an older copy of it, and the size of each file can be seen without the key. The key is in memory while the database is open.

• Every file the engine writes goes through one encrypting layer, so the log, the journals and the sort files are encrypted as well as the database file.
• Each page is encrypted with its own random nonce and carries a tag, so a changed page is reported as damaged and never read back as rows.
• Crash tests cut the power at every step of a commit and of a checkpoint on an encrypted database, and every cut recovers.
• The ciphers are written in the project and checked against the test vectors their standards publish.

Both packages call the same C library as the Python, Node, Go and PHP packages, so a statement gives the same answer on a phone as on a laptop. The Swift package links the C library as a static library inside an XCFramework, and Swift imports its header as the module InillucentFFI. The Android library links the C library and a small JNI layer into one libinillucent_jni.so for each processor family.

Everything in the earlier chapters runs on a phone: tables, joins, transactions, full text search with bm25, VECTOR columns, HNSW indexes and the inillucent_search table. Two things do not. embed() needs ONNX Runtime and a model file in the app, and neither package includes them, so embed() fails with the status unsupported. Compute the vectors with a model the app already runs, such as a Core ML model on iOS or a TensorFlow Lite model on Android, and bind them as a parameter. The command line, the shell and the MCP server are programs for a computer, and an app calls the library instead.

Put the file in the app's own storage: Application Support on iOS, and context.filesDir on Android. Bind every value with ?1, ?2 and so on. A vector binds as text in the JSON array form, or as a blob of little endian 32 bit floats, which is what an array of Float holds on both platforms.

The engine is single threaded for each file. Every Database holds a lock, and every call on it, on its connections and on its statements takes that lock, so the app can call the library from any thread. cancel() on a connection does not take the lock. Another thread can call it to stop a long statement, which then fails with the status interrupted.

Open the file with a key and the file and its log are encrypted with XChaCha20 and Poly1305, as chapter 26 describes. Keep the key in the Keychain on iOS and in the Android Keystore on Android.

Neither package is on Maven Central or a Swift package index yet. Both build from a checkout of the repository. Its page docs/mobile.md and the readmes in packages/swift and packages/android have the full API.

• iOS 13 and macOS 11 or later. The XCFramework has slices for the iPhone, the simulator and the Mac. The package carries a privacy manifest that declares no tracking and no collected data.
• Android 7.0, which is API 24, or later, for arm64-v8a, armeabi-v7a and x86_64. Every native library is aligned to 16 KB pages, which Google Play requires of an app that targets Android 15.
• Both packages pass the same 35 case conformance suite as every other client library, on the iPhone simulator and on an Android emulator. The last example in this chapter shows both runs.

Two npm packages carry it. inillucent-web is the client most pages want: connect, query, scalar, transactions, and openPersistent for a database the browser keeps. inillucent-wasm is the layer under it: the module inillucent.wasm, the WASI host it runs on, the two file systems, and the C ABI's calls in JavaScript. Install the first with npm install inillucent-web; it brings the second.

connect(name) is awaited once, because the browser compiles the module asynchronously. Every call after it is synchronous, as in the other clients. That database lives in the page's memory and is gone when the page is. exportDatabase() returns it as the bytes of one .rdb file, which a page can offer as a download, and connect(name, { bytes }) opens a file the page was given. This works on a page's main thread, in a worker and in Node 20 or later.

openPersistent(name) keeps a database in the browser's origin private file system, between visits. It starts a worker and runs the engine there, because a browser gives synchronous access to those files only inside a worker, and the engine reads and writes synchronously. Every call on it returns a promise. A database is one folder in OPFS, named after the first argument, and removePersistent(name) deletes it. The log and recovery are the same as on a disk, so a tab closed in the middle of a write is recovered on the next open.

A browser lets one worker hold an OPFS file. A second openPersistent of the same name, in the same tab or another, is refused with DatabaseBusyError until the first is closed. OPFS with synchronous access exists in Chrome 108, Firefox 111 and Safari 16.4, and later versions.

A bundler such as Vite, webpack, esbuild or Rollup finds the module and the worker through new URL(..., import.meta.url), which all four understand. When yours moves the module somewhere else, pass its URL as wasm to connect or openPersistent. Serve it as application/wasm, so the browser compiles it while it downloads.

embed() is not in the WebAssembly build, because the ONNX runtime it needs does not build for WebAssembly. Compute the vectors with a model the page already runs, or ahead of time, and bind them. The explorer on this site does the second: its film database was embedded on our machine with inillucent embed, and it carries precomputed query vectors for its example phrases.

A statement with thousands of nested parentheses can run out of the browser's own call stack before the engine's stack check sees it. The client then replaces the engine with a spare it keeps ready, reopens the database and throws too_big. Nothing committed is lost, and a transaction that was open is rolled back, as after a crash.

• SQL, FTS5 with bm25, VECTOR columns, HNSW indexes, inillucent_search and encryption all work, with the same code as the native library.
• The module is 6.6 MiB, and 2.3 MiB compressed with gzip.
• One thread. A parallel HNSW build runs the same work in order.
• Both packages pass the 35 cases of the driver conformance suite under Node, and openPersistent is tested in Chromium and Firefox through Playwright.