Embeddings & RAG

Turn text into vectors for semantic search, then retrieve the most relevant chunks of your own documents to ground the model's answers — all on-device.

Overview

Retrieval-augmented generation (RAG) lets a small on-device model answer questions over data it was never trained on — your notes, your docs, your app's content. You embed the text into vectors, store them, and at query time retrieve the chunks most similar to the question and feed them to sendMessage() or generateText(). Because everything runs locally, your data never leaves the device.

expo-ai-kit gives you two pieces:

• embed() — turns text into embedding vectors on both platforms: Apple's zero-download NLContextualEmbedding on iOS, and EmbeddingGemma 300M via MediaPipe TextEmbedder on Android (opt-in).
• A pure-JS toolkit — chunkText, cosineSimilarity, and createVectorStore — that does the chunking and retrieval on both platforms, with any source of vectors.

Platform Support

One embedding backend per platform — with very different asset models:

Android Setup (opt-in)

The Android backend is off by default — zero bytes added to your APK, and embed() throws a typed EMBEDDINGS_NOT_ENABLED error explaining the flag. Enable it with the config plugin:

{
  "expo": {
    "plugins": [["expo-ai-kit", { "androidEmbeddings": true }]]
  }
}

Embedding Text

Pass an array of strings; get one vector back per string, in order, plus the shared dimensions and the model identity that produced them.

import { embed } from 'expo-ai-kit';

const { embeddings, dimensions, model } = await embed([
  'The Eiffel Tower is in Paris.',
  'Mount Fuji is in Japan.',
]);

embeddings.length; // 2 — one vector per input
dimensions;        // 512 on iOS (per script model), 768 on Android
model;             // { id, revision } — see Model Identity below

Task Types

options.task says what the vectors are for. EmbeddingGemma (Android) is task-conditioned — telling it the task measurably improves retrieval — while iOS accepts the value as semantic intent only (vectors are identical across tasks).

Languages (iOS)

options.language (BCP-47, default 'en') selects which iOS script model to load — Latin, Cyrillic, or CJK — and is passed through to the tokenizer. An unsupported language throws a typed LANGUAGE_NOT_SUPPORTED error naming the language — never a silent fall-back to the Latin model — and there is deliberately no auto-detection (a mixed batch would silently mix vector spaces).

import { embed, getSupportedEmbeddingLanguages } from 'expo-ai-kit';

await embed(['привет мир'], { language: 'ru' });      // Cyrillic model
await embed(['你好世界'], { language: 'zh-Hans' });   // CJK model
await embed(['hello'], { language: 'tlh' });          // throws LANGUAGE_NOT_SUPPORTED

// Enumerate what the running device actually supports:
const languages = await getSupportedEmbeddingLanguages();

Verified on a physical iOS 27 device, the catalog covers: bg, cs, da, de, en, es, fi, fr, hr, hu, id, it, ja, kk, ko, nb, nl, pl, pt, ro, ru, sk, sv, tr, uk, vi, zh-Hans, zh-Hant. Always prefer getSupportedEmbeddingLanguages() at runtime — the list is OS-managed and can differ per device/OS. On Android the option is accepted and ignored: EmbeddingGemma is natively multilingual with a single vector space, so there is nothing to select (and getSupportedEmbeddingLanguages() returns []).

Model Identity

On iOS the identity is the resolved Apple model identifier and OS asset revision. On Android it pins every artifact — the MediaPipe version, model SHA-256, dimensions, and prompt-protocol version (e.g. 1.0.0-913b7a1edc7c-d768-tfc1) — so it changes whenever any of them does.

Model Lifecycle

embed() never downloads anything — on Android it throws a typed MODEL_NOT_DOWNLOADED until you prepare the model explicitly:

import {
  getEmbeddingModelStatus,
  prepareEmbeddingModel,
  cancelEmbeddingModelDownload,
  deleteEmbeddingModel,
} from 'expo-ai-kit';

const { status, sizeBytes, model } = await getEmbeddingModelStatus();
// status: 'not-downloaded' | 'downloading' | 'downloaded'
// sizeBytes: 183_816_181 on Android (~184 MB); 0 on iOS (OS-managed)

if (status !== 'downloaded') {
  await prepareEmbeddingModel({ onProgress: (p) => console.log(p) });
  // Android: SHA-256-verified atomic install; partial/corrupt downloads fail closed.
}

// cancelEmbeddingModelDownload() aborts an in-flight Android download;
// deleteEmbeddingModel() reclaims the ~184 MB. Both are safe no-ops on iOS,
// where the OS owns the assets.

On iOS the same calls drive the OS-managed per-language asset flow: prepareEmbeddingModel({ language: 'ru' }) prefetches the Cyrillic model ahead of the first embed (no progress granularity — the OS doesn't expose it).

Retrieval-Augmented Generation

The full loop: index your document once, then retrieve and answer at query time.

import { embed, chunkText, createVectorStore, sendMessage } from 'expo-ai-kit';

// --- Index once -------------------------------------------------------------
const chunks = chunkText(document);            // overlapping, sentence-aware
const { embeddings } = await embed(chunks, { task: 'retrieval-document' });

const store = createVectorStore<{ text: string }>();
store.addMany(
  chunks.map((text, i) => ({ id: `c${i}`, vector: embeddings[i], metadata: { text } })),
);

// --- Answer at query time ---------------------------------------------------
async function ask(question: string) {
  const { embeddings: [q] } = await embed([question], { task: 'retrieval-query' });
  const hits = store.search(q, { topK: 4 });
  const context = hits.map((h) => h.metadata!.text).join('\n\n');

  return sendMessage([
    { role: 'system', content: `Answer using ONLY this context:\n${context}` },
    { role: 'user', content: question },
  ]);
}

Chunking

chunkText() splits a document into overlapping pieces sized for embedding. It breaks on sentence and paragraph boundaries where possible so chunks read coherently, and repeats a little context (overlap) across boundaries so a fact split between two chunks still appears whole in at least one.

import { chunkText } from 'expo-ai-kit';

const chunks = chunkText(longDocument, {
  chunkSize: 1000, // target characters of new content per chunk (default 1000)
  overlap: 200,    // characters repeated into the next chunk (default min(200, chunkSize / 5))
});

Returns [] for empty input and a single chunk when the text already fits in chunkSize. Tune chunkSize to your retrieval granularity — smaller chunks pinpoint facts; larger chunks keep more context together.

The Vector Store

createVectorStore() is a lightweight in-memory store. Add records (id + vector + optional metadata), then search() by a query vector for the top matches by cosineSimilarity.

import { createVectorStore } from 'expo-ai-kit';

const store = createVectorStore<{ text: string; source: string }>();

store.add('a', vectorA, { text: '…', source: 'faq.md' });
store.addMany(records);

store.search(queryVector, { topK: 5, minScore: 0.3 });
// → [{ id, vector, metadata, score }, …] sorted by score, highest first

store.get('a');
store.remove('a');
store.size;
store.clear();

Persistence

The store owns no I/O — persistence is yours. toJSON() hands you a plain-array snapshot to write anywhere (AsyncStorage, a file, SQLite); pass it back to createVectorStore() to rehydrate. Re-embedding on every launch would be slow and wasteful, so index once and persist.

import AsyncStorage from '@react-native-async-storage/async-storage';
import { createVectorStore } from 'expo-ai-kit';

// Save
await AsyncStorage.setItem('kb', JSON.stringify(store.toJSON()));

// Restore
const saved = await AsyncStorage.getItem('kb');
const store = createVectorStore(saved ? JSON.parse(saved) : undefined);

Tips

• Embed chunks and the query with the same model — vectors from different models aren't comparable (and a dimension mismatch throws from cosineSimilarity).
• Store the source text as metadata so you can pass it straight into the prompt after a search.
• Use topK to bound how much context you inject — on-device models have small context windows, so 3–5 focused chunks usually beat stuffing in more.
• Set a minScore to drop weak matches rather than padding the prompt with irrelevant text.