API Reference

The complete public API of expo-ai-kit, grouped by capability, Text, Speech, Vision, Embeddings, then models, the AI SDK provider, the config plugin, types, and errors. Everything runs on-device, on both iOS and Android.

iOSAndroid

Messages everywhere use the same shape: { role: 'system' | 'user' | 'assistant'; content: string }. On-device models are stateless, pass the full conversation history on every call.

LLM

Generate and stream text with the OS model or a downloaded one, get typed objects back, or let the model call your functions. Requires the llm build option. See the LLM guide.

isAvailable()

Check whether the current device supports its built-in on-device model. Returns false on unsupported platforms and devices, and in apps built without the llm option (it never throws; the generation calls throw LLM_NOT_ENABLED instead). On Android, true means ML Kit is supported even when its OS-managed model still needs to download.

function isAvailable(): Promise<boolean>

import { isAvailable } from 'expo-ai-kit';

if (await isAvailable()) {
  await prepareBuiltInModel();
}

Availability is not preparation

Call isAvailable() to decide whether to show the feature. Await prepareBuiltInModel() before inference so Android can finish its first-use model download.

prepareBuiltInModel()

Make the platform's built-in generation model ready. Android downloads the AICore-managed ML Kit model when needed. iOS validates Apple Foundation Models availability. Repeated calls are safe and resolve immediately when the model is already ready.

function prepareBuiltInModel(): Promise<void>

import { isAvailable, prepareBuiltInModel } from 'expo-ai-kit';

if (!(await isAvailable())) {
  throw new Error('On-device AI is not supported on this device');
}

await prepareBuiltInModel();

Throws DEVICE_NOT_SUPPORTED when the built-in model cannot run and DOWNLOAD_FAILED if Android cannot prepare its model.

sendMessage()

Send a conversation and get a single response.

function sendMessage(
  messages: LLMMessage[],
  options?: LLMSendOptions,
): Promise<LLMResponse>

Options: systemPrompt?: string (used only if the array has no system message), signal?: AbortSignal.

import { sendMessage } from 'expo-ai-kit';

const { text } = await sendMessage(
  [{ role: 'user', content: 'Capital of France?' }],
  { systemPrompt: 'Answer in one word.' },
);

Cancellation

On-device, non-streaming generation can't always be interrupted mid-decode, signal always unblocks the caller, but the model may keep running in the background (a new call throws INFERENCE_BUSY until it finishes). To truly interrupt, use streamMessage().stop().

streamMessage()

Stream a response token-by-token. Returns a handle with a promise that resolves with the final text and a stop() to cancel.

function streamMessage(
  messages: LLMMessage[],
  onToken: (event: LLMStreamEvent) => void,
  options?: LLMStreamOptions,
): LLMStreamHandle // { promise, stop }

import { streamMessage } from 'expo-ai-kit';

const { promise, stop } = streamMessage(
  [{ role: 'user', content: 'Write a short story' }],
  (event) => {
    setText(event.accumulatedText); // event.token, event.isDone also available
  },
);

await promise; // resolves with { text }; call stop() to cancel early

generateObject()

Get a typed object validated against a JSON Schema. See the Structured Output guide for the full story.

function generateObject<T = unknown>(
  messages: LLMMessage[],
  schema: JSONSchema,
  options?: GenerateObjectOptions,
): Promise<GenerateObjectResult<T>> // { object, text }

Options: systemPrompt?, signal?, maxRepairAttempts? (default 2). Throws INFERENCE_FAILED if no schema-valid JSON is produced after the repair attempts.

import { generateObject } from 'expo-ai-kit';

const { object } = await generateObject<{ title: string; minutes: number }>(
  [{ role: 'user', content: 'A quick weeknight pasta.' }],
  {
    type: 'object',
    properties: { title: { type: 'string' }, minutes: { type: 'integer' } },
    required: ['title', 'minutes'],
  },
);

generateText()

Generate text, optionally letting the model call tools you provide. See the Tool Calling guide.

function generateText(
  messages: LLMMessage[],
  options?: GenerateTextOptions,
): Promise<GenerateTextResult>
// { text, steps, toolCalls, toolResults, finishReason }

Options: tools?: ToolSet, maxSteps? (default 5), systemPrompt?, signal?, maxRepairAttempts? (default 2).

import { generateText } from 'expo-ai-kit';

const { text } = await generateText(
  [{ role: 'user', content: 'Weather in Paris?' }],
  {
    tools: {
      getWeather: {
        description: 'Get current weather for a city.',
        parameters: {
          type: 'object',
          properties: { city: { type: 'string' } },
          required: ['city'],
        },
        execute: async ({ city }: { city: string }) => fetchWeather(city),
      },
    },
  },
);

stripThinking()

Pure helper that splits <think>…</think> reasoning (Qwen3-style models) from the answer. generateObject and generateText apply it internally.

function stripThinking(text: string): { text: string; reasoning: string }

Speech

iOSAndroidNew

On-device transcription, live microphone streaming and audio-file transcripts. Opt-in via the config plugin's speech flag; runs its own single-flight (SPEECH_BUSY), independent of text generation. See the Speech-to-Text guide for platform behavior and examples.

function transcribe(options: {
  audio: { uri: string } | { base64: string; mediaType?: string };
  locale?: string;        // BCP-47; defaults to the device locale
  signal?: AbortSignal;
}): Promise<{
  text: string;
  segments: { text: string; startSeconds: number; endSeconds: number }[]; // iOS; [] on Android
  language?: string;
  durationSeconds?: number;
}>

function streamTranscription(
  onUpdate: (update: { text: string; isFinal: boolean }) => void,
  options?: { locale?: string }
): { promise: Promise<TranscribeResult>; stop: () => void }

function getSpeechRecognitionAvailability(options?: { locale?: string }): Promise<
  | { status: 'available' }
  | { status: 'downloadable' | 'downloading' }
  | { status: 'unavailable';
      reason: 'platform' | 'os-version' | 'device' | 'locale' | 'not-enabled' }
>
function prepareSpeechRecognition(options?: {
  locale?: string;
  onProgress?: (progress: number) => void; // 0–1
}): Promise<void>
function getSupportedSpeechLocales(): Promise<string[]>
function getSpeechPermissionsAsync(): Promise<SpeechPermissionResponse>
function requestSpeechPermissionsAsync(): Promise<SpeechPermissionResponse>

Vision

iOSAndroidNew

Background removal, image labels, and text recognition (OCR) with Apple Vision on iOS and ML Kit on Android. Android is opt-in via the config plugin's vision flag; iOS needs nothing. Independent of the LLM and speech guards. Image operations take an image as { uri } (a file:// URI or path). See the Vision guide.

removeBackground()

Cut the subject out of a photo. The cutout is written to the app cache and returned as a file:// URI (PNG with transparency by default), together with where the subject sits in the source image. Coordinates are normalized (origin top-left, 0–1) unless namedpixel….

function removeBackground(
  image: { uri: string },
  options?: {
    subject?: NormalizedPoint;   // keep only the subject under this point (default: all)
    mask?: boolean;              // also write the grayscale mask PNG (default false)
    trim?: boolean;              // crop to the subject (default true)
    format?: 'png' | 'jpeg';     // default 'png'; JPEG flattens onto white
    quality?: number;            // JPEG quality 0–1 (default 0.9)
    maxPixels?: number;          // decode budget (default 6_000_000, max 25_000_000)
  },
): Promise<{
  uri: string;
  maskUri?: string;              // when mask: true
  width: number; height: number;
  sourceWidth: number; sourceHeight: number;
  bounds: NormalizedRect; pixelBounds: PixelRect;
  foregroundCoverage: number; centroid: NormalizedPoint;
  instanceCount: number; trimOrigin: NormalizedPoint;
}>

iOS 17+ on a physical device; Android with Google Play services after prepareVision(). Throws NO_SUBJECT_FOUND when the image has no foreground subject.

labelImage()

Ranked labels describing the image, highest confidence first. iOS needs a physical device (the Simulator cannot run Vision's classifier); Android's model is bundled with the app. Label vocabularies differ per platform (Vision identifiers such as consumer_electronics, ML Kit words such as Dog).

function labelImage(
  image: { uri: string },
  options?: { maxResults?: number; minConfidence?: number }, // defaults 10 (0 = all), 0.5
): Promise<{ label: string; confidence: number }[]>

recognizeText()

Read the text in an image, with normalized bounds for every block and line. languages selects Android's script models (Latin, Chinese, Japanese, Korean, Devanagari); iOS auto-detects when omitted.recognitionLevel, usesLanguageCorrection, and customWords are iOS-only.

function recognizeText(
  image: { uri: string },
  options?: {
    languages?: string[];                    // BCP-47, priority order
    minTextHeight?: number;                  // fraction of image height, 0–1
    recognitionLevel?: 'accurate' | 'fast';  // iOS
    usesLanguageCorrection?: boolean;        // iOS, default true
    customWords?: string[];                  // iOS
  },
): Promise<{
  text: string;
  blocks: {
    text: string; bounds: NormalizedRect; language?: string; cornerPoints?: NormalizedPoint[];
    lines: { text: string; bounds: NormalizedRect; confidence?: number; language?: string; cornerPoints?: NormalizedPoint[] }[];
  }[];
}>

detectFaces()

Find every face in a local image. No preparation or download needed. Enable vision on Android and rebuild. See the face detection guide for profile-photo and crop recipes.

function detectFaces(options: { uri: string }): Promise<FaceDetectionResult>

type FaceDetectionResult = {
  width: number;          // upright image size in pixels (EXIF applied)
  height: number;
  faces: DetectedFace[];  // largest first; [] when none
};
type DetectedFace = {
  bounds: NormalizedRect;   // 0–1, origin top-left, clamped to the image
  pixelBounds: PixelRect;   // the same box in upright image pixels
  confidence?: number;      // 0–1, Apple Vision only
};

Runs at full resolution; calls may run concurrently. Failures use IMAGE_DECODE_FAILED, VISION_FAILED, VISION_NOT_ENABLED, or DEVICE_NOT_SUPPORTED (iOS Simulator).

Vision lifecycle

Per-feature availability and the one call that downloads. Android's segmentation and OCR models are Google Play services modules installed by prepareVision(); the label and face models are bundled; iOS ships everything with the OS and resolves immediately.

function getVisionAvailability(): Promise<{
  backgroundRemoval: VisionFeatureAvailability;
  imageLabeling: VisionFeatureAvailability;
  textRecognition: VisionFeatureAvailability;
  faceDetection: VisionFeatureAvailability;
}>
// VisionFeatureAvailability =
//   | { status: 'available' }
//   | { status: 'downloadable' | 'downloading' }
//   | { status: 'unavailable'; reason: 'platform' | 'os-version' | 'device' | 'not-enabled' }

function prepareVision(options?: {
  features?: ('background-removal' | 'image-labeling' | 'text-recognition' | 'face-detection')[]; // default: all
  languages?: string[];                      // OCR script models to fetch (Android)
  onProgress?: (progress: number) => void;   // 0–1
}): Promise<void>

function getSupportedTextRecognitionLanguages(): Promise<string[]>

Embeddings

Turn text into vectors for semantic search and retrieval, then search them with dependency-free helpers. See the Embeddings guide.

embed()

iOSAndroidNew

Turn text into embedding vectors for semantic search and retrieval. See the Embeddings guide. iOS (17+): Apple's zero-download NLContextualEmbedding:language selects the script model. Android: EmbeddingGemma 300M via MediaPipe TextEmbedder, opt-in via the androidEmbeddings config-plugin flag, model prepared with prepareEmbeddingModel(); embed() itself never downloads.

function embed(
  texts: string[],
  options?: {
    task?: 'semantic-similarity' | 'retrieval-query' | 'retrieval-document';
    language?: string; // BCP-47, default 'en', selects the iOS script model; ignored on Android
  },
): Promise<EmbedResult>
// { embeddings: number[][]; dimensions: number; model: { id: string; revision: string } }

import { embed } from 'expo-ai-kit';

const { embeddings, dimensions, model } = await embed(
  ['hello world', 'goodbye'],
  { task: 'retrieval-document' },
);
embeddings.length; // 2, one vector per input, in order
model;             // identity, indexes are only comparable under identical identity

Embedding model lifecycle

iOSAndroidNew

Readiness and asset management for the embedding model. prepareEmbeddingModel() is the only call that downloads, on Android it fetches the ~184 MB EmbeddingGemma bundle (SHA-256-verified, atomic install, fails closed on partial/corrupt downloads); on iOS it prefetches the OS-managed assets for a language. Cancel/delete are Android-side (safe no-ops on iOS).

function getEmbeddingModelStatus(options?: { language?: string }): Promise<{
  status: 'not-downloaded' | 'downloading' | 'downloaded';
  sizeBytes: number; // ~184 MB pinned on Android; 0 on iOS (OS-managed)
  model: { id: string; revision: string };
}>
function prepareEmbeddingModel(options?: {
  language?: string;
  onProgress?: (progress: number) => void; // 0–1 (Android)
}): Promise<void>
function cancelEmbeddingModelDownload(): Promise<void>
function deleteEmbeddingModel(): Promise<void>
function getSupportedEmbeddingLanguages(): Promise<string[]> // iOS catalog; [] on Android

Retrieval toolkit

iOSAndroid

Pure-JS helpers for retrieval. They work on every platform with any source of vectors, since they only deal in plain number[].

chunkText()

Split a document into overlapping, sentence-aware chunks sized for embedding.

function chunkText(
  text: string,
  options?: { chunkSize?: number; overlap?: number }, // defaults: 1000, min(200, chunkSize / 5)
): string[]

cosineSimilarity()

Magnitude-invariant relevance score in [-1, 1]. Throws on a length mismatch.

function cosineSimilarity(a: number[], b: number[]): number

createVectorStore()

A lightweight in-memory vector store. Add records, then search() by a query vector for the top-k most similar. Snapshot with toJSON() and rehydrate by passing it back in.

function createVectorStore<M = unknown>(
  initial?: VectorRecord<M>[],
): VectorStore<M>

// VectorStore<M>:
//   add(id, vector, metadata?) · addMany(records) · get(id) · remove(id)
//   clear() · size · toJSON()
//   search(query, { topK = 10, minScore? }): VectorSearchResult<M>[]

import { createVectorStore } from 'expo-ai-kit';

const store = createVectorStore<{ text: string }>();
store.addMany(chunks.map((text, i) => ({ id: `c${i}`, vector: embeddings[i], metadata: { text } })));

const hits = store.search(queryVector, { topK: 4 });
// → [{ id, vector, metadata, score }, …] sorted by score, highest first

Models

Switch between the OS built-in models and downloadable ones. See the Models guide for a walkthrough.

getBuiltInModels()

List the OS built-in models (Apple FM on iOS, ML Kit on Android).

function getBuiltInModels(): Promise<BuiltInModel[]>

getDownloadableModels()

The full downloadable catalog (built-in registry + any registerModel() entries), enriched with per-device status, size, license, and meetsRequirements.

function getDownloadableModels(): Promise<DownloadableModel[]>

getDownloadedModels()

Return only downloadable models already present on the device, including models currently loading or ready for inference.

function getDownloadedModels(): Promise<DownloadableModel[]>

getRecommendedModel()

The most capable model the current device can actually run, or null.

function getRecommendedModel(): Promise<DownloadableModel | null>

downloadModel()

Download a model with integrity verification (SHA256). Reports progress 0–1.

function downloadModel(
  modelId: string,
  options?: { onProgress?: (progress: number) => void },
): Promise<void>

cancelDownload()

Cancel an in-flight download; the downloadModel promise rejects with DOWNLOAD_CANCELLED.

function cancelDownload(modelId: string): Promise<void>

deleteModel()

Delete a downloaded model file from disk (unloads it first if active).

function deleteModel(modelId: string): Promise<void>

setModel()

Activate a model for inference, the sole gatekeeper of model validity. For downloadable models this loads weights into memory; only one is loaded at a time.

function setModel(modelId: string, options?: SetModelOptions): Promise<void>
// SetModelOptions: { backend?: 'auto' | 'gpu' | 'cpu'; generation?: GenerationConfig }

await setModel('qwen3-1.7b', { generation: { temperature: 0.7, topK: 40 } });

unloadModel()

Unload the current downloadable model and revert to the OS built-in.

function unloadModel(): Promise<void>

getActiveModel()

The id of the currently active model (e.g. 'apple-fm').

function getActiveModel(): string

Custom Models

registerModel()

Add a custom downloadable model. Validates the entry and rejects ids that collide with a built-in (curated or native) model.

function registerModel(entry: ModelRegistryEntry): void

unregisterModel()

Remove a custom model (returns true if one was removed). Does not delete any downloaded file.

function unregisterModel(modelId: string): boolean

getRegisteredModels()

All custom models registered this session.

function getRegisteredModels(): ModelRegistryEntry[]

fetchModelMetadata()

Look up a model file's sha256 and sizeBytes from a HuggingFace resolve URL, to fill in a registerModel() entry.

function fetchModelMetadata(
  downloadUrl: string,
): Promise<{ sha256: string; sizeBytes: number }>

Pin the hash

Run this once at dev time and hardcode the returned sha256. Fetching it at runtime only catches transit corruption, not a changed upstream repo.

AI SDK Provider

iOSAndroidNew

A Vercel AI SDK provider (LanguageModelV3, AI SDK 6+) over the on-device engine, exported from the expo-ai-kit/ai subpath. See the Vercel AI SDK guide for setup (polyfills), examples, and the on-device caveats.

import { expoAiKit, createExpoAiKit } from 'expo-ai-kit/ai';

// LanguageModelV3, pass to generateText / streamText / generateObject
expoAiKit(modelId?: string, settings?: ExpoAiKitModelSettings): LanguageModelV3
//   modelId: 'auto' (default, the active model) or any setModel() id
//   settings: same shape as setModel() options; applied on activation

// EmbeddingModelV3 over embed(), resolves the platform default
// ('apple-nl-contextual' on iOS, 'embedding-gemma-300m' on Android)
expoAiKit.embeddingModel(modelId?: string, settings?: { task?: EmbeddingTask; language?: string }): EmbeddingModelV3

// Factory (a fresh provider instance)
createExpoAiKit(): ExpoAiKitProvider

import { generateText } from 'ai';
import { expoAiKit } from 'expo-ai-kit/ai';

const { text } = await generateText({
  model: expoAiKit(),
  prompt: 'Capital of France?',
});

Config Plugin

All options are off by default. An app compiles and ships only the native code, models, and permissions for the options it turns on, so apps that don't use a feature pay nothing for it. Without an option the matching APIs throw a typed *_NOT_ENABLED error (and isAvailable() returns false for the LLM). Changing an option requires a new native build (dev client / EAS, not an OTA update).

{
  "expo": {
    "plugins": [
      ["expo-ai-kit", {
        "llm": true,
        "speech": true,             // or { "microphonePermission": "…" }
        "vision": true,             // or ["face-detection", "text-recognition"]
        "androidEmbeddings": true
      }]
    ]
  }
}

Flag	Unlocks	What it adds
`llm`	`sendMessage`, `streamMessage`, `generateObject`, `generateText`, the model catalog, the AI SDK provider	LiteRT-LM runtime for downloadable models (about 30 MB of iOS arm64 code, 21 MB on Android) and the ML Kit Prompt API client; Android `minSdkVersion` 26. Built-in models are OS-provided.
`speech`	`transcribe`, `streamTranscription`, speech lifecycle	Android ML Kit speech backend + `RECORD_AUDIO`; iOS `NSMicrophoneUsageDescription`
`vision`	Android `removeBackground`, `labelImage`, `recognizeText`, `detectFaces`	`true`: every ML Kit vision client plus the bundled face and label models (no permissions). An array of feature names (`background-removal`, `image-labeling`, `text-recognition`, `face-detection`) compiles only those; a face-only build adds about 14 MB less than `true`. Features left out report `not-enabled` and throw `VISION_NOT_ENABLED`. iOS needs no option.
`androidEmbeddings`	Android `embed` and the embedding lifecycle	MediaPipe TextEmbedder (~25 MB APK); the ~184 MB model downloads at runtime

The plugin writes one expoAiKit.* key per option to android/gradle.properties and, for llm, to ios/Podfile.properties.json; the library's Gradle file and podspec read them at build time. A React Native app without Expo prebuild sets the same keys by hand (or $ExpoAiKitLLM = true in the Podfile), adds RECORD_AUDIO and the microphone usage string itself for speech, then runs pod install and rebuilds.

Types

type LLMRole = 'system' | 'user' | 'assistant';
type LLMMessage = { role: LLMRole; content: string };
type LLMResponse = { text: string };

type LLMSendOptions = { systemPrompt?: string; signal?: AbortSignal };
type LLMStreamOptions = { systemPrompt?: string };
type LLMStreamHandle = { promise: Promise<LLMResponse>; stop: () => void };
type LLMStreamEvent = {
  sessionId: string; token: string; accumulatedText: string; isDone: boolean;
};

// Sampling, applied at setModel(), best-effort per backend
type InferenceBackend = 'auto' | 'gpu' | 'cpu';
type GenerationConfig = {
  temperature?: number; topK?: number; topP?: number; seed?: number; maxTokens?: number;
};
type SetModelOptions = { backend?: InferenceBackend; generation?: GenerationConfig };

// Structured output
type JSONSchema = {
  type?: JSONSchemaType | JSONSchemaType[];
  properties?: Record<string, JSONSchema>;
  required?: string[];
  items?: JSONSchema;
  enum?: ReadonlyArray<string | number | boolean | null>;
  [key: string]: unknown;
};
type GenerateObjectOptions = {
  systemPrompt?: string; signal?: AbortSignal; maxRepairAttempts?: number;
};
type GenerateObjectResult<T> = { object: T; text: string };

// Tool calling
type Tool<TArgs = any, TResult = any> = {
  description: string;
  parameters: JSONSchema;
  execute?: (args: TArgs) => TResult | Promise<TResult>;
};
type ToolSet = Record<string, Tool>;
type ToolCall = { toolName: string; args: unknown };
type ToolResult = { toolName: string; args: unknown; result: unknown };
type StepResult = { text: string; toolCalls: ToolCall[]; toolResults: ToolResult[] };
type GenerateTextFinishReason = 'stop' | 'tool-calls' | 'max-steps';
type GenerateTextOptions = {
  tools?: ToolSet; maxSteps?: number;
  systemPrompt?: string; signal?: AbortSignal; maxRepairAttempts?: number;
};
type GenerateTextResult = {
  text: string; steps: StepResult[];
  toolCalls: ToolCall[]; toolResults: ToolResult[];
  finishReason: GenerateTextFinishReason;
};

// Embeddings
type EmbeddingTask = 'semantic-similarity' | 'retrieval-query' | 'retrieval-document';
type EmbedOptions = { task?: EmbeddingTask; language?: string };
type EmbeddingModelIdentity = { id: string; revision: string };
type EmbedResult = {
  embeddings: number[][]; dimensions: number; model: EmbeddingModelIdentity;
};
type EmbeddingModelState = {
  status: 'not-downloaded' | 'downloading' | 'downloaded';
  sizeBytes: number; model: EmbeddingModelIdentity;
};
type ChunkOptions = { chunkSize?: number; overlap?: number };
type VectorRecord<M = unknown> = { id: string; vector: number[]; metadata?: M };
type VectorSearchResult<M = unknown> = VectorRecord<M> & { score: number };
type VectorSearchOptions = { topK?: number; minScore?: number };
type VectorStore<M = unknown> = {
  add(id: string, vector: number[], metadata?: M): void;
  addMany(records: VectorRecord<M>[]): void;
  get(id: string): VectorRecord<M> | undefined;
  remove(id: string): boolean;
  clear(): void;
  readonly size: number;
  search(query: number[], options?: VectorSearchOptions): VectorSearchResult<M>[];
  toJSON(): VectorRecord<M>[];
};

// Vision
type VisionImageSource = { uri: string };
type NormalizedRect = { x: number; y: number; width: number; height: number }; // origin top-left, 0–1
type NormalizedPoint = { x: number; y: number };
type PixelRect = { x: number; y: number; width: number; height: number };
type VisionFeature = 'background-removal' | 'image-labeling' | 'text-recognition';
type VisionFeatureAvailability =
  | { status: 'available' }
  | { status: 'downloadable' | 'downloading' }
  | { status: 'unavailable'; reason: 'platform' | 'os-version' | 'device' | 'not-enabled' };
type VisionAvailability = {
  backgroundRemoval: VisionFeatureAvailability;
  imageLabeling: VisionFeatureAvailability;
  textRecognition: VisionFeatureAvailability;
  faceDetection: VisionFeatureAvailability;
};
type PrepareVisionOptions = {
  features?: VisionFeature[]; languages?: string[]; onProgress?: (progress: number) => void;
};
type RemoveBackgroundOptions = {
  subject?: NormalizedPoint; mask?: boolean;
  trim?: boolean; format?: 'png' | 'jpeg'; quality?: number; maxPixels?: number;
};
type RemoveBackgroundResult = {
  uri: string; maskUri?: string; width: number; height: number; sourceWidth: number; sourceHeight: number;
  bounds: NormalizedRect; pixelBounds: PixelRect; foregroundCoverage: number;
  centroid: NormalizedPoint; instanceCount: number; trimOrigin: NormalizedPoint;
};
type LabelImageOptions = { maxResults?: number; minConfidence?: number };
type ImageLabel = { label: string; confidence: number };
type RecognizeTextOptions = {
  languages?: string[]; recognitionLevel?: 'accurate' | 'fast';
  usesLanguageCorrection?: boolean; customWords?: string[]; minTextHeight?: number;
};
type RecognizedTextLine = {
  text: string; bounds: NormalizedRect; confidence?: number; language?: string;
  cornerPoints?: NormalizedPoint[];
};
type RecognizedTextBlock = {
  text: string; bounds: NormalizedRect; lines: RecognizedTextLine[]; language?: string;
  cornerPoints?: NormalizedPoint[];
};
type RecognizeTextResult = { text: string; blocks: RecognizedTextBlock[] };

// Models
type BuiltInModel = {
  id: string; name: string; available: boolean;
  platform: 'ios' | 'android'; contextWindow: number;
};
type DownloadableModelStatus =
  | 'not-downloaded' | 'downloading' | 'downloaded' | 'loading' | 'ready';
type DownloadableModel = {
  id: string; name: string; parameterCount: string; license: string;
  sizeBytes: number; contextWindow: number; minRamBytes: number;
  meetsRequirements: boolean; status: DownloadableModelStatus;
};
type ModelRegistryEntry = {
  id: string; name: string; parameterCount: string; quantization: string;
  downloadUrl: string; sha256: string; sizeBytes: number;
  contextWindow: number; minRamBytes: number;
  supportedPlatforms: ('ios' | 'android')[]; license: string;
  preferredBackend?: 'auto' | 'gpu' | 'cpu'; // used when setModel gets no backend
};

Errors

Failures throw a ModelError with a typed .code and .modelId, so you can branch on the cause:

import { ModelError } from 'expo-ai-kit';

try {
  await setModel('gemma-e4b');
} catch (e) {
  if (e instanceof ModelError && e.code === 'MODEL_NOT_DOWNLOADED') {
    await downloadModel('gemma-e4b');
  }
}

ModelErrorCode is one of: MODEL_NOT_FOUND, MODEL_NOT_DOWNLOADED, DOWNLOAD_FAILED, DOWNLOAD_CORRUPT, DOWNLOAD_STORAGE_FULL, DOWNLOAD_CANCELLED, INFERENCE_OOM, INFERENCE_FAILED, INFERENCE_BUSY, INFERENCE_CANCELLED, MODEL_LOAD_FAILED, DEVICE_NOT_SUPPORTED, LLM_NOT_ENABLED, EMBEDDINGS_NOT_ENABLED, LANGUAGE_NOT_SUPPORTED, SPEECH_BUSY, SPEECH_NOT_ENABLED, MIC_PERMISSION_DENIED, AUDIO_DECODE_FAILED, TRANSCRIPTION_FAILED, VISION_NOT_ENABLED, IMAGE_DECODE_FAILED, NO_SUBJECT_FOUND, VISION_FAILED, UNKNOWN.

The *_NOT_ENABLED codes mean the app was built without the matching config-plugin option (llm, speech, vision, androidEmbeddings); enabling one requires a new native build. LANGUAGE_NOT_SUPPORTED means no on-device model handles the requested language for embeddings, speech, or text recognition (the message names it; there is never a silent fall-back). Vision adds IMAGE_DECODE_FAILED (unreadable input), NO_SUBJECT_FOUND (nothing to cut out), and VISION_FAILED (engine failure).

Single-flight inference

Only one generation runs at a time. A concurrent sendMessage / streamMessage / generateObject / generateText rejects with INFERENCE_BUSY, wait for the active one, or stop() the active stream first. Speech has its own guard (SPEECH_BUSY); vision and embeddings have none, so they run alongside either.