LLM

Chat with the model your users' phones already have, or one you download, with two functions: sendMessage() and streamMessage().

Overview

By default the LLM runs on the OS model, Apple Foundation Models on iOS 26+ and the ML Kit Prompt API on supported Android devices, and you can switch to a downloadable Gemma, Qwen, or Phi model with setModel() (see Models). Every text function shares the same message shape:

type LLMMessage = { role: 'system' | 'user' | 'assistant'; content: string };

The message APIs are stateless: pass the complete history on every call. The Multi-turn guide shows the patterns.

Enable the LLM

Like every capability, the LLM is opt-in at build time. Turn it on in your app config and make a new native build (dev client or EAS, not an OTA update):

{
  "expo": {
    "plugins": [["expo-ai-kit", { "llm": true }]]
  }
}

The option links the LiteRT-LM runtime that runs downloadable models (about 30 MB of iOS arm64 code, 21 MB on Android) and, on Android, the ML Kit Prompt API client, which needs minSdkVersion 26. The built-in models are OS-provided and add nothing further. Without the option, isAvailable() returns false and the generation, activation, and download calls throw a typed LLM_NOT_ENABLED error; an app using only speech, vision, or embeddings ships none of this.

Availability & preparation

import { isAvailable, prepareBuiltInModel } from 'expo-ai-kit';

if (!(await isAvailable())) {
  // No built-in model here, offer a downloadable one (getRecommendedModel) or a fallback UI.
}

// Android may download its OS-managed model on first use; iOS validates availability.
// Resolves immediately when the model is already ready.
await prepareBuiltInModel();

Generate

import { sendMessage } from 'expo-ai-kit';

const { text } = await sendMessage([
  { role: 'user', content: 'Explain local-first AI in one sentence.' },
]);

Stream

For a ChatGPT-style experience, stream tokens as they are produced. The handle's promise resolves with the final text and stop() ends the generation early with what was produced so far. A generation started right after stop() may briefly reject with INFERENCE_BUSY while the model finishes stopping.

import { streamMessage } from 'expo-ai-kit';

const { promise, stop } = streamMessage(
  [{ role: 'user', content: 'Write a very short story.' }],
  (event) => setText(event.accumulatedText) // event.token, event.isDone also available
);

const { text } = await promise;

System prompts

Put a system message first in the array, or pass systemPrompt in the options, it is used only when the array has no system message.

const { text } = await sendMessage(
  [{ role: 'user', content: 'Tell me a joke' }],
  { systemPrompt: 'You are a comedian who specializes in dad jokes.' }
);

Cancellation

sendMessage() accepts an AbortSignal; aborting rejects with INFERENCE_CANCELLED. On-device, non-streaming generation cannot always be interrupted mid-decode, the caller is unblocked immediately, but the model may keep computing briefly (a new call throws INFERENCE_BUSY until it finishes). To truly interrupt a long generation, prefer streamMessage().stop().

One generation at a time

The device runs a single model context, so sendMessage, streamMessage, generateObject, and generateText share one guard: a second concurrent call rejects with INFERENCE_BUSY. Speech, vision, and embeddings have their own paths and never trip it, a voice → model → answer pipeline works as expected.

Beyond plain text

• Structured Output:generateObject() returns a typed object validated against a JSON Schema, with a bounded repair loop.
• Tool Calling:generateText() lets the model call functions you provide and answer from their results.
• Embeddings: retrieve the most relevant chunks of your own data and add them to the conversation.
• Vercel AI SDK: the same engine behind generateText / streamText from ai.