Vercel AI SDK

Use the AI SDK's generateText, streamText, generateObject, and embed with on-device models — the same code you'd write for a cloud provider, with no API key and no data leaving the phone.

Overview

expo-ai-kit/ai is a first-class Vercel AI SDK provider implementing the LanguageModelV3 spec — native to AI SDK 6, and accepted by AI SDK 7 as well. Point model: at expoAiKit() and everything the SDK offers rides on the same on-device engine as the core API: the ecosystem's patterns, examples, and abstractions, running locally.

It's a thin wrapper over the same sendMessage / streamMessage / embed calls — you can mix AI SDK calls and core calls freely in one app. The core package stays zero-dependency; @ai-sdk/provider is an optional peer used only for types.

Install & Polyfills

npx expo install expo-ai-kit
npm i ai   # AI SDK 6+

import 'web-streams-polyfill/polyfill';
import 'text-encoding-polyfill';
import structuredClone from '@ungap/structured-clone';

if (!('structuredClone' in globalThis)) {
  (globalThis as any).structuredClone = structuredClone;
}

Generate & Stream Text

expoAiKit() with no arguments targets whatever model is currently active — the OS built-in by default.

import { generateText, streamText } from 'ai';
import { expoAiKit } from 'expo-ai-kit/ai';

// One-shot
const { text } = await generateText({
  model: expoAiKit(),
  prompt: 'Capital of France?',
});

// Streaming — token-by-token for plain text
const result = streamText({
  model: expoAiKit(),
  messages: [{ role: 'user', content: 'Write a short story' }],
});
for await (const chunk of result.textStream) {
  setText((t) => t + chunk);
}

Choosing a Model

Pass any model id setModel() accepts — a downloadable id like 'gemma-e2b', a built-in, or an id you registered with registerModel(). The provider activates it before generating (the model must already be downloaded). Settings take the same shape as setModel()'s options and apply on activation:

import { downloadModel } from 'expo-ai-kit';
import { expoAiKit } from 'expo-ai-kit/ai';

await downloadModel('gemma-e2b');

const gemma = expoAiKit('gemma-e2b', {
  generation: { temperature: 0.7, topK: 40 },
});

const { text } = await generateText({ model: gemma, prompt: '…' });

Tool Calling

AI SDK tools work — they ride the exact same prompt protocol as the core generateText(), so a model behaves identically through either API. The SDK owns the loop (stopWhen, step callbacks, etc.); the provider translates each round.

import { generateText, tool, stepCountIs } from 'ai';
import { expoAiKit } from 'expo-ai-kit/ai';
import { z } from 'zod';

const { text } = await generateText({
  model: expoAiKit(),
  prompt: 'What should I wear in Paris today?',
  tools: {
    getWeather: tool({
      description: 'Get the current weather for a city.',
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => fetchWeather(city),
    }),
  },
  stopWhen: stepCountIs(5),
});

Keep tool sets small and schemas flat — on-device models pick tools far more reliably that way. toolChoice: 'required' and { type: 'tool' } are best-effort prompt nudges (flagged with a compatibility warning) until native constrained decoding lands.

Structured Output

import { generateObject } from 'ai';
import { expoAiKit } from 'expo-ai-kit/ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: expoAiKit(),
  prompt: 'A quick weeknight pasta.',
  schema: z.object({
    title: z.string(),
    minutes: z.number(),
    ingredients: z.array(z.string()),
  }),
});

Embeddings

expoAiKit.embeddingModel() wraps embed() and resolves the platform default honestly — Apple's NLContextualEmbedding on iOS (reported as apple-nl-contextual), EmbeddingGemma 300M on Android (reported as embedding-gemma-300m; needs the androidEmbeddings config-plugin flag plus a prepareEmbeddingModel() download — see the Embeddings guide). Pass { task, language } as settings or per-call via providerOptions['expo-ai-kit']; dev builds warn when you embed without an explicit task, since EmbeddingGemma vectors are task-conditioned.

import { embed, embedMany, cosineSimilarity } from 'ai';
import { expoAiKit } from 'expo-ai-kit/ai';

const { embedding } = await embed({
  model: expoAiKit.embeddingModel(undefined, { task: 'retrieval-query' }),
  value: 'sunny day at the beach',
});

const { embeddings } = await embedMany({
  model: expoAiKit.embeddingModel(undefined, { task: 'retrieval-document' }),
  values: chunks,
});

On-Device Caveats

On-device models aren't cloud models; the provider reports every mismatch honestly through the AI SDK's warning system rather than failing silently:

• One generation at a time. On-device models share a single KV-cache, so concurrent calls reject with INFERENCE_BUSY. Await one call before starting the next.
• Per-call sampling is ignored (with an unsupported warning) — see Choosing a Model.
• Streaming buffers when tools or JSON output are requested — a half-streamed tool-call envelope would surface as garbage text deltas, so those runs emit the parsed result in one piece. Plain text streams token-by-token.
• No token usage numbers — on-device runtimes don't report them; all usage fields are undefined.
• No image or file prompt parts — text only for now (vision input is on the roadmap); they throw DEVICE_NOT_SUPPORTED.
• Errors are still typed — everything the provider throws is the same ModelError (with .code) as the core API.