> ## Documentation Index
> Fetch the complete documentation index at: https://growthx-refactor-llm.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# @outputai/llm

> Generate and stream text, structured outputs, and images with prompt file management

The `@outputai/llm` package is how you call LLMs from your steps and evaluators. It wraps the [AI SDK](https://sdk.vercel.ai/docs) and adds [prompt files](/prompts) - version-controlled `.prompt` files that live alongside your code and define the provider, model, temperature, and prompt template in one place. Call arguments are the lists under [Call arguments](#call-arguments). AI SDK helpers and types (`Output`, `tool`, `stepCountIs`, `ToolSet`) are on the `aiSdk` namespace: `import { generateText, aiSdk } from '@outputai/llm'`.

## Generate Functions

`generateText` is the primary function for LLM calls. Use the `output` parameter with `aiSdk.Output.*` helpers to control the response shape. When you need progress as text arrives, prefer `generateTextWithStreaming` in workflow steps: it uses streaming internally but returns a complete result and rejects on provider or transport errors. Use `streamText` when you need direct control over the stream:

| Output Shape                           | How                                                                  | Use when you need                                                |
| -------------------------------------- | -------------------------------------------------------------------- | ---------------------------------------------------------------- |
| Unstructured text                      | `generateText({ prompt })`                                           | Summaries, emails, explanations                                  |
| Complete text with streaming callbacks | `generateTextWithStreaming({ prompt, onChunk })`                     | Progress updates with normal promise-based error handling        |
| Raw text stream                        | `streamText({ prompt })`                                             | Direct access to `textStream`, `fullStream`, and stream promises |
| Typed object                           | `generateText({ prompt, output: aiSdk.Output.object({ schema }) })`  | Structured data, evaluator judgments                             |
| Array of objects                       | `generateText({ prompt, output: aiSdk.Output.array({ element }) })`  | Lists, multiple items                                            |
| One of N choices                       | `generateText({ prompt, output: aiSdk.Output.choice({ options }) })` | Classification, routing                                          |
| Image                                  | `generateImage({ prompt })`                                          | Text-to-image, image-to-image, and image edits                   |

### Text Output

Generate unstructured text from a prompt file:

```typescript steps.ts theme={null}
import { step } from '@outputai/core';
import { generateText } from '@outputai/llm';
import { GenerateSummaryInput, GenerateSummaryOutput } from './types.js';

export const generateSummary = step({
  name: 'generateSummary',
  description: 'Generate a company summary from research data',
  inputSchema: GenerateSummaryInput,
  outputSchema: GenerateSummaryOutput,
  fn: async (input) => {
    const { result } = await generateText({
      prompt: 'generate_summary@v1',
      variables: {
        companyName: input.name,
        industry: input.industry,
        size: input.size
      }
    });

    return result;
  }
});

// types.ts
// import { z } from '@outputai/core';
//
// export const GenerateSummaryInput = z.object({
//   name: z.string(),
//   industry: z.string(),
//   size: z.number()
// });
//
// export const GenerateSummaryOutput = z.string();
```

`result` is a convenience alias for `response.text`.

### Streaming

#### Complete result over streaming transport

`generateTextWithStreaming` behaves like `generateText`: await it to receive the complete response, including `result`, `text`, `output`, `usage`, `finishReason`, and `cost`. Internally it uses streaming transport and invokes `onChunk` as parts arrive.

This is the recommended streaming API for Output workflow steps. Provider, transport, and abort errors reject the returned promise, so the step fails and Temporal can apply its retry policy.

```typescript steps.ts theme={null}
import { step } from '@outputai/core';
import { generateTextWithStreaming } from '@outputai/llm';
import { GenerateContentInput, GenerateContentOutput } from './types.js';

export const generateContent = step({
  name: 'generateContent',
  description: 'Generates content with streaming progress',
  inputSchema: GenerateContentInput,
  outputSchema: GenerateContentOutput,
  fn: async ({ topic }) => {
    const chunks: string[] = [];
    const result = await generateTextWithStreaming({
      prompt: 'stream_content@v1',
      variables: { topic },
      onChunk({ chunk }) {
        if (chunk.type === 'text-delta') {
          chunks.push(chunk.text);
        }
      }
    });

    return {
      content: result.result,
      chunkCount: chunks.length,
      avgChunkSize: chunks.length > 0 ? Math.round(result.result.length / chunks.length) : 0
    };
  }
});
```

`generateTextWithStreaming` also supports structured output. Pass an `aiSdk.Output.*` specification and read the parsed value from `result.output`, just as with `generateText`.

#### Direct stream access

`streamText` remains available when you need to choose how the stream is consumed. It is not `async`: it returns a stream result synchronously, with `textStream` and `fullStream` iterables plus promise-based properties such as `text`, `usage`, and `finishReason`.

AI SDK streaming reports provider and transport failures through `onError`; consuming `textStream` does not reliably throw that original error. In a workflow step, capture the mapped error and throw it after consumption so Temporal records a failed activity instead of an empty successful result:

```typescript theme={null}
const captured: { error: unknown } = { error: null };
const result = streamText({
  prompt: 'generate@v1',
  variables: { topic: 'AI safety' },
  onError({ error }) {
    captured.error = error;
  }
});

const chunks: string[] = [];
for await (const chunk of result.textStream) {
  chunks.push(chunk);
}

if (captured.error) {
  throw captured.error;
}

const content = chunks.join('');
```

Registering `onError` alone is not enough to fail the step. Output treats it as a fire-and-forget observer: exceptions and rejected promises from the callback are ignored to avoid a secondary stream failure. Capture the mapped error and throw it after consumption; awaiting a completion property can produce a generic no-output error instead of the original provider error.

### Object Output

Generate a structured object matching a Zod schema. This is what you'll use most in evaluators:

```typescript evaluators.ts theme={null}
import { evaluator, EvaluationBooleanResult } from '@outputai/core';
import { generateText, aiSdk } from '@outputai/llm';
import { z } from '@outputai/core';
import { JudgeSummaryInput } from './types.js';

export const judgeSummaryQuality = evaluator({
  name: 'judgeSummaryQuality',
  description: 'Judge whether a company summary is accurate and useful',
  inputSchema: JudgeSummaryInput,
  fn: async (input) => {
    const { output } = await generateText({
      prompt: 'judge_summary@v1',
      variables: {
        summary: input.summary,
        companyName: input.companyName
      },
      output: aiSdk.Output.object({
        schema: z.object({
          reasoning: z.string(),
          passes: z.boolean(),
          confidence: z.number()
        })
      })
    });

    return new EvaluationBooleanResult({
      value: output.passes,
      confidence: output.confidence,
      reasoning: output.reasoning
    });
  }
});

// types.ts
// import { z } from '@outputai/core';
//
// export const JudgeSummaryInput = z.object({
//   summary: z.string(),
//   companyName: z.string()
// });
```

`output` contains the typed object matching your schema.

### Image Output

Generate images from a prompt file with `generateImage`. Image prompt files use plain instructions, not chat role tags like `<system>` or `<user>`. Keep plain text as the first meaningful body content so Output selects [instruction mode](/prompts#prompt-body-modes):

```yaml prompts/nascar_race@v1.prompt theme={null}
---
provider: openai
model: gpt-image-1
size: 1024x1024
n: 1
providerOptions:
  openai:
    quality: high
---

Create a cinematic motorsport image.

Scene:
{{ scene }}
```

Call `generateImage` from a step:

```typescript steps.ts theme={null}
import { randomUUID } from 'node:crypto';
import { mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { step, z } from '@outputai/core';
import { generateImage } from '@outputai/llm';

export const generateRaceImage = step({
  name: 'generateRaceImage',
  description: 'Generate a race image from a prompt file',
  inputSchema: z.object({
    scene: z.string()
  }),
  outputSchema: z.object({
    fileName: z.string()
  }),
  fn: async ({ scene }) => {
    const response = await generateImage({
      prompt: 'nascar_race@v1',
      variables: { scene }
    });

    if (!response.result?.base64) {
      throw new Error('Image generation did not return base64 image data.');
    }

    mkdirSync('_temp', { recursive: true });
    const fileName = `race-${randomUUID()}.png`;
    writeFileSync(join('_temp', fileName), Buffer.from(response.result.base64, 'base64'));

    return { fileName };
  }
});
```

`result` is a convenience alias for the first generated image (`response.images[0]`). The returned image exposes AI SDK image fields such as `base64` and `mediaType`.

For image-to-image or edit flows, pass runtime image inputs with `images` and optionally `mask`. Output forwards these to the AI SDK prompt object:

```typescript theme={null}
import { readFileSync } from 'node:fs';
import { generateImage } from '@outputai/llm';

const referenceImage = readFileSync('nascar-reference.jpg');

const response = await generateImage({
  prompt: 'nascar_race@v1',
  variables: {
    scene: 'Use the reference car as the hero car in a night race restart.'
  },
  images: [referenceImage]
});

const image = response.result;
```

Supported image inputs follow the AI SDK shape: `Buffer`, `Uint8Array`, `ArrayBuffer`, raw base64 strings, or `{ data, mediaType }` objects. `mask` uses the same input shape and requires `images`.

<Note>
  `generateImage` does not upload generated images, download remote images, or normalize provider-specific values like `size: "auto"`. Download or upload files in your workflow/client code, pass image bytes to `images`, and set concrete provider options in prompt front matter (`size`, `n`, `aspectRatio`, `seed`, `providerOptions`).
</Note>

Common image options can live in prompt front matter:

| Option             | Description                                                                               |
| ------------------ | ----------------------------------------------------------------------------------------- |
| `n`                | Number of images to request when supported by the provider/model                          |
| `maxImagesPerCall` | AI SDK image batching limit                                                               |
| `size`             | Concrete image size such as `1024x1024`                                                   |
| `aspectRatio`      | Aspect ratio such as `1:1` or `16:9`                                                      |
| `seed`             | Seed for deterministic output when supported                                              |
| `providerOptions`  | Provider-specific options, for example `openai.quality` or `vertex.imageConfig.imageSize` |

### Array Output

Generate an array of structured items:

```typescript theme={null}
import { generateText, aiSdk } from '@outputai/llm';
import { z } from '@outputai/core';

const { output } = await generateText({
  prompt: 'extract_contacts@v1',
  variables: { companyData: JSON.stringify(company) },
  output: aiSdk.Output.array({
    element: z.object({
      name: z.string(),
      role: z.string(),
      email: z.string().optional()
    })
  })
});

// output is an array of { name, role, email } objects
```

### Choice Output

Select one value from a set of options:

```typescript theme={null}
import { generateText, aiSdk } from '@outputai/llm';

const { output } = await generateText({
  prompt: 'classify_lead@v1',
  variables: { activity: leadActivity },
  output: aiSdk.Output.choice({ options: ['hot', 'warm', 'cold', 'unknown'] })
});

// output is one of 'hot', 'warm', 'cold', 'unknown'
```

## Agents

The `Agent` class wraps AI SDK's `ToolLoopAgent` with Output [prompt files](/prompts) and the [skills](/prompts/skills) system. Use it when you need multi-step tool execution, conversation history, or a reusable agent instance with a fixed configuration. For single-shot LLM calls without tools, `generateText` is simpler.

### Construction

The prompt file is loaded and rendered at construction time. Variables and tools are fixed at construction. Skills and `maxSteps` come from the prompt file. The agent is ready to call `generate()`, `generateWithStreaming()`, or `stream()` immediately.

Each call seeds authored `<user>` blocks from the prompt. `<system>` blocks become `instructions`. Authored `<assistant>` blocks are dropped; use `generateText` when the prompt is a few-shot or prefilled thread.

```typescript steps.ts theme={null}
import { step } from '@outputai/core';
import { Agent, aiSdk } from '@outputai/llm';

export const reviewContent = step({
  name: 'reviewContent',
  description: 'Review content with structured feedback',
  inputSchema: ReviewContentInput,
  outputSchema: ReviewContentOutput,
  fn: async (input) => {
    const agent = new Agent({
      prompt: 'writing_assistant@v1',
      variables: {
        content_type: input.contentType,
        focus: input.focus,
        content: input.content
      },
      output: aiSdk.Output.object({ schema: reviewSchema })
    });
    const { output } = await agent.generate();
    return output;
  }
});
```

**Constructor options:**

| Option         | Type                    | Default      | Description                                                          |
| -------------- | ----------------------- | ------------ | -------------------------------------------------------------------- |
| `prompt`       | `string`                | *(required)* | Prompt file name (e.g. `'writing_assistant@v1'`)                     |
| `promptDir`    | `string`                | -            | Override the stack-resolved prompt directory                         |
| `variables`    | `PromptVariables`       | -            | Template variables rendered at construction                          |
| `tools`        | AI SDK tools            | -            | Caller tools; merged with prompt YAML tools (`load_skill` last)      |
| `stopWhen`     | function or function\[] | -            | Custom stop condition (overrides prompt `maxSteps` when tools exist) |
| `output`       | `aiSdk.Output`          | -            | Structured output spec (e.g. `aiSdk.Output.object({ schema })`)      |
| `messageStore` | `MessageStore`          | -            | Pluggable store for multi-turn history                               |

### generate()

Run the agent and return when complete:

```typescript theme={null}
const result = await agent.generate();
console.log(result.text);   // Generated text
console.log(result.output); // Structured output (when using aiSdk.Output.object)
console.log(result.usage);  // Token counts
```

The result has the same shape as `generateText`: `text`, `result` (alias for `text`), `output`, `usage`, `finishReason`, `toolCalls`, etc.

Pass additional messages to extend the conversation. You can also pass `abortSignal` and `toolChoice`:

```typescript theme={null}
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Now focus on the introduction section.' }]
});
```

### generateWithStreaming()

Use `generateWithStreaming()` when you want streaming progress and a complete result. It accepts the same `messages`, `abortSignal`, and `toolChoice` as `generate()`, plus `onChunk`:

```typescript theme={null}
const result = await agent.generateWithStreaming({
  onChunk({ chunk }) {
    if (chunk.type === 'text-delta') {
      process.stdout.write(chunk.text);
    }
  }
});
```

Like `generate()`, the method returns the complete response, rejects on stream errors, and automatically appends messages to the configured message store. Prefer it over `stream()` in workflow steps unless you need direct access to the stream result.

### stream()

Use `stream()` when you need direct access to the agent's stream result. It accepts the same `messages`, `abortSignal`, and `toolChoice` as `generate()`, plus `onChunk`, `onFinish`, and `onError`:

```typescript theme={null}
const captured: { error: unknown } = { error: null };
const stream = await agent.stream({
  onError({ error }) {
    captured.error = error;
  }
});

for await (const chunk of stream.textStream) {
  process.stdout.write(chunk);
}

if (captured.error) {
  throw captured.error;
}
```

Like `streamText`, the stream result provides `textStream` and `fullStream` iterables, plus promise-based properties (`text`, `usage`, `finishReason`) that resolve on completion. In a workflow step, capture and rethrow `onError` as shown so a failed stream cannot become an empty successful activity.

### Structured Output

Use `aiSdk.Output.object()` with Agent to get typed responses:

```typescript steps.ts theme={null}
import { Agent, aiSdk } from '@outputai/llm';
import { z } from '@outputai/core';

const reviewSchema = z.object({
  issues: z.array(z.string()).describe('List of issues found'),
  suggestions: z.array(z.string()).describe('Actionable suggestions'),
  score: z.number().describe('Quality score 0-100'),
  summary: z.string().describe('Brief overall assessment')
});

const agent = new Agent({
  prompt: 'writing_assistant@v1',
  variables: { content_type: 'documentation', focus: 'clarity', content: markdownContent },
  output: aiSdk.Output.object({ schema: reviewSchema })
});

const { output } = await agent.generate();
// output: { issues: string[], suggestions: string[], score: number, summary: string }
```

### Message Store

By default, Agent is stateless. Each `generate()` / `stream()` call starts from the prompt seed (authored `<user>` blocks) plus this turn's `messages`. Pass a `messageStore` to keep history across calls.

The store holds this turn's caller messages plus the model reply. It does not persist the prompt seed. Reconstructing an agent is the same prompt (and variables) plus a hydrated store.

```typescript theme={null}
import { Agent } from '@outputai/llm';
import type { MessageStore } from '@outputai/llm';

const messages: Parameters<MessageStore['addMessages']>[0] = [];
const messageStore: MessageStore = {
  getMessages: () => messages,
  addMessages: incoming => {
    messages.push(...incoming);
  }
};

const chatbot = new Agent({
  prompt: 'chatbot@v1',
  messageStore
});

const r1 = await chatbot.generate({
  messages: [{ role: 'user', content: 'Hello, tell me about Output.' }]
});
// r1.text: "Output is an AI framework for..."

const r2 = await chatbot.generate({
  messages: [{ role: 'user', content: 'How does it handle retries?' }]
});
// r2 sees the full history from r1
```

`MessageStore` is:

```typescript theme={null}
interface MessageStore {
  getMessages(): ModelMessage[] | Promise<ModelMessage[]>;
  addMessages(messages: ModelMessage[]): void | Promise<void>;
}
```

`ModelMessage` is an AI SDK type (`aiSdk` / `ai`). There is no built-in store. Implement the interface in memory for a single process, or with your database for durable history.

<Note>
  `generate()`, `generateWithStreaming()`, and `stream()` append messages to the message store. `stream()` stores in its wrapped `onFinish` when `finishReason` is not `'error'`.
</Note>

### When to Use Agent vs generateText

|                          | `generateText`          | `Agent`                         |
| ------------------------ | ----------------------- | ------------------------------- |
| **Best for**             | Single-shot LLM calls   | Multi-step tool loops           |
| **Tools**                | Supported               | Supported                       |
| **Skills**               | Supported               | Supported                       |
| **Conversation history** | Manual                  | Built-in with `messageStore`    |
| **Reusable instance**    | No (function call)      | Yes (construct once, call many) |
| **Structured output**    | `aiSdk.Output.object()` | `aiSdk.Output.object()`         |

Start with `generateText`. Move to `Agent` when you need conversation state or a reusable instance with a fixed configuration.

## Response Object

`generateText`, `generateTextWithStreaming`, `Agent.generate()`, and `Agent.generateWithStreaming()` return the complete [AI SDK response](https://sdk.vercel.ai/docs/reference/ai-sdk-core/generate-text#returns):

| Field          | Description                                                                                                        |
| -------------- | ------------------------------------------------------------------------------------------------------------------ |
| `result`       | Convenience alias for `text`                                                                                       |
| `text`         | The raw generated text                                                                                             |
| `output`       | The structured output when using `aiSdk.Output.*` helpers                                                          |
| `usage`        | Token counts: `inputTokens`, `outputTokens`, `totalTokens`                                                         |
| `finishReason` | Why generation stopped (`'stop'`, `'length'`, `'tool-calls'`, etc.)                                                |
| `response`     | Raw provider response metadata                                                                                     |
| `warnings`     | Any warnings from the provider                                                                                     |
| `toolCalls`    | Tool calls made by the model (when using tools)                                                                    |
| `sources`      | Merged tool + provider sources (`sourceType: 'url'` or `'document'`). Always an array.                             |
| `cost`         | LLM usage attribute (`type`, `modelId`, `usage`, `total`, `tokensUsed`); `null` when pricing could not be computed |

The `cost` property is an LLM usage attribute:

```json theme={null}
{
  "type": "llm:usage",
  "modelId": "gpt-4o",
  "usage": [
    { "type": "input", "ppm": 5, "amount": 217, "total": 0.001085 },
    { "type": "output", "ppm": 15, "amount": 9, "total": 0.000135 }
  ],
  "total": 0.00122,
  "tokensUsed": 226
}
```

Only available, finite usage dimensions are included in `usage`. For example, `reasoning` is omitted when the model does not define separate reasoning pricing.

**Direct streaming response shape.** `streamText` and `Agent.stream()` return a different result type. Stream iterables (`textStream`, `fullStream`) provide real-time chunks, while scalar properties (`text`, `usage`, `finishReason`, etc.) are promises that resolve when the stream completes:

| Field          | Type                            | Description                                                         |
| -------------- | ------------------------------- | ------------------------------------------------------------------- |
| `textStream`   | `AsyncIterable<string>`         | Async iterable of text chunks                                       |
| `fullStream`   | `AsyncIterable<TextStreamPart>` | Async iterable of all stream events (text deltas, tool calls, etc.) |
| `text`         | `Promise<string>`               | Full text, resolved on completion                                   |
| `usage`        | `Promise<LanguageModelUsage>`   | Token counts, resolved on completion                                |
| `finishReason` | `Promise<FinishReason>`         | Why generation stopped, resolved on completion                      |
| `toolCalls`    | `Promise`                       | Tool calls made during streaming, resolved on completion            |
| `response`     | `Promise`                       | Raw provider response metadata                                      |
| `warnings`     | `Promise`                       | Any warnings from the provider                                      |

`streamText` / `Agent.stream()` `onFinish` receives the wrapped finish payload: `result` (alias for `text`), `cost` (`null` when pricing is missing), and merged `sources` (always an array).

## Prompt Files

Instead of hardcoding model config and messages in your code, you write `.prompt` files that live in your workflow's `prompts/` folder. See the [Prompts Guide](/prompts) for the full documentation.

```yaml prompts/generate_summary@v1.prompt theme={null}
---
provider: anthropic
model: claude-sonnet-4-20250514
temperature: 0.7
---

<system>
You write concise company summaries for sales teams.
</system>

<user>
Write a 2-3 paragraph summary of {{ companyName }}.

Industry: {{ industry }}
Company size: {{ size }} employees
</user>
```

### Configuration Options

| Option                                                 | Type                                                                                   | Description                                                                                                                                                                                                 |
| ------------------------------------------------------ | -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `provider`                                             | `string`                                                                               | `anthropic`, `openai`, `azure`, `amazon-bedrock`, `google-vertex`, `perplexity`, or a provider registered with `registerProvider`. Legacy aliases `bedrock` and `vertex` are deprecated but still accepted. |
| `model`                                                | `string`                                                                               | Model identifier                                                                                                                                                                                            |
| `temperature`                                          | `number`                                                                               | Sampling temperature (0.0-2.0)                                                                                                                                                                              |
| `maxTokens`                                            | `number`                                                                               | Maximum output tokens                                                                                                                                                                                       |
| `maxSteps`                                             | `number` (default 10 after `loadPrompt`)                                               | Tool-loop iterations when tools or skills are present                                                                                                                                                       |
| `skills`                                               | `string` or `string[]` in YAML; always `string[]` after `loadPrompt` (`[]` if omitted) | Skill file or directory paths, relative to this prompt                                                                                                                                                      |
| `tools`                                                | `object`                                                                               | Provider-specific tools (web search, etc.)                                                                                                                                                                  |
| `providerOptions`                                      | `object`                                                                               | Provider-specific options - see [ProviderOptions Guide](/prompts#provider-options)                                                                                                                          |
| `messageOptions`                                       | `object`                                                                               | Named per-message `providerOptions` sets                                                                                                                                                                    |
| `n`, `size`, `aspectRatio`, `seed`, `maxImagesPerCall` |                                                                                        | Image generation fields                                                                                                                                                                                     |

Unknown top-level keys throw `Invalid prompt file`. Snake\_case aliases of known fields (`max_tokens`) include a suggestion (`use "maxTokens"`). Put provider-specific keys such as `effort`, `reasoningEffort`, and `topP` under `providerOptions`.

## Providers

`@outputai/llm` ships built-in support for common AI SDK providers. The provider packages are peer dependencies with supported version ranges:

| Prompt `provider` | Peer dependency          | Supported range |
| ----------------- | ------------------------ | --------------- |
| `anthropic`       | `@ai-sdk/anthropic`      | `>=3 <4`        |
| `openai`          | `@ai-sdk/openai`         | `>=3 <4`        |
| `azure`           | `@ai-sdk/azure`          | `>=3 <4`        |
| `amazon-bedrock`  | `@ai-sdk/amazon-bedrock` | `>=4 <5`        |
| `google-vertex`   | `@ai-sdk/google-vertex`  | `>=4 <5`        |
| `perplexity`      | `@ai-sdk/perplexity`     | `>=3 <4`        |

Legacy aliases `bedrock` → `amazon-bedrock` and `vertex` → `google-vertex` are deprecated but still work; using them logs a deprecation warning.

Built-in provider instances are initialized lazily. Output creates the provider instance only when a prompt or API call first requests that provider, then reuses it for later calls.

### Anthropic

```yaml theme={null}
---
provider: anthropic
model: claude-sonnet-4-20250514
---
```

Requires `ANTHROPIC_API_KEY` environment variable.

### OpenAI

```yaml theme={null}
---
provider: openai
model: gpt-4o
---
```

Requires `OPENAI_API_KEY` environment variable.

### Azure OpenAI

```yaml theme={null}
---
provider: azure
model: gpt-4o
---
```

Requires `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, and `AZURE_OPENAI_API_VERSION`.

### Google Vertex AI

```yaml theme={null}
---
provider: google-vertex
model: gemini-1.5-pro
---
```

Requires Google Cloud authentication and configuration. The legacy alias `vertex` is still accepted.

### Amazon Bedrock

```yaml theme={null}
---
provider: amazon-bedrock
model: anthropic.claude-sonnet-4-20250514-v1:0
---
```

Requires AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) or IAM role-based authentication. Set `AWS_SESSION_TOKEN` when using temporary credentials (e.g., from `aws sts assume-role`). The legacy alias `bedrock` is still accepted.

For cross-region inference, use the regional inference profile format: `us.anthropic.claude-sonnet-4-20250514-v1:0`.

Always set `maxTokens` in your Bedrock prompt files. Unlike the direct Anthropic provider (which auto-detects per-model limits), the Bedrock SDK has no client-side defaults and relies on server-side defaults that may be lower than the model's capacity.

When using `providerOptions`, use the AI SDK `bedrock` namespace (not `anthropic`):

```yaml theme={null}
providerOptions:
  bedrock:
    guardrailConfig:
      guardrailIdentifier: my-guardrail
      guardrailVersion: "1"
```

### Custom Providers

Use `registerProvider` when you want prompt files to reference an AI SDK provider that is not built in, or when you need a custom provider instance:

```typescript theme={null}
import { createVertexAnthropic } from '@ai-sdk/google-vertex/anthropic';
import { registerProvider } from '@outputai/llm';

registerProvider('vertex-anthropic', createVertexAnthropic({
  project: process.env.GOOGLE_VERTEX_PROJECT,
  location: process.env.GOOGLE_VERTEX_LOCATION
}));
```

Then use the registered provider name in prompt front matter:

```yaml theme={null}
---
provider: vertex-anthropic
model: claude-haiku-4-5
---
```

Built-in providers use Output's default fetch configuration, including longer Undici response timeouts for LLM calls that take time before returning headers or body chunks. Custom registered providers are used exactly as you register them; they do not automatically receive that custom fetch. If your custom provider also needs longer network timeouts, configure its provider instance directly.

LLM cost estimation looks up rates by `provider` + `model` in the [models.dev](https://models.dev) catalog. Built-in provider names match that catalog. A custom `registerProvider` name (such as `vertex-anthropic` above) will not match, so `response.cost` is `null` and a missing-cost warning is logged - register under a models.dev provider id if you need automatic pricing.

## Prompt Caching

When a prompt sends the same large prefix on every call - a long system prompt, few-shot examples, a pasted reference document - you can cache that prefix so the provider skips reprocessing it. Cached input is about 90% cheaper and faster to first token. How you enable it depends on the provider.

### Anthropic

Anthropic caches only what you explicitly mark. Define a `cacheControl` set in `messageOptions` and attach it - with `options` - to the block that ends your static prefix. Everything up to and including that block is cached and reused on the next call:

```text prompts/generate_summary@v1.prompt theme={null}
---
provider: anthropic
model: claude-sonnet-4-20250514
messageOptions:
  cached:
    anthropic:
      cacheControl:
        type: ephemeral
---

<system options="cached">
You write concise company summaries for sales teams. Follow this style guide:
{{ style_guide }}
</system>

<user>
Summarize {{ companyName }}.
</user>
```

Only the `<user>` block - the part that changes each call - is re-billed at full price; the cached `<system>` prefix is charged at the much cheaper cache-read rate. For the 1-hour cache instead of the default 5 minutes, add `ttl: 1h` under `cacheControl`. A block can reference several sets (`options="cached fast"`), and a set can be reused across blocks. Bare `options` and names missing from `messageOptions` throw when the prompt loads.

<Warning>
  Attach the set to the last **static** block, never one containing per-call `{{ variables }}`. A breakpoint on changing content rewrites the cache on every call and never gets a hit. Order your blocks static-first, dynamic-last.
</Warning>

Each set is a provider-namespaced `providerOptions` object - the same shape and [namespace rules](/prompts#provider-options) as prompt-file `providerOptions`. On Vertex with a Claude model, use the same `anthropic` namespace.

### OpenAI

OpenAI caches automatically - there are no breakpoints to set, so the `messageOptions` mechanism above isn't needed. Any prompt of 1024 tokens or longer is cached for you, with no markup. To improve hit rates across calls, set a stable `promptCacheKey` (and, on GPT-5.1+, extend retention) via `providerOptions`:

```yaml prompts/enrich_company@v1.prompt theme={null}
---
provider: openai
model: gpt-5
providerOptions:
  openai:
    promptCacheKey: enrich-company-v1
    promptCacheRetention: 24h
---

<system>
{{ enrichment_playbook }}
</system>

<user>
Enrich {{ company }}.
</user>
```

### Confirming a cache hit

Cache activity appears in the response usage and the [cost event](/costs/cost-events): the first call reports cache-creation tokens, and later calls within the TTL report cache-read tokens (`cachedInputTokens`), already priced at the cheaper rate in `response.cost`.

<Note>
  Anthropic caches only prefixes above a model-specific minimum - around 1,024 tokens for most Sonnet and Opus models, higher for some. Shorter prefixes are silently not cached, with no error. A request supports at most four cache breakpoints.
</Note>

## Provider Tools

Many providers offer built-in tools like web search. Configure them in YAML front matter:

```yaml prompts/research@v1.prompt theme={null}
---
provider: google-vertex
model: gemini-2.0-flash
tools:
  googleSearch:
    mode: MODE_DYNAMIC
    dynamicThreshold: 0.8
---

<user>
Research {{ topic }} and provide sources
</user>
```

This is equivalent to calling the Vertex provider's `tools.googleSearch({ mode: 'MODE_DYNAMIC', dynamicThreshold: 0.8 })` at the code level, but keeps your prompt self-contained.

YAML tools are merged with code-level tools, so you can combine provider tools (from YAML) with custom tools (from code). Code-level tools take precedence if names conflict.

For provider-specific tool options, see:

* [Vertex AI Tools](https://sdk.vercel.ai/docs/ai-sdk-providers/google-vertex#provider-instance-methods)
* [OpenAI Tools](https://sdk.vercel.ai/docs/ai-sdk-providers/openai#provider-instance-methods)
* [Anthropic Tools](https://sdk.vercel.ai/docs/ai-sdk-providers/anthropic#provider-instance-methods)
* [Amazon Bedrock Tools](https://sdk.vercel.ai/docs/ai-sdk-providers/amazon-bedrock#provider-instance-methods)

## Tool Calling

Use tools with `generateText` to enable function calling:

```typescript theme={null}
import { generateText, aiSdk } from '@outputai/llm';
import { z } from '@outputai/core';

const { result, toolCalls } = await generateText({
  prompt: 'agent@v1',
  variables: { task: 'Research competitor pricing' },
  tools: {
    searchWeb: aiSdk.tool({
      description: 'Search the web for information',
      inputSchema: z.object({ query: z.string() }),
      execute: async ({ query }) => fetchSearchResults(query)
    })
  },
  toolChoice: 'auto'
});
```

## Call arguments

`variables` accepts Liquid values, including nested objects and arrays. Use dot notation and Liquid loops to read structured values in the prompt template.

### Text APIs

| Argument      | `generateText` | `generateTextWithStreaming` | `streamText` |
| ------------- | -------------- | --------------------------- | ------------ |
| `prompt`      | required       | required                    | required     |
| `promptDir`   | optional       | optional                    | optional     |
| `variables`   | optional       | optional                    | optional     |
| `tools`       | optional       | optional                    | optional     |
| `output`      | optional       | optional                    | optional     |
| `toolChoice`  | optional       | optional                    | optional     |
| `stopWhen`    | optional       | optional                    | optional     |
| `abortSignal` | optional       | optional                    | optional     |
| `onChunk`     | -              | optional                    | optional     |
| `onFinish`    | -              | -                           | optional     |
| `onError`     | -              | -                           | optional     |

`toolChoice`, `stopWhen`, and prompt `maxSteps` apply only when tools exist. With tools, an explicit `stopWhen` takes precedence; otherwise Output uses `aiSdk.stepCountIs(maxSteps)` from the prompt (default 10).

### generateImage

| Argument      | Required | Description                                  |
| ------------- | -------- | -------------------------------------------- |
| `prompt`      | yes      | Prompt file name                             |
| `promptDir`   | no       | Override the stack-resolved prompt directory |
| `variables`   | no       | Template variables                           |
| `images`      | no       | Source images for image-to-image             |
| `mask`        | no       | Inpainting mask; requires `images`           |
| `abortSignal` | no       | Cancel the request                           |

### Agent

Constructor options are set on `new Agent(...)`. `generate()`, `generateWithStreaming()`, and `stream()` take an optional args object (or omit it). `messages` defaults to `[]`.

| Argument       | `new Agent` | `.generate` | `.generateWithStreaming` | `.stream` |
| -------------- | ----------- | ----------- | ------------------------ | --------- |
| `prompt`       | required    | -           | -                        | -         |
| `promptDir`    | optional    | -           | -                        | -         |
| `variables`    | optional    | -           | -                        | -         |
| `tools`        | optional    | -           | -                        | -         |
| `output`       | optional    | -           | -                        | -         |
| `stopWhen`     | optional    | -           | -                        | -         |
| `messageStore` | optional    | -           | -                        | -         |
| `messages`     | -           | optional    | optional                 | optional  |
| `abortSignal`  | -           | optional    | optional                 | optional  |
| `toolChoice`   | -           | optional    | optional                 | optional  |
| `onChunk`      | -           | -           | optional                 | optional  |
| `onFinish`     | -           | -           | -                        | optional  |
| `onError`      | -           | -           | -                        | optional  |

## Retries and Network Timeouts

Output always sets AI SDK `maxRetries` to 0. In Output workflows, LLM calls usually run inside steps, and steps are Temporal activities. When a provider error fails the step, Temporal records the failed activity attempt and retries it according to the workflow's retry policy.

`generateText`, `generateTextWithStreaming`, `Agent.generate()`, and `Agent.generateWithStreaming()` reject on failures. With direct `streamText` or `Agent.stream()` usage, capture the error in `onError` and throw it after consuming the stream, as shown in their direct-stream examples.

Built-in providers are initialized with a custom fetch that extends Undici's `headersTimeout` and `bodyTimeout` to 15 minutes. This helps long-running LLM responses where the provider accepts the request but takes longer to return response headers or body chunks, for example reasoning-heavy calls. Active cancellation still works: if you pass `abortSignal`, or the AI SDK/provider aborts the request, that cancellation wins.

## LLM call cost event

Each completed text generation call emits a `cost:llm:request` event after the LLM responds and cost can be computed. For direct streams, the event is emitted when the stream finishes. You can observe it with the same [hooks mechanism](/packages/core#error-hooks) as error hooks: register a handler with `on('cost:llm:request', handler)` from `@outputai/core/hooks` in a hook file listed under `outputai.hookFiles`. The handler receives an event envelope whose `payload` field is the same LLM usage attribute exposed on `response.cost`. For payload details, see [Cost Events](/costs/cost-events).

## loadPrompt

Load and render a prompt file without generating - useful for debugging:

```typescript theme={null}
import { loadPrompt } from '@outputai/llm';

const prompt = loadPrompt('generate_summary@v1', {
  companyName: 'Acme Corp',
  industry: 'SaaS',
  size: 250
});

console.log(prompt.config);       // includes skills: string[] and maxSteps (default 10)
console.log(prompt.messages);     // PromptMessage[] with system, user, or assistant roles
console.log(prompt.instructions); // string | null
console.log(prompt.variables);    // { companyName: 'Acme Corp', industry: 'SaaS', size: 250 }
```

For instruction-mode prompts, `messages` is empty and `instructions` contains the rendered body. For message-mode prompts, `messages` contains the parsed role blocks and `instructions` is `null`. See [Prompt Body Modes](/prompts#prompt-body-modes).

`PromptMessage.role` is typed as `'system' | 'user' | 'assistant'`, matching the authored role blocks accepted in message mode.

## API Reference

For complete TypeScript API documentation, see the [LLM Module API Reference](https://output-ai-reference-code-docs.onrender.com/modules/llm_src.html).