Agent Engine in Your App

Writing an agent loop that holds up in production is hard work. You need multi-step tool dispatch, history compaction before the context window fills up, spend limits per turn, retry logic that can tell a rate limit apart from a bad request, and loop detection for when the model calls the same tool nine times in a row. None of that is your actual product, but all of it sits between you and shipping it.

stella already does all of this, and you can run it inside your own app without giving up control of your models, your tools, or your data.

What each side owns

Engine asks, app answers

Understand this before you write any code, because it flips the usual shape of an agent loop.

stella's engine does not call your tools. It asks you to call them, then waits. It's the same for the model: the engine doesn't hold an API key and doesn't make an HTTP request itself. It sends a request and waits until your app answers.

This matters for two reasons:

The engine can't leak what it doesn't have. stella-serve depends on only two crates, stella-protocol and stella-core. It has no HTTP client, no TLS stack, and no provider adapters. It can't reach the network or read a provider credential, so a bug in it can't leak your keys. That's guaranteed by what the code depends on, not just a promise.

Your tests don't need an API key. Since your app plays the role of the model, you can drive a full multi-step turn from a unit test using a scripted reply. That gives you full agent-loop test coverage, with no network calls, no spend, in milliseconds.

Pick an integration shape

The choice changes your licensing obligations, so make it before you build.

The sidecar is the shape we recommend for a GenAI app, for engineering reasons: process isolation, a host you can write in TypeScript or Python, and an engine that never shares memory with your application code.

It's not automatically the cheaper option for licensing, though. Linking the crates into a closed-source product clearly needs a commercial license. Whether the sidecar does depends on what you do with it, and the section below spells out exactly which facts decide that. Don't read "sidecar" as "free."

Step 1: Build a serve-capable binary

There's no stella serve subcommand, and no release download contains the server. stella-serve is a separate crate with its own binary, and the release build only packages the CLI. Build it yourself:

git clone https://github.com/macanderson/stella
cd stella
cargo build --release -p stella-serve --bin stella-serve

This is a fast build. The server pulls in only stella-protocol and stella-core, not the CLI, the terminal interface, or the tool crates.

Confirm what you built:

./target/release/stella-serve --version
# stella-serve <version>   — the program name, one space, a bare semver

Record that version in your repo next to your client code. The wire format only adds fields over time, but pinning a version plus a round-trip test is how you find out about a change on your own schedule instead of in production.

Step 2: Start the server

Every setting is an environment variable. There are no flags beyond healthcheck, --version, and --help.

export STELLA_SERVE_BIND=127.0.0.1:8137
export STELLA_SERVE_TOKEN="$(openssl rand -base64 32)"
export STELLA_SERVE_TOOLS=remote
./target/release/stella-serve
# stella-serve listening on 127.0.0.1:8137

In tests, bind port 0 and read the actual port back from the listening on line on stdout. Picking a random port and hoping it's free is a common cause of flaky integration tests.

Step 3: The six routes this walkthrough uses

This is the whole surface this walkthrough touches, not everything stella-serve offers. See the full route reference below for the rest.

A turn is the top-level resource in this walkthrough. Your app owns the conversation history and sends it with each turn. The server also offers an alternative where it owns the conversation instead: POST /v1/sessions, covered in the full route surface below, where the engine keeps the transcript for you. This guide sticks to the turn-only path, because it leaves your app with no state to track beyond the turn itself.

Six frame types arrive on the event stream, tagged by a type field:

The full route surface

The six routes above cover this walkthrough completely, but stella-serve ships many more. For the full list, see the module documentation at the top of crates/stella-serve/src/server.rs and the route list in crates/stella-serve/src/observe/event.rs:

Any other method on a known path returns a 405 with an Allow header. Any other path returns a 404.

Step 4: Write the host

This is a complete host. It runs on Node 18 or newer with no dependencies. Save it as host.mjs.

const BASE = process.env.STELLA_BASE_URL ?? "http://127.0.0.1:8137";
const TOKEN = process.env.STELLA_SERVE_TOKEN;
const H = {
  authorization: `Bearer ${TOKEN}`,
  "content-type": "application/json",
};

// 1. Your model. Swap the body of this function for your gateway call.
async function runModel(request, callNumber) {
  if (callNumber === 1) {
    return {
      text: "",
      tool_calls: [
        { call_id: "c1", name: "get_weather", input: { city: "Paris" } },
      ],
      usage: { input_tokens: 42, output_tokens: 17 },
      model: "your-model-id",
      cost_usd: 0.0002,
      finish_reason: "tool_calls", // not "tool_use"
    };
  }
  return {
    text: "It is 18C and clear in Paris.",
    usage: { input_tokens: 96, output_tokens: 12 },
    model: "your-model-id",
    cost_usd: 0.0003,
    finish_reason: "stop",
  };
}

// 2. Your tools. Run them through whatever authorization your app already has.
async function runTool(name, input) {
  if (name === "get_weather") {
    return { ok: { content: `18C, clear skies in ${input.city}.` } };
  }
  return { error: { message: `no such tool: ${name}` } };
}

// 3. Start the turn.
const created = await fetch(`${BASE}/v1/turns`, {
  method: "POST",
  headers: H,
  body: JSON.stringify({
    provider_id: "my-app",
    tools: [
      {
        name: "get_weather",
        description: "Look up the current weather for a city.",
        input_schema: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
        // read_only lets the engine run this concurrently with other
        // reads. Do NOT also claim speculation_safe here: a speculated
        // call can run twice per step, and a metered lookup like weather
        // is not free to bill twice. Leave it unset (false) for anything
        // network-backed.
        read_only: true,
      },
    ],
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "What is the weather in Paris?" },
    ],
    budget: { mode: "off" },
    max_steps: 10,
    reverse_request_timeout_ms: 30000,
  }),
});
if (!created.ok) throw new Error(`create failed: ${created.status}`);
const { turn_id } = await created.json();

// 4. Stream frames and answer what the engine asks for.
const stream = await fetch(`${BASE}/v1/turns/${turn_id}/events`, { headers: H });
const post = (route, body) =>
  fetch(`${BASE}/v1/turns/${turn_id}/${route}`, {
    method: "POST",
    headers: H,
    body: JSON.stringify(body),
  });

let modelCalls = 0;
let buffer = "";
const decoder = new TextDecoder();

for await (const chunk of stream.body) {
  buffer += decoder.decode(chunk, { stream: true });
  let split;
  while ((split = buffer.indexOf("\n\n")) !== -1) {
    const record = buffer.slice(0, split);
    buffer = buffer.slice(split + 2);
    if (!record.startsWith("data: ")) continue;
    const frame = JSON.parse(record.slice(6));

    if (frame.type === "event") {
      console.log("  event:", frame.event.type);
    } else if (frame.type === "provider_request") {
      const result = await runModel(frame.request, ++modelCalls);
      await post("provider-result", {
        request_id: frame.request_id,
        status: "ok", // sits beside request_id, not nested
        result,
      });
    } else if (frame.type === "tool_request") {
      const output = await runTool(frame.name, frame.input);
      await post("tool-result", { request_id: frame.request_id, output });
    } else if (frame.type === "turn_complete") {
      console.log("\noutcome:", frame.outcome);
      process.exit(frame.outcome.status === "completed" ? 0 : 1);
    }
  }
}

Run it against the server you started in step 2:

node host.mjs

  event: stage
  event: budget_tick
  event: block_registered
  event: block_registered
  event: step_manifest
  event: step_usage
  event: tool_start
  event: tool_result
  event: budget_tick
  event: block_registered
  event: block_registered
  event: step_manifest
  event: step_usage
  event: text
  event: stage
  event: complete

outcome: {
  status: 'completed',
  text: 'It is 18C and clear in Paris.',
  cost_usd: 0.0005
}

You can read the loop right in that output. The first step_usage is your first model call. tool_start and tool_result are the tool running. The second budget_tick and step_usage are the second model call, and text is the final answer. The block_registered and step_manifest events are the context engine keeping track of what went into each prompt.

Two model calls, one tool call, one final cost. cost_usd is the sum of what your app reported on each model call. That's why your billing system stays the authority, and the engine is just reporting numbers to it.

What the engine handled

Look at the conversation the engine handed you on the second model call:

[
  { "role": "system",    "content": "You are a helpful assistant." },
  { "role": "user",      "content": "What is the weather in Paris?" },
  { "role": "assistant", "tool_calls": [
      { "call_id": "c1", "name": "get_weather", "input": { "city": "Paris" } } ] },
  { "role": "tool",      "tool_results": [
      { "call_id": "c1",
        "output": { "ok": { "content": "18C, clear skies in Paris." } } } ] }
]

It recorded the assistant's tool call, matched your result to it using call_id, and wove both back into the history in the shape your provider expects. That's the whole loop. Everything after this point is just the same four frame types, repeated.

Step 5: Wire in your real model and tools

These changes take the example to production.

Your model. Replace runModel with your gateway call. Map your provider's response onto CompletionResult. usage, model, and cost_usd are required fields, and finish_reason must be stop, length, tool_calls, or content_filter. Anthropic's tool_use spelling gets rejected with a 400 error that names the valid values.

Your tools. Replace runTool with a dispatch into the functions your app already exposes, behind the authorization you already enforce. Return { ok: { content } } or { error: { message } }. Send failures through the error field instead of throwing an exception: the model reads that text and can often correct itself, which is usually what you want.

Your errors. When a model call fails, say why. The engine's retry behavior depends on the error type you report:

await post("provider-result", {
  request_id: frame.request_id,
  status: "error",
  error: { kind: "rate_limited", message: "slow down", retry_after_ms: 500 },
});

transport and rate_limited get retried with backoff. auth, unknown_model, malformed, cancelled, and terminal fail the turn right away. If you report a rate limit as terminal, you silently lose the backoff you wanted.

Production checklist

• One process per trust boundary. Several provider and sandbox settings apply to the whole process, so running multiple tenants in one process isn't safe. Run one engine per tenant, inside whatever isolation you already use for untrusted code.
• Keep the port private. Bind it to loopback, or to a private network behind the token. There's no Host header check and no CORS handling, so never expose it directly to a browser.
• Set a reverse-request deadline. reverse_request_timeout_ms limits how long the engine waits for your app to respond. It defaults to five minutes and can't be set above one hour. If a model request times out, the turn fails. If a tool request times out, it becomes a tool error the model can react to.
• Cancel turns you abandon. Each live turn holds onto an OS thread. POST /v1/turns/{id}/cancel releases it. The turn shuts down cleanly rather than getting killed outright, so a client still streaming events gets a final frame with an aborted outcome and its final cost.
• Handle 429s. At most 32 turns can run at once. Go over that limit and you get a 429 response with Retry-After: 5.
• The event stream allows only one subscriber. A second GET on the same turn's events returns 409. If you need to fan updates out to multiple users, do it from your own app, not by opening the stream a second time.
• Record step_usage events. They carry per-step tokens, cost, model, and duration, which is exactly what you need to cross-check your own metering.

Not built yet

These are the limits today, so plan around them instead of hitting them by surprise:

• A resume point can expire. Frames are sequenced and can be replayed (see below), but only the most recent 4,096 frames per live turn are kept. If you ask to resume from a seq older than that window, you're told the point is gone, instead of getting a stream with a silent gap in it. Save frames yourself as you receive them if you need durability beyond that window.
• No way to skip /approve. The route exists, but a turn that raises scope_review blocks until something answers it. If you're running unattended, set auto_approve on the turn. Otherwise, your app has to answer, and if it never does, the turn stays stuck until its deadline passes.
• retry_policy and loop_detection are set by whoever runs the server, not by whoever calls it. They limit what one request can cost the process, so no per-turn setting can widen them. Unknown keys in the engine object get rejected instead of silently ignored, so a typo in a setting name fails loudly instead of quietly doing nothing.

Other capabilities

• Stream resumption. Use ?after=<seq>, or rely on an EventSource's automatic Last-Event-ID. Every frame carries an increasing seq, and each SSE frame carries an id:.
• Server-side conversation state. POST /v1/sessions, then POST /v1/sessions/{id}/turns. The server keeps one transcript for you, so the unchanging prefix earns its cache discount and compaction becomes possible.
• The approval route. POST /v1/turns/{id}/approve answers a scope_review by request_id.
• Token-level streaming. POST /v1/turns/{id}/provider-delta sends batched text and reasoning fragments for a completion in progress. Treat it as advisory, the same as text_delta.
• Graceful shutdown, /readyz, /metrics. SIGTERM and SIGINT start a graceful shutdown, bounded by STELLA_SERVE_SHUTDOWN_GRACE_SECS, and both endpoints are live.
• Per-turn engine settings. An optional engine object on turn creation sets the output cap, temperature, effort, and more, each capped by a limit the server operator sets.

Licensing: AGPL or commercial

stella is dual-licensed. You choose which track applies to you.

No license needed

• Use stella as a developer tool at work, on proprietary code. Using stella to write closed-source software does not make that software AGPL. The license covers stella and things built from it, not the output of running it.
• Run stella unmodified, including on your own servers.
• Evaluate, benchmark, audit, or study it.
• Build on the Context Graph Protocol, a separate project licensed under MIT OR Apache-2.0 that stays that way. Depending on CGP does not put your project under the AGPL.

License required

• Ship stella, or anything built from it, inside a product whose source code you don't publish.
• Offer a modified version of stella to other people over a network without publishing your changes. This is covered by AGPL section 13, and hosting isn't a loophole.
• Keep a proprietary fork private, or license stella to your own customers under different terms.

Where the sidecar lands

Here's the direct answer, since this is the shape most GenAI apps end up choosing.

Linking stella-core into a closed-source binary is the clear case: that's a combined work, and it needs a commercial license. A developer running the CLI on their own machine is the other clear case, and needs nothing at all.

The sidecar sits between those two cases. The answer depends on facts about your deployment: whether you modified the engine, and whether your users interact with that modified engine over a network. If you run the binary unmodified and your app talks to it over HTTP, you're in very different territory than if you patched it and shipped the patched version as part of a hosted product.

A docs page can't guess at your specific architecture. Email licensing@oxagen.sh with a short description, and you'll get a straight answer about whether you need a license at all, including "you do not."

How to buy a commercial license

1. Email licensing@oxagen.sh. One message is all it takes to start. There's no portal and no sales process to go through.
2. Tell us what you're building and roughly how you'll deploy it. What the product does, whether stella is linked in or run as a sidecar, whether you modify it, and roughly how big it is (self-hosted or SaaS, number of environments, rough number of users). Pricing depends on scope and deployment size, so this is what determines the price.
3. Say what terms you need. A commercial license removes the obligations described above. Warranty, indemnity, and support terms, which the AGPL specifically rules out, are available, and it's worth mentioning them up front if your procurement process requires them.
4. Get an answer, including whether you need one at all. If your use is already fine under the AGPL, we'll tell you that instead of trying to sell you something.

Full terms and the reasoning behind the dual license are in LICENSING.md, including the note that every release up to v0.5.14 stays available under MIT OR Apache-2.0 for good.

Where to go next

• Drop stella in as your agent engine — a guide-shaped version of this page: which shape to pick, how to port an existing feature onto the engine in an afternoon, and an integration test that exercises the whole loop with no API key.
• Gate on engine quality in CI — once the engine is a dependency, run it side by side with Claude Code on every pull request and block merges on loop correctness.
• Agent Engine Paths — the command-line doors into the same engine.
• Event Stream Compatibility — the full AgentEvent vocabulary you receive in event frames.
• Plugins — what a wrapper adds around a step.
• Telemetry — what the engine reports, and how to record it.