Limiting and Timing Out Calls

Capping how often a tool runs

Exporting tickets reads every ticket in the database. The help desk wants at most two exports an hour, so the tool sets rateLimit. Open the Tests tab:

import { Tool, ToolContext } from "@frontmcp/sdk";
import { tickets } from "./store";

@Tool({
  name: "export_tickets",
  description: "Export every support ticket as CSV. Reads the whole ticket database, so at most 2 exports an hour.",
  inputSchema: {},
  annotations: { readOnlyHint: true },
  rateLimit: { maxRequests: 2, windowMs: 60 * 60_000 },
})
export class ExportTickets extends ToolContext {
  async execute() {
    const rows = tickets.map((t) => `${t.id},${t.title},${t.status}`);
    return { csv: ["id,title,status", ...rows].join("\n") };
  }
}

The export above ran once when the example started. Press Call in the Call tab twice more: the second call works, the third is refused. rateLimit takes:

What the caller gets back

A call over the limit doesn't reach execute(). The client gets an ordinary tool error:

{
  "content": [{ "type": "text", "text": "Rate limit exceeded. Retry after 2274 seconds" }],
  "isError": true,
  "_meta": { "errorId": "err_…", "code": "RATE_LIMIT_EXCEEDED" }
}

• The retry hint is in the text, for the model. The HTTP status is still 200 and there's no Retry-After header, so the model is the one that reads when to try again. The message survives in production too.
• The wait is until the window ends. Windows line up with the clock, not with the first call: an hour-long window runs from one full hour (in UTC) to the next, and a minute-long one from one full minute to the next. The number is the time left in the current window, so it can be anything from a second up to the whole windowMs. Refused at 10:40 UTC by an hourly limit, a caller is told to retry in about 1200 seconds.

Say the limit in the description as well, as export_tickets does ("at most 2 exports an hour"). A model that knows the rule can plan around it, or tell the user, instead of finding out from an error.

Whose calls a limit counts

partitionBy decides which calls share a count:

"userId" is the natural choice for "two exports per person", and for signed-in callers it does exactly that. Anonymous callers have no user id, so they all share one count:

import { Tool, ToolContext } from "@frontmcp/sdk";

@Tool({
  name: "export_tickets",
  description:
    "Export every support ticket as CSV. At most 2 exports an hour per signed-in caller, and 2 an hour for all anonymous callers together.",
  inputSchema: {},
  annotations: { readOnlyHint: true },
  rateLimit: { maxRequests: 2, windowMs: 60 * 60_000, partitionBy: "userId" },
})
export class ExportTickets extends ToolContext {
  async execute() {
    return { csv: "id,title,status\nT-1,Cannot log in,open" };
  }
}

Each of these calls arrives as a new anonymous caller with a new id, but FrontMCP doesn't count anonymous callers by those ids, so the third one is refused like the third call from a single person. That's the safe choice: nobody gets around a limit by not signing in. It also means one busy anonymous client can use up the exports for every other anonymous client. If that matters, have callers sign in, so each gets a count of their own, or, on a real server, count anonymous callers by address with a function: partitionBy: (ctx) => ctx.userId ?? ctx.clientIp ?? "anonymous".

Counts are kept in the server's memory unless you say otherwise, so each instance of a server counts on its own: two instances behind a load balancer let through four exports an hour, not two. With throttle.storage on one Redis, the instances share a count: five exports sent to two instances in turn let two through. Running Several Instances sets that up. If that Redis can't be reached, a production server doesn't start, unless throttle.storage has fallback: "memory", which starts it with each instance counting on its own. When Redis goes away while the server runs, a call to a tool with a limit fails with GUARD_STORAGE_UNAVAILABLE, or, with the fallback, is counted in the instance's memory until Redis answers again: the guard reference shows what a caller gets, and what happens at startup.

One at a time

Merging two customer records moves all their tickets from one record to the other. Two merges running at once can move the same tickets twice. concurrency caps how many calls of a tool run at the same moment:

import { Tool, ToolContext, z } from "@frontmcp/sdk";

export const merges: string[] = [];

@Tool({
  name: "merge_customers",
  description:
    "Merge one customer record into another, moving all its tickets. One merge runs at a time; if another is running, wait a minute and try again.",
  inputSchema: { from: z.string().describe("Customer to merge away, like C-7"), into: z.string().describe("Customer to keep") },
  annotations: { destructiveHint: true },
  concurrency: { maxConcurrent: 1 },
})
export class MergeCustomers extends ToolContext {
  async execute({ from, into }: { from: string; into: string }) {
    await new Promise((resolve) => setTimeout(resolve, 300)); // moving tickets takes a while
    merges.push(`${from} → ${into}`);
    return { merged: from, into };
  }
}

The slot is taken when a call starts and given back when its execute() ends, whether it succeeded or failed. concurrency takes maxConcurrent, partitionBy (the same choices as for rateLimit, "global" by default), and queueTimeoutMs. By default, a call that finds no free slot is refused at once. With a queue, it waits for a slot, and is only refused if none frees up in time:

concurrency: { maxConcurrent: 1, queueTimeoutMs: 5_000 },

A queued call that runs out of time gets Queue timeout for "merge_customers" after waiting 5000ms for a concurrency slot. For a tool a model is likely to call twice in a row, a short queue is kinder than a refusal.

Every limit answers the same way: an isError tool result, over HTTP 200, with a code of its own in _meta.code and a message that is kept in production too:

So the model can tell "busy, try again soon" from "broken". It still helps to put the rule in the tool's description, as merge_customers does, so the model knows what to do about it.

Deadlines

A tool that calls another service waits as long as that service takes. timeout: { executeMs } puts a limit on it. Customer C-9 sends the CRM into a slow path:

import { Tool, ToolContext, z } from "@frontmcp/sdk";

// Stands in for a CRM that is sometimes very slow to answer.
async function crmLookup(id: string) {
  await new Promise((resolve) => setTimeout(resolve, id === "C-9" ? 2_000 : 50));
  return { id, name: id === "C-9" ? "Globex" : "Initech", plan: "business" };
}

@Tool({
  name: "lookup_customer",
  description: "Look up a customer in the CRM by id. Gives up after half a second if the CRM is slow.",
  inputSchema: { id: z.string().describe("Customer id, like C-1") },
  annotations: { readOnlyHint: true, openWorldHint: true },
  timeout: { executeMs: 500 },
})
export class LookupCustomer extends ToolContext {
  async execute({ id }: { id: string }) {
    return crmLookup(id);
  }
}

After half a second the call fails with Execution of "lookup_customer" timed out after 500ms. Try C-1: it answers in time.

A timeout ends the call, not the work

When a call times out, FrontMCP stops waiting for execute() and answers the client. execute() itself carries on unless it's written to stop (the next section shows how), side effects and all. Its concurrency slot stays taken until it really ends, so runs never overlap, and a call that arrives in the meantime is refused:

import { Tool, ToolContext, z } from "@frontmcp/sdk";

const merges: string[] = [];
let running = 0;
let mostAtOnce = 0;

@Tool({
  name: "merge_customers",
  description: "Merge one customer record into another, moving all its tickets. One merge runs at a time.",
  inputSchema: { from: z.string(), into: z.string() },
  annotations: { destructiveHint: true },
  concurrency: { maxConcurrent: 1 },
  timeout: { executeMs: 200 },
})
export class MergeCustomers extends ToolContext {
  async execute({ from, into }: { from: string; into: string }) {
    running++;
    mostAtOnce = Math.max(mostAtOnce, running);
    await new Promise((resolve) => setTimeout(resolve, 500)); // takes longer than the timeout
    merges.push(`${from} → ${into}`);
    running--;
    return { merged: from, into };
  }
}

@Tool({ name: "merge_log", description: "List finished merges.", inputSchema: {} })
export class MergeLog extends ToolContext {
  async execute() {
    return { merges, mostAtOnce };
  }
}

For a tool that changes things, a timeout shorter than the work is worse than none. The model is told the merge failed while it's still going, then that merges are busy, and it may well run the same merge again once the first one is done. Give such tools no timeout, or one comfortably longer than the work can take, and make them safe to repeat, for example by checking whether the merge already happened before starting it.

Stopping the work

A timeout also aborts this.signal, the call's AbortSignal. Work that listens to it stops with the call. fetch() and most clients for databases and other services take a signal option, and a loop can call this.signal?.throwIfAborted() between steps. (The type says signal may be missing, hence the ?..) This export reads the ticket database a page at a time, and a slow database makes it take longer than its timeout:

import { Tool, ToolContext } from "@frontmcp/sdk";

export const pagesRead: number[] = [];
export const stops: string[] = [];

// Stands in for a slow database: 10 pages, a tenth of a second each.
async function readPage(page: number) {
  await new Promise((resolve) => setTimeout(resolve, 100));
  pagesRead.push(page);
  return [`T-${page},Ticket ${page},open`];
}

@Tool({
  name: "export_tickets",
  description: "Export every support ticket as CSV. Gives up after 350 ms.",
  inputSchema: {},
  annotations: { readOnlyHint: true },
  timeout: { executeMs: 350 },
})
export class ExportTickets extends ToolContext {
  async execute() {
    const rows: string[] = [];
    try {
      for (let page = 1; page <= 10; page++) {
        this.signal?.throwIfAborted(); // ✅ stop between pages once the call has timed out
        rows.push(...(await readPage(page)));
      }
    } catch (error) {
      stops.push(String(this.signal?.reason?.message ?? error));
      throw error;
    }
    return { csv: ["id,title,status", ...rows].join("\n") };
  }
}

@Tool({ name: "export_stats", description: "How many database pages exports have read, and why they stopped.", inputSchema: {} })
export class ExportStats extends ToolContext {
  async execute() {
    return { pagesRead: pagesRead.length, stops };
  }
}

Take out the throwIfAborted() line and run the test again: the call still times out after 350 ms, but the export goes on to read all ten pages that nobody will see. this.signal.reason is the same timeout error the client got. Stopping is right for work that only reads. For work that changes things, stopping halfway can be worse than finishing, which is why the merge above doesn't listen to the signal.

Turning limits off

The limits in @Tool are enforced by FrontMCP's guard, which starts by itself when a tool declares rateLimit or concurrency. @FrontMcp's throttle option configures the guard, and throttle: { enabled: false } turns it off: every rateLimit and concurrency on the server is ignored. That can be what you want for a local test run that calls the same tool many times:

import { App, FrontMcp } from "@frontmcp/sdk";
import { ExportTickets } from "./export-tickets.tool";

@App({ id: "help-desk", name: "Help Desk", tools: [ExportTickets] })
class HelpDeskApp {}

@FrontMcp({
  info: { name: "help-desk", version: "1.0.0" },
  apps: [HelpDeskApp],
  throttle: { enabled: false }, // 🚩 for local testing only
})
export default class Server {}

The description still says "at most 2 exports an hour", and nothing enforces it, so keep enabled: false out of anything a real client can reach. timeout is the one limit it doesn't turn off.

Limits for every tool

throttle can also set defaults, for tools that don't set their own. Unlike a tool's own limits, throttle needs enabled: true:

@FrontMcp({
  info: { name: "help-desk", version: "1.0.0" },
  apps: [HelpDeskApp],
  throttle: {
    enabled: true,
    defaultRateLimit: { maxRequests: 60, windowMs: 60_000, partitionBy: "ip" },
    defaultConcurrency: { maxConcurrent: 5 },
    defaultTimeout: { executeMs: 30_000 },
  },
})
export default class Server {}

A tool's own rateLimit, concurrency or timeout replaces the default. The @FrontMcp reference shows defaultRateLimit running.

Choosing limits

Whatever you choose, write it in the description. Limits the model knows about are limits it can work with.

Recap

Try some challenges

Each challenge runs hidden checks against your code. Edit the code, then press Check.

import { App, FrontMcp, Tool, ToolContext } from "@frontmcp/sdk";

@Tool({
  name: "export_tickets",
  description: "Export every support ticket as CSV.",
  inputSchema: {},
  annotations: { readOnlyHint: true },
})
class ExportTickets extends ToolContext {
  async execute() {
    return { csv: "id,title,status\nT-1,Cannot log in,open\nT-2,Invoice total is wrong,closed" };
  }
}

@App({ id: "help-desk", name: "Help Desk", tools: [ExportTickets] })
class HelpDeskApp {}

@FrontMcp({
  info: { name: "help-desk", version: "1.0.0" },
  apps: [HelpDeskApp],
})
export default class Server {}