Build your first AI agent, step by step

A first agent is a loop: take what the caller said, decide what they want, call a tool or two, say something back, and hand off when unsure. Build it in five steps, intent design, tools, guardrails, testing, then the phone and SMS hookup, and write each step down as you go. That written record is your spec pack.

What is an agent, in plain terms?

A program that runs a loop. Something comes in, a sentence from a caller or a text message. The loop works out what the person wants, which is the intent. It collects what it needs to act, which are the slots: a name, an address, a time. It calls a tool, a function in your code that reads or writes a real system. It says something back. When it is unsure, it hands off to a person with a summary.

The model sits in the middle of that loop and does two jobs: it reads language and it chooses. It does not book anything. Your code books things, on the model's request, after your code has checked the request. Keep that line clear from the first day and most of what follows is easy. Lose it and every bug is a mystery.

How do I design the intents?

Start from traffic, not from imagination. Pull a week of call recordings or message threads and, for each, write down in a few words what the person wanted. "Book a tune-up." "How much for a panel swap." "Where is the tech." "Water everywhere." Group the lines. You will end up with a short list, and short is the point: every intent you add is a set of test cases you now owe.

Keep two intents you did not find in the log. other is where everything unclear lands, and it routes to a person or a clarifying question, never to a tool. emergency is the one you never want the model to decide on its own.

For each intent, write three things: the slots you must have before acting, the tool the intent ends in, and what ends it. A booking ends with a job id read back to the caller. A quote request ends with a promise that a person will call, and the time. An emergency ends with a transfer.

Then write the router. The phrases you cannot afford to misread are matched by a rule before the model runs. Everything else goes to the model with a closed list of intents and a fallback, so the model can only ever answer with something on the list.

type Intent = "emergency" | "book" | "quote" | "status" | "other";
const INTENTS: readonly Intent[] = ["emergency", "book", "quote", "status", "other"];

// Anything the model must never get wrong is decided by a rule before the model runs.
const HARD_RULES: ReadonlyArray<[Intent, RegExp]> = [
  ["emergency", /\b(smell gas|gas smell|sparks|burst pipe|flooding|no heat)\b/],
  ["status", /\b(where is|running late|eta)\b/],
];

type Classifier = (text: string, intents: readonly Intent[]) => Promise<string>;

export async function routeIntent(utterance: string, classify: Classifier): Promise<Intent> {
  const text = utterance.trim().toLowerCase();
  for (const [intent, pattern] of HARD_RULES) {
    if (pattern.test(text)) return intent;
  }
  const guess = await classify(text, INTENTS);
  return (INTENTS as readonly string[]).includes(guess) ? (guess as Intent) : "other";
}

// A stand-in so the file runs without a model key. Replace it with a model call that is
// given the intent list and asked to return exactly one of them.
const keywordClassifier: Classifier = async (text) => {
  if (/\b(book|schedule|appointment|come out)\b/.test(text)) return "book";
  if (/\b(quote|estimate|how much|price)\b/.test(text)) return "quote";
  return "other";
};

const call = "Can someone come out Tuesday? Also, I smell gas.";
routeIntent(call, keywordClassifier).then(console.log);
// emergency: the hard rule wins even though the caller also asked for a booking

Save it as router.ts and run it with npx tsx router.ts. The stand-in classifier is there so the file runs on your laptop with no key; the shape of the model call that replaces it is the same, a text in, one intent from the list out.

What tools does the agent need?

A tool is a function with a typed input and a typed output that the model is allowed to ask for by name. A booking agent needs four. lookupCustomer(phone) returns who is calling and their history, or nothing. findSlots(serviceType, date) returns the open windows. bookJob(customerId, slot, notes) writes the job and returns its id. transferToHuman(reason, summary) moves the call and posts the summary where the person will see it.

Four rules keep tools from hurting you. Read tools run first and often; write tools run last and once. A write tool refuses to run until every slot for that intent is filled, and it returns an id you read back to the caller so both of you know it happened. Tools are allowed per intent, so the quote intent cannot call bookJob however the conversation goes. And every tool call is logged with its inputs and outputs, with phone numbers and addresses masked, because the log is how you will debug the call you were not on.

Where the tools point is the integration. For a service business that is the API of whatever runs the schedule, ServiceTitan or Jobber for example, and the write tool is a job or an appointment created in that system while the caller is still on the line. Build the read tool against the real API on day one. A fake read tool hides the field you will trip over later.

What guardrails stop it from doing damage?

Guardrails go in three places, and all three are code, not sentences in a prompt. A prompt asks the model to behave. Code refuses to let it misbehave.

Before the model: identify the caller from the number, say at the start of the call that this is an AI, and rate-limit by number so one caller cannot run up the bill. After the model: check the drafted reply before it is spoken or sent. Around the tools: the allow-list per intent, a person in the loop for anything that costs money, and a timeout with a fallback line for when a tool does not answer.

The reply check is the one people skip. It is small.

type Draft = { intent: string; text: string };
type Verdict = { ok: true } | { ok: false; reason: string; sayInstead: string };

const PRICE = /\$\s?\d|\b\d+\s?(dollars|bucks)\b/i;
const COMMITMENT = /\b(guarantee|promise|definitely|for sure|we will be there at)\b/i;
const MAX_CHARS_FOR_VOICE = 320;

function trimToSentence(text: string, max: number): string {
  const cut = text.lastIndexOf(". ", max);
  return cut > 0 ? text.slice(0, cut + 1) : text.slice(0, max);
}

// `canQuote` lists the intents allowed to say a number, and only from the price book.
export function checkReply(draft: Draft, canQuote: ReadonlySet<string>): Verdict {
  if (PRICE.test(draft.text) && !canQuote.has(draft.intent)) {
    return {
      ok: false,
      reason: "price outside the price book",
      sayInstead: "I'll have the office confirm pricing before anyone quotes a number.",
    };
  }
  if (COMMITMENT.test(draft.text)) {
    return {
      ok: false,
      reason: "commitment language",
      sayInstead: "Let me get someone who can confirm that for you.",
    };
  }
  if (draft.text.length > MAX_CHARS_FOR_VOICE) {
    return {
      ok: false,
      reason: "too long to say out loud",
      sayInstead: trimToSentence(draft.text, MAX_CHARS_FOR_VOICE),
    };
  }
  return { ok: true };
}

const draft = { intent: "quote", text: "That would be about $450, guaranteed." };
console.log(checkReply(draft, new Set()));
// { ok: false, reason: "price outside the price book", sayInstead: "I'll have the office ..." }

Three checks: no price unless the intent is allowed to quote from a price book, no commitment language, and nothing too long to say out loud. When a check fails, the agent says the fallback line instead of the draft, and the failure is logged with the reason. Over a month, the log of reasons tells you which prompt to fix. Add checks as the log earns them; do not start with fifty.

How do I test it before it talks to a customer?

Build a test set from real transcripts. Redact names, numbers, and addresses; keep the words. Each case is four fields: the input, the intent you expect, the tool calls you expect with their arguments, and the things that must not appear in the reply. Run the whole set on every change to a prompt, a rule, or a model, and keep the score where the team can see it. A prompt change that moves the score down does not ship, however good it sounded.

Put the hard cases in on purpose. The caller who wants a discount. The caller who says "emergency" to jump the line. The caller who talks over the greeting. The caller who gives an address in a town you do not serve. The caller who asks whether they are talking to a person. Each of those has a right answer, and the right answer is a test case.

Then shadow mode. The agent listens to live calls beside your team, proposes what it would do, and a person decides. You compare the two for as long as it takes for the agent's proposals to match the person's decisions on the calls that matter. Only after that does it take a call alone, and it starts with the calls nobody was answering, the after-hours ones, where the alternative was voicemail.

How do I hook it to a phone line and SMS?

Phone first. Buy a number from a telephony provider; Twilio is one. Point the number at a webhook on your server. When a call comes in, the provider opens a media stream, a websocket that carries the caller's audio to you and your audio back. Your side runs speech to text on the incoming audio, runs the loop above on the text, and runs text to speech on the reply. A realtime voice model can take the audio directly and skip the two conversions; the loop, the router, and the reply check still live in your code, because that is where you can test them.

A transfer is the provider's transfer verb on the live call, with the summary from transferToHuman sent to the person before the call lands, so they pick up knowing who it is and what they want. Tell callers the call is recorded when the law where you operate requires it, and keep the recording where the guardrail log can point at it.

SMS is the same loop with a different front door. An inbound text hits a webhook, the loop runs, the reply goes out through the provider's messaging API. Keep the conversation state keyed by phone number with a timeout, so a text that arrives an hour later still has its context. Honor STOP and every other opt-out word and keep a record of consent. Keep replies short: a text is not a call, and a wall of text on a phone screen reads as spam.

What goes in the agent spec pack?

The written record of the five steps, one page each. The intent table: intent, slots, tool, exit. The tool contracts: name, input, output, which intents may call it. The guardrail list: rule, where it runs, what it says instead. The test set and its last score. The escalation matrix: which failure goes to whom, and how fast. The channel plan: numbers, hours, opt-out handling, recording notice. And one page for what you decided not to build yet.

The agent spec pack template arrives with the demo dashboard. Until it does, those seven headings are the template. An agent whose pack you can hand to another engineer is an agent you own. One that lives only in a prompt is a demo.