Error Handling, Retries and Timeouts

A demo works when everything goes right. A real product must also work when things go wrong, and in the real world they do go wrong: the network drops for a second, the service is busy, you hit a rate limit. In this lesson you learn to handle these problems calmly, so that a short problem does not become an angry user.

Why AI features fail

Calls to an AI service are slower and heavier than normal database calls, so more can go wrong. These are the usual problems.

• Your request is wrong. A parameter is invalid, the text is too long, or the history is not valid. The API replies with a 400-level error.
• Your key is wrong. A missing or invalid key gives 401, and a missing permission gives 403.
• You are going too fast. You exceeded a rate limit. The error status is 429.
• The service has a temporary problem. A 500-level error, or an "overloaded" message, means the problem is on the provider's side and will probably pass.
• The network has a problem. The connection drops or times out.

The golden rule: retry only what can heal

Not all errors deserve a retry. Ask yourself: if I send exactly the same request again, could it succeed? If your request is invalid, the answer is no. It will fail again, and again, and waste time. If the service was just busy, the answer is yes. So:

• Do retry: 429, 500-level errors, connection problems, timeouts.
• Do not retry: 400 (fix the request), 401 and 403 (fix the key or permission).

This small decision is one function in our lab, isRetryable().

Exponential backoff with jitter

When you do retry, do not retry instantly. If thousands of users all retry at the same moment, they overload the service again, and nothing improves. The standard solution is exponential backoff: wait a short time before the first retry, then twice as long before the second, then twice as long again. For example, wait 0.5 seconds, then 1 second, then 2 seconds.

Add jitter, a small random extra delay. Without it, all your clients that failed together would retry together in a neat wave. With a little randomness, the retries spread out. The formula is easy: wait = base × 2 to the power of (attempt − 1), plus a random amount.

Also set a maximum number of tries, such as three. Retrying forever just hides a real outage.

The SDK already retries

The official SDK retries a few kinds of errors by itself a couple of times, using a similar backoff. That is helpful in production. In this lesson we turn that off for one client (maxRetries: 0) so that you can see and control your own retry logic and learn how it works. In your real product you can choose either. Using the built-in retries is simpler, and writing your own gives you more control, for example to log each attempt.

Timeouts and cancellation

A request that never finishes is also a failure. Always set a timeout, a maximum time you are willing to wait. After that time, you stop waiting and handle it as an error. In the lab we give the client a 20-second timeout. On the user's side, give them a Cancel button. In Angular, if you subscribe to an HttpClient call, calling unsubscribe() on the subscription tells the browser to drop the request.

Friendly messages, safe details

When a request finally fails, the user needs a short, calm message and a clear next step, for example: "The AI service is busy right now. You can try again." Do not show raw error text. It can reveal internal details, and it means nothing to normal people. Write the full details to your server log, including the status code and a request id, so that you can investigate later. Tell the browser whether the error is retryable, so the page can show a "Try again" button only when it makes sense.

Graceful degradation

For important features, plan what happens when the AI is not available. Can the user continue without it? A resume builder should still let people edit and download their resume if the AI suggestion button fails. Some teams add a circuit breaker: after several failures in a row, stop calling the service for a short time instead of piling more requests on it. Others keep a second model or provider as a backup. You do not need all of these on day one, but you should decide on purpose, not by accident.

Testing errors without waiting for a real outage

You cannot wait for a real failure to test your error screen. In the lab, the route accepts a "simulate" setting with three fake modes. In flaky mode, the first two attempts fail with a 429 and the third succeeds, so you can watch the retry work. In down mode, every attempt fails with a 503, so you see the final error. In bad mode, the request fails with a 400, which is never retried. Adding such test switches to a development tool is a great habit, as long as you do not leave them active for real users in production.

Your task: build the Reliable page

Add a retry helper on the server and a page that shows retries, errors, a "Try again" button and a Cancel button.

What you will build

• server/retry.js with isRetryable() and withRetry().
• A POST /api/reliable-ask route that uses a client with maxRetries 0 and a 20-second timeout, supports the three fake modes, and returns the number of attempts.
• A Reliable page that shows the answer or the error, the number of attempts, a Try again button only for retryable errors, and a Cancel button.

Steps

1. Start from your lesson 5 project, or download the lesson 5 solution.
2. Write isRetryable() (429, 500 and above, connection errors) and withRetry() with three tries, doubling waits and jitter. Return both the value and the number of attempts.
3. Create the route. Inside the function you pass to withRetry, throw fake errors in the three modes, otherwise call Claude.
4. In the catch block, log the details, then reply with a friendly message, the retryable flag and the attempts.
5. In Angular, use this.http.post(...).subscribe(...) and keep the subscription so Cancel can unsubscribe.
6. Test all three modes, and test Cancel while a request is running.

Hints

• Remember to wait only between attempts, not after the last one.
• Store the number of attempts on the error object so the route can report it.
• Choose the HTTP status of your reply from the error, for example 503 for a service problem, and 400 for a bad request.

You know it works when flaky mode succeeds after 3 attempts, down mode shows a friendly error with a Try again button, bad mode shows an error without that button, and Cancel stops a running request.

Solution

The retry helper:

// Retry only the errors that can go away by themselves.
export const isRetryable = (e) =>
  e?.status === 429 || e?.status >= 500 || e?.name === 'APIConnectionError' || e?.name === 'APIConnectionTimeoutError';

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

// Run fn(). If it fails with a retryable error, wait and try again.
// Waits grow each time (exponential backoff) with a little randomness (jitter).
export async function withRetry(fn, { tries = 3, baseMs = 500 } = {}) {
  let attempts = 0;
  for (;;) {
    attempts += 1;
    try {
      return { value: await fn(attempts), attempts };
    } catch (e) {
      if (!isRetryable(e) || attempts >= tries) {
        e.attempts = attempts;
        throw e;
      }
      await sleep(baseMs * 2 ** (attempts - 1) + Math.random() * 200);
    }
  }
}

The route:

import { Router } from 'express';
import Anthropic from '@anthropic-ai/sdk';
import { MODEL, textOf } from '../claude.js';
import { withRetry, isRetryable } from '../retry.js';

const router = Router();

// This client does NOT retry by itself and gives up after 20 seconds,
// so you can see our own retry logic working.
const strict = new Anthropic({ maxRetries: 0, timeout: 20_000 });

const fake = (status) => Object.assign(new Error('simulated error'), { status });

// POST /api/reliable-ask  { question, simulate? }   simulate: 'flaky' | 'down' | 'bad'
router.post('/reliable-ask', async (req, res) => {
  const question = String(req.body?.question || '').trim().slice(0, 2000);
  const simulate = String(req.body?.simulate || '');
  if (!question) return res.status(400).json({ error: { message: 'Please type a question.' } });

  try {
    const { value, attempts } = await withRetry(async (attempt) => {
      // Fake failures so you can test without waiting for a real outage.
      if (simulate === 'flaky' && attempt < 3) throw fake(429);
      if (simulate === 'down') throw fake(503);
      if (simulate === 'bad') throw fake(400);

      return strict.messages.create({
        model: MODEL,
        max_tokens: 400,
        messages: [{ role: 'user', content: question }],
      });
    });
    res.json({ text: textOf(value), attempts });
  } catch (e) {
    console.error('[reliable]', e?.status, e?.message);
    const retryable = isRetryable(e);
    res.status(e?.status >= 400 && e?.status < 600 ? e.status : 502).json({
      error: {
        message: retryable ? 'The AI service is busy right now. You can try again.' : 'We could not process that request.',
        retryable,
        attempts: e.attempts ?? 1,
      },
    });
  }
});

export default router;

The Angular page:

import { Component, inject, signal } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpClient } from '@angular/common/http';
import { Subscription } from 'rxjs';

@Component({
  selector: 'app-reliable',
  imports: [FormsModule],
  template: `
    <h1>Reliable requests</h1>
    <p class="hint">The server retries temporary errors. The page shows a clear message, lets you try again, and lets you cancel.</p>

    <div class="card">
      <label for="q">Question</label>
      <textarea id="q" [(ngModel)]="question" style="min-height:70px"></textarea>

      <label for="sim">Test mode</label>
      <select id="sim" [(ngModel)]="simulate">
        <option value="">Real request</option>
        <option value="flaky">Fake: fails twice, then works (retry succeeds)</option>
        <option value="down">Fake: service is down (all retries fail)</option>
        <option value="bad">Fake: bad request (never retried)</option>
      </select>

      <button (click)="send()" [disabled]="busy() || !question.trim()">{{ busy() ? 'Working...' : 'Send' }}</button>
      <button class="secondary" (click)="cancel()" [disabled]="!busy()">Cancel</button>

      @if (error(); as e) {
        <div class="err">
          {{ e.message }} (attempts: {{ e.attempts }})
          @if (e.retryable) { <button class="secondary" (click)="send()">Try again</button> }
        </div>
      }
      @if (text()) {
        <div class="answer">{{ text() }}</div>
        <p class="muted">Worked after {{ attempts() }} attempt(s).</p>
      }
    </div>
  `,
})
export class ReliablePage {
  private http = inject(HttpClient);
  private sub: Subscription | null = null;

  question = 'Give me one tip for a strong resume headline.';
  simulate = 'flaky';
  busy = signal(false);
  text = signal('');
  attempts = signal(0);
  error = signal<{ message: string; retryable: boolean; attempts: number } | null>(null);

  send() {
    this.busy.set(true);
    this.error.set(null);
    this.text.set('');
    this.sub = this.http
      .post<{ text: string; attempts: number }>('/api/reliable-ask', { question: this.question, simulate: this.simulate })
      .subscribe({
        next: (r) => { this.text.set(r.text); this.attempts.set(r.attempts); this.busy.set(false); },
        error: (e) => {
          this.error.set(e?.error?.error ?? { message: 'Something went wrong.', retryable: true, attempts: 1 });
          this.busy.set(false);
        },
      });
  }

  cancel() {
    this.sub?.unsubscribe(); // the browser drops the request
    this.busy.set(false);
  }
}

Download the lesson 6 solution (zip)

What you learned

• Retry only errors that can heal: 429, server errors and connection problems.
• Use exponential backoff with jitter, and a maximum number of tries.
• Set timeouts, allow cancelling, and show friendly messages while logging the details.
• Test failures on purpose with fake error modes.

In lesson 7 we look at a way to make repeated requests cheaper and faster: prompt caching.