Prompt Caching and Batch Processing

Imagine you run a resume review service. Every review starts with the same long set of rules: how to write a summary, how to word bullet points, what to avoid. Only the resume changes. If you send those rules again and again for every user, you pay to process the same text every time. Prompt caching is a feature that fixes exactly this. In this lesson you will learn how it works, and you will build a page that shows it working.

The problem: the same text, again and again

Remember from lesson 3 that you pay for every input token. A long system prompt of a thousand words might be fine once, but if one hundred users run a review in an hour, you send and pay for that long text one hundred times, even though it never changed. It also takes Claude a little time to read it each time. This wastes both money and seconds.

Think of a teacher who must explain the same classroom rules to every new student. Instead of reading the full rulebook out loud each time, the teacher could keep a copy that every student can look at. Caching is like that copy.

How prompt caching works

With prompt caching, you tell the API: "this first part of my prompt will not change, so please remember it." The first time, Claude processes that part and stores it for a short time. This first call is called a cache write. On later calls that start with exactly the same text, Claude reads it from storage instead of processing it again. This is a cache read. Cache reads are cheaper than normal input tokens and are usually faster. Cache writes can cost a little more than normal input, so caching pays off when the same prefix is used several times. The exact price rules are in the official documentation.

You mark the part to cache with a small setting called cache_control. Here is how our lab does it.

const reply = await client.messages.create({
  model: MODEL,
  max_tokens: 500,
  // stable text FIRST, marked for caching
  system: [{ type: 'text', text: RULES, cache_control: { type: 'ephemeral' } }],
  // changing text LAST
  messages: [{ role: 'user', content: '<resume>' + resume + '</resume> Review this resume.' }],
});

Notice that "system" is now a list of blocks instead of a plain string. That is how you attach the cache_control setting to a block. The word "ephemeral" means the cache lasts only a short time. If nobody uses it for a few minutes, it expires, and the next call writes it again. Busy features keep refreshing it, so they enjoy cache reads all day.

The most important rule: stable first, changing last

Caching works on the beginning of your prompt. The cache is used only when the start of the new request is exactly the same as the cached start. So you must put the parts that never change at the beginning, and the parts that change at the end. A common mistake is to put the user's name or the current date at the top of the system prompt. Because that text differs every time, nothing matches, and the cache is never used. Keep the top clean and stable. Put anything that changes, like the user's resume, after it.

Minimum size and other limits

Caching is meant for large, repeated text, so there is a minimum size for the cached part. If your text is too short, the cache setting is simply ignored, and you will see zero cache tokens. That is not an error, it just means there was nothing worth caching. The minimum can differ between models, so check the documentation. In our lab, the file server/data/resume-rules.txt contains about a thousand words of useful resume-review rules. If you still see zeros, make the rules text longer. You should also know that the cache is kept for a short time and that only exact matches count: changing even one character in the cached part creates a new cache entry.

Reading the numbers

The usage object tells you what happened. Besides the normal input and output tokens, it can include two extra numbers.

• cache_creation_input_tokens: how many tokens were written to the cache in this call.
• cache_read_input_tokens: how many tokens were read from the cache in this call.

Your first call should show a cache write and your second call, within a few minutes, should show a cache read of the same size. If both stay zero, check the minimum size and the order of your prompt. Showing these numbers on a dashboard is the best way to be sure that your caching works. You should log them in a real product, so you can see how much you save.

Batch processing: when the answer can wait

Caching is for requests that users are waiting for. Another feature, the Message Batches API, is for work that can wait. You submit a large group of requests at once, and Anthropic processes them in the background. You collect the results later, usually at a lower price than normal requests. The answers are not instant: a batch finishes within a time window, not in a second.

Good uses for batches are tasks with many items and no hurry: scoring ten thousand stored resumes overnight, writing descriptions for a whole catalogue, or running a big test of a new prompt. A bad use is anything a person is waiting for on screen. You can combine batches and caching, because both reduce cost in different ways.

• Same long prefix, many live requests: prompt caching.
• Many independent jobs, no hurry: batches.
• Both together: possible, for large offline jobs with a shared long prefix.

We will practise caching in the lab, because it works with the normal request you already know. Batches use a separate set of endpoints and a "submit, wait, collect" workflow, which you can add to your own project later using the documentation.

Caching is not magic

Before you rely on caching, test it. Measure the input token counts with and without it, and check the cost. Caching will not help a feature that is called only once a day, or where every prompt is different. For such cases, shorter prompts and a smaller model are better ways to save money.

Your task: build the Caching page

Add a page that reviews a resume using a long, cached rules prompt, and shows a table of token usage including cache writes and cache reads for each call.

What you will build

• A data file server/data/resume-rules.txt with at least a thousand words of review rules (the zip has a ready-made one; try writing your own version first).
• A POST /api/cached route that sends the rules as a cached system block and the resume as the user message.
• A Caching page with a resume text box, a "Review resume" button and a table with columns for normal input, cache write, cache read and output.

Steps

1. Start from your lesson 6 project, or download the lesson 6 solution.
2. Write the rules text. Include clear rules about honesty, structure, the summary, bullet points and the format of the answer.
3. Read the file once when the server starts (fs.readFileSync) and keep it in a constant. Do not read it again for every request.
4. Create the route. Put the rules in a system block with cache_control. Wrap the resume in <resume> tags in the user message.
5. Return the usage in a simple shape: input, output, cacheWrite and cacheRead (use 0 when a value is missing).
6. In Angular, append each response's usage to a list and show it as a table. Highlight non-zero cache values in green.
7. Press the button twice, quickly. Check that the first call has a cache write and the second a cache read. Then wait several minutes and try again to see the cache expire.

Hints

• The usage fields may be missing when caching is not used, so use "?? 0".
• If the numbers are always zero, check that the rules text is long enough and that nothing at the top of the prompt changes between calls.
• Do not cache the resume. It is different every time.

You know it works when the second call shows a large cache read and a small normal input count, and when you can explain in one sentence why putting the resume first would break the cache.

Solution

The route (the rules file is in the zip, in server/data):

import { Router } from 'express';
import fs from 'node:fs';
import { client, MODEL, textOf } from '../claude.js';

const router = Router();

// The long, never-changing rules. Caching works best on big, stable text.
const RULES = fs.readFileSync(new URL('../data/resume-rules.txt', import.meta.url), 'utf8');

// POST /api/cached  { resume }
router.post('/cached', async (req, res) => {
  const resume = String(req.body?.resume || '').trim().slice(0, 12000);
  if (!resume) return res.status(400).json({ error: { message: 'Please paste a resume.' } });

  const reply = await client.messages.create({
    model: MODEL,
    max_tokens: 500,
    // Stable text FIRST, marked for caching. The changing resume goes in the user message.
    system: [{ type: 'text', text: RULES, cache_control: { type: 'ephemeral' } }],
    messages: [{ role: 'user', content: `<resume>\n${resume}\n</resume>\nReview this resume.` }],
  });

  const u = reply.usage;
  res.json({
    text: textOf(reply),
    usage: {
      input: u.input_tokens,
      output: u.output_tokens,
      cacheWrite: u.cache_creation_input_tokens ?? 0, // tokens saved into the cache
      cacheRead: u.cache_read_input_tokens ?? 0,      // tokens read back from the cache
    },
  });
});

export default router;

The Angular page:

import { Component, inject, signal } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpClient } from '@angular/common/http';
import { firstValueFrom } from 'rxjs';
import { errorText } from '../util';

type Row = { input: number; output: number; cacheWrite: number; cacheRead: number };

@Component({
  selector: 'app-caching',
  imports: [FormsModule],
  template: `
    <h1>Prompt caching</h1>
    <p class="hint">Run the same check twice within a few minutes. The first call writes the long rules into the cache. The second reads them back at a lower cost.</p>

    <div class="card">
      <label for="r">Resume text</label>
      <textarea id="r" [(ngModel)]="resume"></textarea>
      <button (click)="run()" [disabled]="busy() || !resume.trim()">{{ busy() ? 'Working...' : 'Review resume' }}</button>
      @if (error()) { <div class="err">{{ error() }}</div> }
    </div>

    @if (rows().length) {
      <div class="card">
        <h2>Token usage per call</h2>
        <table>
          <thead><tr><th>Call</th><th>Normal input</th><th>Cache write</th><th>Cache read</th><th>Output</th></tr></thead>
          <tbody>
            @for (r of rows(); track $index) {
              <tr>
                <td>{{ $index + 1 }}</td><td>{{ r.input }}</td>
                <td [class.ok]="r.cacheWrite > 0">{{ r.cacheWrite }}</td>
                <td [class.ok]="r.cacheRead > 0">{{ r.cacheRead }}</td>
                <td>{{ r.output }}</td>
              </tr>
            }
          </tbody>
        </table>
        <p class="muted">If both cache columns stay at 0, the rules text may be under the minimum size for caching on your model. Check the docs.</p>
        <div class="answer">{{ lastText() }}</div>
      </div>
    }
  `,
})
export class CachingPage {
  private http = inject(HttpClient);

  resume = 'Ravi Kumar, Pune. Electrician, 5 years. Installed wiring in houses and small shops. ITI certificate 2019. Skills: wiring, panel work, testing. Languages: Hindi, English.';
  busy = signal(false);
  error = signal('');
  rows = signal<Row[]>([]);
  lastText = signal('');

  async run() {
    this.busy.set(true);
    this.error.set('');
    try {
      const r = await firstValueFrom(this.http.post<{ text: string; usage: Row }>('/api/cached', { resume: this.resume }));
      this.rows.update((list) => [...list, r.usage]);
      this.lastText.set(r.text);
    } catch (e) {
      this.error.set(errorText(e));
    } finally {
      this.busy.set(false);
    }
  }
}

Download the lesson 7 solution (zip)

What you learned

• Prompt caching reuses a repeated start of your prompt, so repeated calls are cheaper and often faster.
• Put stable text first and changing text last, and make the cached part long enough.
• Check cache_creation_input_tokens and cache_read_input_tokens to confirm it works.
• Use the Batches API for large jobs that can wait.

In lesson 8 Claude learns to read PDFs and images.