AI Literacy for the LLM Age

Most talk about large language models (LLMs) is about writing: essays, code, emails produced on demand. But most of what an LLM actually does is read. This site starts by teaching you to read them, and works up to writing them.

We won't collect training data or train any models. You don't need to understand neural networks, GPUs, or data centers. You do need a working picture of the artifact in front of you, the same way a good reader of newspapers knows something about editors, wire services and advertisers without ever running a press.

By the end you will be able to write AI agents that learn from experience, share that knowledge with other people's agents, and run models on devices you own.

Reading, writing, and counting

UNESCO defines literacy as "the ability to identify, understand, interpret, create, communicate and compute, using printed and written materials associated with varying contexts." Note the last verb. Literacy has always included numeracy, and AI literacy needs it more than most: tokens get counted, models get measured in billions, bills arrive per million, and the claims you'll hear about any of these are only as good as your ability to check them.

So every exercise on this site asks for all three, in small doses. You read a situation, you write something (sometimes a sentence, sometimes just a choice made by clicking), and you count: you make a rough numerical judgment and see how close you came. Badges on each exercise show where each happens.

How this site works

Three threads run through every scene.

• Teach: direct explanation, like this paragraph.
• Work: you commit to a prediction, which unlocks an explorable. To continue in order you have to work it. The contents menu lets you jump anywhere.
• Argue: a running argument about what AI literacy should be, in the margin. It starts with you as a consumer and ends with you as a platform operator. It will interrupt you shortly, and like the other threads it will ask something of you before you continue.

LLMs don't see letters or words. They see tokens: words or pieces of words, each identified by a number from a closed vocabulary. Spaces and punctuation are often built into the tokens themselves, so " cat" (with its leading space) and "cat" can be different tokens.

Different models use different vocabularies. A vocabulary tuned mostly on English may cut a long German compound word, or a line of Chinese, into many more pieces than an English sentence of the same meaning. So the same idea can cost different numbers of tokens in different languages.

Tokens are not money

You often pay per token, so it is tempting to think of tokens as a currency. They aren't: they are a unit of counting, like words or pages. Reading a word and writing a word take different effort, for people and for computers. The same is true of tokens: processing input tokens (reading) costs differently from generating output tokens (writing).

A language model answers one question: given the surrounding tokens, which token goes here? Its answer is not one token but a distribution over the whole vocabulary. That's different from predicting the single most likely token: the model has to represent ambiguity and keep several readings of the context in play at once.

Most models you meet today only predict the next token at the end of a sequence, in reading order. These are called causal language models. That's why long replies stream out as if someone were typing them.

The sampler

The model by itself only outputs scores. To produce text, a second process, the sampler, picks a token from the distribution, adds it to the context, and asks again. A low temperature makes it nearly always pick the top choice; a higher one gives the long tail a chance.

The prompt and the context mean nearly the same thing: the text that shapes what gets sampled next.

Language models are trained to predict tokens from context. You can build a tiny one by scanning a big text and counting how often each token follows each other token. With a vocabulary of 100 tokens, that model has 100 × 100 = 10,000 numbers, called parameters, no matter how much text you trained it on.

Today's LLMs have billions to trillions of parameters. At roughly one byte per parameter, that's gigabytes to terabytes. Many models fit on a laptop or even a phone; whether those devices can run them quickly is a separate question.

A rule of thumb

One rule of thumb in model design says to train on about 20 tokens for every parameter. Many modern models are trained on far more than that. If a textbook page holds about 1,000 words, you can estimate how many pages of text are encoded in a model of a given size.

A model trained only on general text (Wikipedia pages, product reviews, video transcripts) ends up encoding a great deal of human culture. It does not automatically know the roles of User and Assistant.

Chatbots are trained in at least two phases: a long pre-training phase on general text, and a shorter post-training phase on examples of a good Assistant answering User requests. A model with only pre-training is called a base or foundation model. It just keeps writing more text in the style of whatever is in its context.

You can chat with a base model if you fill its context with a few examples of User/Assistant turns. It will continue the pattern, but it may also break the format, forget to stop, or wander off topic. A heavily post-trained model stays on topic and takes turns reliably, but picks up quirks: favorite words, favorite punctuation, a habit of flattering you.

To make chatbots more useful, they're trained to write blocks of text between special marker tokens before giving a final answer. In effect, the model talks to itself before it talks to you, which lets it work through arithmetic or other step-by-step problems. This is called reasoning.

In the same way, a chatbot can be trained to write a tool call: a description of an action, between its own markers. When the sampler sees a finished tool call, it carries it out (searching the web, reading a file), puts the result into the context, and lets generation continue.

A chatbot that can use tools on its own to reach a goal is an agent of its user. It can do things with real consequences, from an interface that feels like harmless talking.

With reasoning and tools, the number of tokens a conversation uses stops matching the text you see on screen. One short request can set off a long chain of tool calls, each followed by a long block of the agent thinking through the result.

When a model processes input tokens to decide what to write next, you can say it is reading its inputs and writing its outputs. For people, reading is usually easier than writing, and re-reading something familiar is easier than reading something new. Re-reading what you just wrote takes almost no effort.

Inference engines have the same trick, called caching: input tokens are processed once and then reused cheaply, at the cost of storage space. Providers charge much less for cached input than for fresh input, and much less for either than for output.

Back to the prologue

In agentic sessions, the large majority of tokens are cached input. Mostly, the model is interpreting a context that you, your files, your tools and its own earlier turns have written. Generating new text is almost a side activity.

When people use the ChatGPT, Claude or Gemini apps, it feels like using a single thing. The 1990s online services worked the same way: CompuServe, Prodigy and America Online. You paid a monthly fee and got an icon to double-click. Each service bundled client software (what we'd now call a browser), network access (an ISP), and content (something like a social platform) into one walled garden.

Today we mix interchangeable browsers, interchangeable internet providers, and countless sites. Chat apps bundle three layers in the same way: a client (an app or website), an inference provider (whoever runs the model), and a model.

The same unbundling is under way for LLMs. Clients such as OpenChamber (a desktop app) and Open WebUI (a hosted chat interface) can connect to many providers. Marketplaces such as OpenRouter sell access to many providers, and those providers offer overlapping catalogs of open-weight models. You can run a model from one lab through a different company's servers, using a client made by someone else.

UC Santa Cruz's BayLeaf lets you try different clients, providers and models. It doesn't offer a UCSC-specific model. Instead, it makes it easy to combine campus-specific knowledge with existing models to build campus-specific agents.

A harness is the software that gives a model its tools and runs the loop. Common tools read and write files and run commands. At one extreme, a harness offers a single tool, bash, and the agent does everything through the command line. Such agents can run for a long time and get a lot done, but anyone who doesn't read code will struggle to follow what they're doing.

Tools named read, write, edit, search, fetch and load_skill produce chains of calls that tell a story. Computer-use tools such as click, type and screenshot make it easy to picture how an agent is using your computer, or a remote sandbox.

Self-extension: even a bash-only harness lets an agent extend itself. It can write a new command-line tool, then run it.

Just as apps let you work on big projects and communicate with others, a harness is what lets an agent do the same. Even a simple chatbot has a harness with a single tool: send_reply_to_user.

An agent is a model plus a harness, and it also matters what the harness is connected to. Many harnesses support AGENTS.md: a plain text file, literally named that, in a project folder. Its contents are shown to the agent when a session starts. Without changing anything else, you can give your agent a new personality, new initiative, or new background knowledge. You don't need to be a big AI company to author an agent; you just write in the file what you would otherwise put in your first message.

Agent Skills are also plain text, in files named SKILL.md, with one special part: a description. The agent sees the descriptions up front and loads a skill's full text only when a description matches the task in hand. Think of a reference handbook: the agent skims the table of contents and opens the chapters that look useful.

Most harnesses let agents edit their own AGENTS.md and SKILL.md files. You can often author one just by saying: "I like what we just figured out. Let's make a skill for it so it's easier next time." Because these are plain text (lightly formatted with Markdown, hence ".md"), you can audit and edit them yourself.

Most models and harnesses are general-purpose. Which provider you use matters mostly for cost and privacy. So sharing an agent usually comes down to sharing some files. The other person doesn't need your client, harness, provider or model. Their agent just needs to follow your advice.

The most portable way to share a file is the web. Put an AGENTS.md or a few SKILL.md files at a URL (many people use GitHub), and their agent can usually work out how to install them.

robots.txt, llms.txt, openapi.json

Around 1994, web developers agreed on robots.txt, a file telling automated visitors (mostly search crawlers) how to behave on a site. In 2024 a related convention appeared: llms.txt, a plain-text summary of a site for LLM agents, which otherwise have to dig through the heavy markup built for human eyes. Compare llmstxt.org/llms.txt with llmstxt.org. An llms.txt works like an AGENTS.md for a website.

A site with an API may point to an openapi.json: a formal description that a program can use directly, with no LLM needed. For example, api.bayleaf.dev/llms.txt links to its openapi.json, the machine-readable version of its documentation. With that file, your agent can often build its own tools for the service. Machines usually can't log in with an email and password, so expect your agent to ask for an API key.

Sharing on purpose is one thing. There is also sharing you didn't intend, and even authorship you didn't intend.

Many first-party providers (companies that train their own models) log your activity, and some use it as training data for future models. A casual chat can end up feeding someone else's project of writing their next AI.

Two terms

Zero data retention (ZDR) is a retention promise: the provider doesn't store your content at all, which is stronger than deleting it after, say, 30 days. Even without training on it, retained data can leak; ZDR shrinks that window to nothing. One wrinkle: a ZDR provider may still analyze your data as it passes through, or keep metadata about it.

Zero operator access (ZOA) is an access property: the operator has no technical means to read your content, even while it is passing through. A ZDR provider could turn into a data-retaining one overnight. ZOA designs make that switch hard, often by using confidential computing with hardware that can prove what code it is running.

BayLeaf as an honest example: its API keeps no prompt or completion content, which is a strong ZOA posture but not a hardware-guaranteed one. Its Sealed lane encrypts requests to an attested enclave, so BayLeaf only carries ciphertext. BayLeaf Chat deliberately stores your conversation history, so it can't be ZOA.

Everything an agent reads lands in the same context: your request, the web pages it fetches, the skills it installs. The model has no reliable way to tell "information I found" from "instructions I should follow." A sentence buried in a fetched page that reads like an instruction can steer your agent. This is called prompt injection.

That's the other side of the previous act. If anyone can publish a SKILL.md, then installing one from a URL is a bit like running a program from a stranger, except the program is written in plain language.

Habits that help

• Read a skill or AGENTS.md yourself before installing it. It's plain text, so you can.
• Give agents only the tools and access a task needs.
• Keep a person in the loop for actions that are hard to undo or that send things out into the world.
• Be most wary when an agent has all three at once: private data, untrusted content, and a way to send things out.

So far we've treated models as interchangeable readers of text, but some are much bigger than others. A two-trillion-parameter model in a data center can surely do things a two-billion-parameter model on your phone can't. What does the bigger one buy you, and what does it cost?

Bigger models generally cost more to use, roughly in proportion to size. Output tokens can differ in price by a factor of 1,000 between models. (Fine print: cost tracks active parameters more than total parameters, faster service costs more, and some providers charge less because they run lower-precision versions.)

To anthropomorphize for a moment: bigger models tend to know more, pay better attention, and use tools more wisely. You can measure knowledge with standardized exams, attention by asking for a detail hidden in a long document, and wisdom by counting how many tool-calling steps an agent needs for a task with a checkable answer. Oddly, a wise model with expensive output tokens can turn out cheaper overall, because it writes fewer tokens to get the job done.

Beyond money, training and running models takes energy, data and hardware. Bigger models take more of all three.

Remember the arithmetic from Act I: many capable models fit on hardware you already have. There are two everyday ways to run one.

• In the browser. Libraries such as Transformers.js run small models right in a web page using your graphics chip (WebGPU). The model's work stays on your device, though the page's own code (which can change when it reloads), your browser extensions and your operating system are still part of the picture.
• On localhost. Apps such as Ollama, LM Studio and oMLX run a model on your own computer and serve it at localhost, using the same kind of API that commercial providers offer. Any client that lets you set the provider address can use it.

BayLeaf's experimental Pouch brings these together. It's a single-page coding agent that can think through a model in your browser, a server on your own machine, or BayLeaf's Sealed lane, which BayLeaf itself can't read. It keeps your files and sessions on your device. It names its trust roots out loud: BayLeaf, your browser, and your own machine.

What changes

When you run inference yourself, you become the provider. No outside provider retains your prompts, as long as everything in the path really is local. Nobody else can switch your model unless you let them. You're also the one who keeps it running.

The final test isn't a quiz. You write something that someone else's agent can read and use, even though they run a different client, provider and model.

1. Write a SKILL.md that captures something you know how to do, for a group you belong to.
2. Publish it at a URL, along with a short llms.txt that says what it's for.
3. Give the URL to someone else. Watch their agent install and use it, and see where it gets your meaning wrong.
4. Revise. If you like, give your group some shared, live state too (BayLeaf Rooms is built for this), so their agents and your static pages can work together.

If you did that, you wrote something other people's agents could read. That's this site's definition of literacy. Notice that this site cannot check it for you: the other person's agent is the examiner.

To be radicalized, in the old sense, is to be changed at the root: not to take up an extreme, but to understand something so thoroughly that your relation to it changes.

You started this site as someone who uses AI products. Along the way you predicted, worked, counted and wrote. Below is the record of where you stood at each step of the argument, kept only in this browser. Go back and change any answer you've outgrown. The record is yours to revise.